2024-04-25T10:23:52.233 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T10:23:52.233 DEBUG:teuthology.run:Teuthology command: teuthology --name pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi --description fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_dirfrags/no prefetch_dirfrags/yes prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mon 2-workunit/suites/iozone}} --verbose --owner scheduled_pdonnell@teuthology --archive /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673079 -- /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673079/orig.config.yaml 2024-04-25T10:23:52.288 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T10:23:52.387 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673079 branch: wip-pdonnell-testing-20240425.015853-debug description: fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_dirfrags/no prefetch_dirfrags/yes prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mon 2-workunit/suites/iozone}} email: pdonnell@redhat.com first_in_suite: false job_id: '7673079' kernel: kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20240425.015853-debug ceph: cephfs: max_mds: 5 session_timeout: 300 conf: client: client mount timeout: 600 client_shutdown_timeout: 120 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds oft prefetch dirfrags: true mds op complaint time: 180 mds verify scatter: true mds_dir_prefetch: true mds_sleep_rank_change: 5000000.0 mon client directed command retry: 5 ms inject delay max: 1 ms inject delay probability: 0.005 ms inject delay type: client mds osd ms inject socket failures: 2500 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon client directed command retry: 5 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 ms inject delay max: 1 ms inject delay probability: 0.005 ms inject delay type: client mds ms inject socket failures: 2500 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - mons down - mon down - out of quorum - overall HEALTH_ - \(MON_DOWN\) sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs check-counter: counters: mds: - mds.exported - mds.imported install: ceph: flavor: default sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240425.015853-debug sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 owner: scheduled_pdonnell@teuthology priority: 85 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - mds.e - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mon.c - mgr.x - mds.b - mds.d - mds.f - osd.4 - osd.5 - osd.6 - osd.7 seed: 1880 sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 sleep_before_teardown: 0 subset: 178/512 suite: fs suite_branch: wip-pdonnell-testing-20240425.015853-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi106.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBETD64HyMKMQLS3kViFb0IHUZamIJNGGh6L6TVEKRWbRk8LgIQs3RtuGal8vlw4QMBYO4JewP8pqw0tojZ11xo= smithi134.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDLZPaQIpOulwMGfS8SDt9ONjvD4gK6qOuiIK2YqaJES2XKgT+OCIlH7sU/VFic6a0EkZg26PLttrOZfoV+6Mxc= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - ceph-fuse: null - mon_thrash: check_mds_failover: true revive_delay: 20 thrash_delay: 10 - workunit: clients: all: - suites/iozone.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-25_10:00:23 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-25T10:23:52.387 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa; will attempt to use it 2024-04-25T10:23:52.388 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa/tasks 2024-04-25T10:23:52.388 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T10:23:52.390 INFO:teuthology.task.internal:Checking packages... 2024-04-25T10:23:52.413 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '994e2f20102a20fa9a27966c14b625cc01884df8' 2024-04-25T10:23:52.413 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T10:23:52.413 INFO:teuthology.packaging:ref: None 2024-04-25T10:23:52.413 INFO:teuthology.packaging:tag: None 2024-04-25T10:23:52.413 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240425.015853-debug 2024-04-25T10:23:52.413 INFO:teuthology.packaging:sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T10:23:52.414 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20240425.015853-debug 2024-04-25T10:23:52.673 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3291.g994e2f20 2024-04-25T10:23:52.675 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T10:23:52.681 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T10:23:52.681 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T10:23:52.695 INFO:teuthology.task.internal:Saving configuration 2024-04-25T10:23:52.715 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T10:23:52.722 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T10:23:52.747 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi106.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673079', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 10:18:46.643976', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBETD64HyMKMQLS3kViFb0IHUZamIJNGGh6L6TVEKRWbRk8LgIQs3RtuGal8vlw4QMBYO4JewP8pqw0tojZ11xo='} 2024-04-25T10:23:52.773 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi134.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673079', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 10:18:46.642096', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDLZPaQIpOulwMGfS8SDt9ONjvD4gK6qOuiIK2YqaJES2XKgT+OCIlH7sU/VFic6a0EkZg26PLttrOZfoV+6Mxc='} 2024-04-25T10:23:52.773 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T10:23:52.779 INFO:teuthology.task.internal:roles: ubuntu@smithi106.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2024-04-25T10:23:52.779 INFO:teuthology.task.internal:roles: ubuntu@smithi134.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2024-04-25T10:23:52.779 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T10:23:52.854 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f6f6a142f70>, signals=[15]) 2024-04-25T10:23:52.854 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T10:23:52.860 INFO:teuthology.task.internal:Opening connections... 2024-04-25T10:23:52.860 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi106.front.sepia.ceph.com 2024-04-25T10:23:52.862 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T10:23:52.942 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi134.front.sepia.ceph.com 2024-04-25T10:23:52.943 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T10:23:53.019 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T10:23:53.025 DEBUG:teuthology.orchestra.run.smithi106:> uname -m 2024-04-25T10:23:53.043 INFO:teuthology.orchestra.run.smithi106.stdout:x86_64 2024-04-25T10:23:53.044 DEBUG:teuthology.orchestra.run.smithi106:> cat /etc/os-release 2024-04-25T10:23:53.101 INFO:teuthology.orchestra.run.smithi106.stdout:NAME="CentOS Stream" 2024-04-25T10:23:53.101 INFO:teuthology.orchestra.run.smithi106.stdout:VERSION="9" 2024-04-25T10:23:53.101 INFO:teuthology.orchestra.run.smithi106.stdout:ID="centos" 2024-04-25T10:23:53.101 INFO:teuthology.orchestra.run.smithi106.stdout:ID_LIKE="rhel fedora" 2024-04-25T10:23:53.101 INFO:teuthology.orchestra.run.smithi106.stdout:VERSION_ID="9" 2024-04-25T10:23:53.102 INFO:teuthology.orchestra.run.smithi106.stdout:PLATFORM_ID="platform:el9" 2024-04-25T10:23:53.102 INFO:teuthology.orchestra.run.smithi106.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T10:23:53.102 INFO:teuthology.orchestra.run.smithi106.stdout:ANSI_COLOR="0;31" 2024-04-25T10:23:53.102 INFO:teuthology.orchestra.run.smithi106.stdout:LOGO="fedora-logo-icon" 2024-04-25T10:23:53.102 INFO:teuthology.orchestra.run.smithi106.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T10:23:53.102 INFO:teuthology.orchestra.run.smithi106.stdout:HOME_URL="https://centos.org/" 2024-04-25T10:23:53.103 INFO:teuthology.orchestra.run.smithi106.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T10:23:53.103 INFO:teuthology.orchestra.run.smithi106.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T10:23:53.103 INFO:teuthology.orchestra.run.smithi106.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T10:23:53.104 INFO:teuthology.lock.ops:Updating smithi106.front.sepia.ceph.com on lock server 2024-04-25T10:23:53.130 DEBUG:teuthology.orchestra.run.smithi134:> uname -m 2024-04-25T10:23:53.149 INFO:teuthology.orchestra.run.smithi134.stdout:x86_64 2024-04-25T10:23:53.149 DEBUG:teuthology.orchestra.run.smithi134:> cat /etc/os-release 2024-04-25T10:23:53.206 INFO:teuthology.orchestra.run.smithi134.stdout:NAME="CentOS Stream" 2024-04-25T10:23:53.206 INFO:teuthology.orchestra.run.smithi134.stdout:VERSION="9" 2024-04-25T10:23:53.207 INFO:teuthology.orchestra.run.smithi134.stdout:ID="centos" 2024-04-25T10:23:53.207 INFO:teuthology.orchestra.run.smithi134.stdout:ID_LIKE="rhel fedora" 2024-04-25T10:23:53.207 INFO:teuthology.orchestra.run.smithi134.stdout:VERSION_ID="9" 2024-04-25T10:23:53.207 INFO:teuthology.orchestra.run.smithi134.stdout:PLATFORM_ID="platform:el9" 2024-04-25T10:23:53.207 INFO:teuthology.orchestra.run.smithi134.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T10:23:53.207 INFO:teuthology.orchestra.run.smithi134.stdout:ANSI_COLOR="0;31" 2024-04-25T10:23:53.207 INFO:teuthology.orchestra.run.smithi134.stdout:LOGO="fedora-logo-icon" 2024-04-25T10:23:53.207 INFO:teuthology.orchestra.run.smithi134.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T10:23:53.207 INFO:teuthology.orchestra.run.smithi134.stdout:HOME_URL="https://centos.org/" 2024-04-25T10:23:53.207 INFO:teuthology.orchestra.run.smithi134.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T10:23:53.207 INFO:teuthology.orchestra.run.smithi134.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T10:23:53.207 INFO:teuthology.orchestra.run.smithi134.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T10:23:53.208 INFO:teuthology.lock.ops:Updating smithi134.front.sepia.ceph.com on lock server 2024-04-25T10:23:53.229 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T10:23:53.238 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T10:23:53.243 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T10:23:53.244 DEBUG:teuthology.orchestra.run.smithi106:> test '!' -e /home/ubuntu/cephtest 2024-04-25T10:23:53.246 DEBUG:teuthology.orchestra.run.smithi134:> test '!' -e /home/ubuntu/cephtest 2024-04-25T10:23:53.262 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T10:23:53.268 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T10:23:53.268 DEBUG:teuthology.orchestra.run.smithi106:> test -z $(ls -A /var/lib/ceph) 2024-04-25T10:23:53.302 DEBUG:teuthology.orchestra.run.smithi134:> test -z $(ls -A /var/lib/ceph) 2024-04-25T10:23:53.393 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T10:23:53.441 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T10:23:53.455 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-04-25T10:23:53.456 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-04-25T10:23:53.456 DEBUG:teuthology.orchestra.run.smithi106:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T10:23:53.456 DEBUG:teuthology.orchestra.run.smithi134:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T10:23:53.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:23:53.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:23:53.476 DEBUG:teuthology.orchestra.run.smithi134:> uname -r 2024-04-25T10:23:53.476 DEBUG:teuthology.orchestra.run.smithi106:> uname -r 2024-04-25T10:23:53.533 INFO:teuthology.orchestra.run.smithi106.stdout:5.14.0-437.el9.x86_64 2024-04-25T10:23:53.533 INFO:teuthology.task.kernel:Running kernel on smithi106: 5.14.0-437.el9.x86_64 2024-04-25T10:23:53.534 DEBUG:teuthology.orchestra.run.smithi106:> sudo yum install -y kernel 2024-04-25T10:23:53.535 INFO:teuthology.orchestra.run.smithi134.stdout:5.14.0-437.el9.x86_64 2024-04-25T10:23:53.535 INFO:teuthology.task.kernel:Running kernel on smithi134: 5.14.0-437.el9.x86_64 2024-04-25T10:23:53.535 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum install -y kernel 2024-04-25T10:23:57.110 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS Stream 9 - BaseOS 4.2 MB/s | 8.0 MB 00:01 2024-04-25T10:23:57.438 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - BaseOS 3.5 MB/s | 8.0 MB 00:02 2024-04-25T10:23:59.247 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - AppStream 25 MB/s | 19 MB 00:00 2024-04-25T10:24:00.495 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS Stream 9 - AppStream 8.3 MB/s | 19 MB 00:02 2024-04-25T10:24:04.664 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - CRB 3.4 MB/s | 6.2 MB 00:01 2024-04-25T10:24:05.153 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS Stream 9 - CRB 5.7 MB/s | 6.2 MB 00:01 2024-04-25T10:24:06.589 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - Extras packages 27 kB/s | 16 kB 00:00 2024-04-25T10:24:07.196 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS Stream 9 - Extras packages 27 kB/s | 16 kB 00:00 2024-04-25T10:24:07.766 INFO:teuthology.orchestra.run.smithi134.stdout:Extra Packages for Enterprise Linux 20 MB/s | 21 MB 00:01 2024-04-25T10:24:08.542 INFO:teuthology.orchestra.run.smithi106.stdout:Extra Packages for Enterprise Linux 17 MB/s | 21 MB 00:01 2024-04-25T10:24:12.916 INFO:teuthology.orchestra.run.smithi134.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-04-25T10:24:13.572 INFO:teuthology.orchestra.run.smithi106.stdout:lab-extras 15 kB/s | 1.7 kB 00:00 2024-04-25T10:24:14.739 INFO:teuthology.orchestra.run.smithi134.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-25T10:24:14.739 INFO:teuthology.orchestra.run.smithi134.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-25T10:24:14.818 INFO:teuthology.orchestra.run.smithi134.stdout:Dependencies resolved. 2024-04-25T10:24:14.819 INFO:teuthology.orchestra.run.smithi134.stdout:Nothing to do. 2024-04-25T10:24:14.819 INFO:teuthology.orchestra.run.smithi134.stdout:Complete! 2024-04-25T10:24:14.946 DEBUG:teuthology.orchestra.run.smithi134:> echo no | sudo yum reinstall kernel || true 2024-04-25T10:24:15.437 INFO:teuthology.orchestra.run.smithi106.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-25T10:24:15.437 INFO:teuthology.orchestra.run.smithi106.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-25T10:24:15.451 INFO:teuthology.orchestra.run.smithi134.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 10:24:12 AM UTC. 2024-04-25T10:24:15.531 INFO:teuthology.orchestra.run.smithi106.stdout:Dependencies resolved. 2024-04-25T10:24:15.532 INFO:teuthology.orchestra.run.smithi106.stdout:Nothing to do. 2024-04-25T10:24:15.532 INFO:teuthology.orchestra.run.smithi106.stdout:Complete! 2024-04-25T10:24:15.570 INFO:teuthology.orchestra.run.smithi134.stdout:Dependencies resolved. 2024-04-25T10:24:15.587 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-04-25T10:24:15.588 INFO:teuthology.orchestra.run.smithi134.stdout: Package Architecture Version Repository Size 2024-04-25T10:24:15.588 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-04-25T10:24:15.588 INFO:teuthology.orchestra.run.smithi134.stdout:Reinstalling: 2024-04-25T10:24:15.588 INFO:teuthology.orchestra.run.smithi134.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T10:24:15.588 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T10:24:15.588 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction Summary 2024-04-25T10:24:15.588 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-04-25T10:24:15.588 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T10:24:15.588 INFO:teuthology.orchestra.run.smithi134.stdout:Total download size: 255 k 2024-04-25T10:24:15.588 INFO:teuthology.orchestra.run.smithi134.stdout:Installed size: 0 2024-04-25T10:24:15.588 INFO:teuthology.orchestra.run.smithi134.stderr:Operation aborted. 2024-04-25T10:24:15.652 INFO:teuthology.orchestra.run.smithi134.stdout:Is this ok [y/N]: 2024-04-25T10:24:15.652 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum reinstall -y kernel || true 2024-04-25T10:24:15.669 DEBUG:teuthology.orchestra.run.smithi106:> echo no | sudo yum reinstall kernel || true 2024-04-25T10:24:16.144 INFO:teuthology.orchestra.run.smithi134.stdout:Last metadata expiration check: 0:00:04 ago on Thu 25 Apr 2024 10:24:12 AM UTC. 2024-04-25T10:24:16.169 INFO:teuthology.orchestra.run.smithi106.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 10:24:13 AM UTC. 2024-04-25T10:24:16.260 INFO:teuthology.orchestra.run.smithi134.stdout:Dependencies resolved. 2024-04-25T10:24:16.261 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-04-25T10:24:16.261 INFO:teuthology.orchestra.run.smithi134.stdout: Package Architecture Version Repository Size 2024-04-25T10:24:16.261 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-04-25T10:24:16.261 INFO:teuthology.orchestra.run.smithi134.stdout:Reinstalling: 2024-04-25T10:24:16.261 INFO:teuthology.orchestra.run.smithi134.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T10:24:16.261 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T10:24:16.261 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction Summary 2024-04-25T10:24:16.261 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-04-25T10:24:16.262 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T10:24:16.262 INFO:teuthology.orchestra.run.smithi134.stdout:Total download size: 255 k 2024-04-25T10:24:16.262 INFO:teuthology.orchestra.run.smithi134.stdout:Installed size: 0 2024-04-25T10:24:16.262 INFO:teuthology.orchestra.run.smithi134.stdout:Downloading Packages: 2024-04-25T10:24:16.289 INFO:teuthology.orchestra.run.smithi106.stdout:Dependencies resolved. 2024-04-25T10:24:16.298 INFO:teuthology.orchestra.run.smithi106.stdout:================================================================================ 2024-04-25T10:24:16.299 INFO:teuthology.orchestra.run.smithi106.stdout: Package Architecture Version Repository Size 2024-04-25T10:24:16.299 INFO:teuthology.orchestra.run.smithi106.stdout:================================================================================ 2024-04-25T10:24:16.299 INFO:teuthology.orchestra.run.smithi106.stdout:Reinstalling: 2024-04-25T10:24:16.299 INFO:teuthology.orchestra.run.smithi106.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T10:24:16.299 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:24:16.299 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction Summary 2024-04-25T10:24:16.299 INFO:teuthology.orchestra.run.smithi106.stdout:================================================================================ 2024-04-25T10:24:16.299 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:24:16.299 INFO:teuthology.orchestra.run.smithi106.stdout:Total download size: 255 k 2024-04-25T10:24:16.300 INFO:teuthology.orchestra.run.smithi106.stdout:Installed size: 0 2024-04-25T10:24:16.300 INFO:teuthology.orchestra.run.smithi106.stderr:Operation aborted. 2024-04-25T10:24:16.376 INFO:teuthology.orchestra.run.smithi106.stdout:Is this ok [y/N]: 2024-04-25T10:24:16.376 DEBUG:teuthology.orchestra.run.smithi106:> sudo yum reinstall -y kernel || true 2024-04-25T10:24:16.864 INFO:teuthology.orchestra.run.smithi106.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 10:24:13 AM UTC. 2024-04-25T10:24:16.979 INFO:teuthology.orchestra.run.smithi106.stdout:Dependencies resolved. 2024-04-25T10:24:16.980 INFO:teuthology.orchestra.run.smithi106.stdout:================================================================================ 2024-04-25T10:24:16.980 INFO:teuthology.orchestra.run.smithi106.stdout: Package Architecture Version Repository Size 2024-04-25T10:24:16.980 INFO:teuthology.orchestra.run.smithi106.stdout:================================================================================ 2024-04-25T10:24:16.980 INFO:teuthology.orchestra.run.smithi106.stdout:Reinstalling: 2024-04-25T10:24:16.980 INFO:teuthology.orchestra.run.smithi106.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T10:24:16.980 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:24:16.980 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction Summary 2024-04-25T10:24:16.980 INFO:teuthology.orchestra.run.smithi106.stdout:================================================================================ 2024-04-25T10:24:16.980 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:24:16.981 INFO:teuthology.orchestra.run.smithi106.stdout:Total download size: 255 k 2024-04-25T10:24:16.981 INFO:teuthology.orchestra.run.smithi106.stdout:Installed size: 0 2024-04-25T10:24:16.981 INFO:teuthology.orchestra.run.smithi106.stdout:Downloading Packages: 2024-04-25T10:24:17.009 INFO:teuthology.orchestra.run.smithi134.stdout:kernel-5.14.0-437.el9.x86_64.rpm 538 kB/s | 255 kB 00:00 2024-04-25T10:24:17.009 INFO:teuthology.orchestra.run.smithi134.stdout:-------------------------------------------------------------------------------- 2024-04-25T10:24:17.010 INFO:teuthology.orchestra.run.smithi134.stdout:Total 341 kB/s | 255 kB 00:00 2024-04-25T10:24:17.027 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction check 2024-04-25T10:24:17.211 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction check succeeded. 2024-04-25T10:24:17.212 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction test 2024-04-25T10:24:17.216 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction test succeeded. 2024-04-25T10:24:17.217 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction 2024-04-25T10:24:17.357 INFO:teuthology.orchestra.run.smithi134.stdout: Preparing : 1/1 2024-04-25T10:24:17.382 INFO:teuthology.orchestra.run.smithi106.stdout:kernel-5.14.0-437.el9.x86_64.rpm 875 kB/s | 255 kB 00:00 2024-04-25T10:24:17.383 INFO:teuthology.orchestra.run.smithi106.stdout:-------------------------------------------------------------------------------- 2024-04-25T10:24:17.383 INFO:teuthology.orchestra.run.smithi106.stdout:Total 634 kB/s | 255 kB 00:00 2024-04-25T10:24:17.404 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction check 2024-04-25T10:24:17.435 INFO:teuthology.orchestra.run.smithi134.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T10:24:17.435 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T10:24:17.596 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction check succeeded. 2024-04-25T10:24:17.597 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction test 2024-04-25T10:24:17.602 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction test succeeded. 2024-04-25T10:24:17.602 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction 2024-04-25T10:24:17.744 INFO:teuthology.orchestra.run.smithi106.stdout: Preparing : 1/1 2024-04-25T10:24:17.825 INFO:teuthology.orchestra.run.smithi106.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T10:24:17.825 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T10:24:18.149 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T10:24:18.149 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T10:24:18.587 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T10:24:18.587 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T10:24:18.840 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T10:24:18.840 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T10:24:18.841 INFO:teuthology.orchestra.run.smithi134.stdout:Reinstalled: 2024-04-25T10:24:18.841 INFO:teuthology.orchestra.run.smithi134.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-25T10:24:18.841 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T10:24:18.841 INFO:teuthology.orchestra.run.smithi134.stdout:Complete! 2024-04-25T10:24:19.055 DEBUG:teuthology.orchestra.run.smithi134:> rpm -q kernel | sort -rV | head -n 1 2024-04-25T10:24:19.101 INFO:teuthology.orchestra.run.smithi134.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-25T10:24:19.102 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-25T10:24:19.102 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-25T10:24:19.102 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-04-25T10:24:19.102 DEBUG:teuthology.orchestra.run.smithi134:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-25T10:24:19.134 INFO:teuthology.orchestra.run.smithi134.stdout:ttyS1 2024-04-25T10:24:19.153 DEBUG:teuthology.parallel:result is None 2024-04-25T10:24:19.321 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T10:24:19.321 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:24:19.321 INFO:teuthology.orchestra.run.smithi106.stdout:Reinstalled: 2024-04-25T10:24:19.321 INFO:teuthology.orchestra.run.smithi106.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-25T10:24:19.321 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:24:19.321 INFO:teuthology.orchestra.run.smithi106.stdout:Complete! 2024-04-25T10:24:19.489 DEBUG:teuthology.orchestra.run.smithi106:> rpm -q kernel | sort -rV | head -n 1 2024-04-25T10:24:19.535 INFO:teuthology.orchestra.run.smithi106.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-25T10:24:19.536 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-25T10:24:19.536 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-25T10:24:19.536 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-25T10:24:19.536 DEBUG:teuthology.orchestra.run.smithi106:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-25T10:24:19.569 INFO:teuthology.orchestra.run.smithi106.stdout:ttyS1 2024-04-25T10:24:19.587 DEBUG:teuthology.parallel:result is None 2024-04-25T10:24:19.587 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T10:24:19.605 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T10:24:19.605 DEBUG:teuthology.orchestra.run.smithi106:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T10:24:19.630 DEBUG:teuthology.orchestra.run.smithi134:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T10:24:19.645 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T10:24:19.661 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T10:24:19.669 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T10:24:19.669 DEBUG:teuthology.orchestra.run.smithi106:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T10:24:19.686 DEBUG:teuthology.orchestra.run.smithi134:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T10:24:19.716 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T10:24:19.723 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T10:24:19.723 DEBUG:teuthology.orchestra.run.smithi106:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T10:24:19.746 DEBUG:teuthology.orchestra.run.smithi134:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T10:24:19.774 INFO:teuthology.orchestra.run.smithi106.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T10:24:19.783 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T10:24:19.783 INFO:teuthology.orchestra.run.smithi106.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T10:24:19.792 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T10:24:19.794 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T10:24:19.800 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T10:24:19.800 DEBUG:teuthology.orchestra.run.smithi106:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T10:24:19.827 DEBUG:teuthology.orchestra.run.smithi134:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T10:24:19.857 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T10:24:19.866 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T10:24:19.866 DEBUG:teuthology.orchestra.run.smithi106:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T10:24:19.890 DEBUG:teuthology.orchestra.run.smithi134:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T10:24:19.912 DEBUG:teuthology.orchestra.run.smithi106:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T10:24:19.984 DEBUG:teuthology.orchestra.run.smithi106:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T10:24:20.036 DEBUG:teuthology.orchestra.run.smithi106:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T10:24:20.117 DEBUG:teuthology.orchestra.run.smithi106:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T10:24:20.155 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T10:24:20.156 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T10:24:20.219 DEBUG:teuthology.orchestra.run.smithi134:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T10:24:20.258 DEBUG:teuthology.orchestra.run.smithi134:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T10:24:20.310 DEBUG:teuthology.orchestra.run.smithi134:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T10:24:20.391 DEBUG:teuthology.orchestra.run.smithi134:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T10:24:20.431 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-25T10:24:20.431 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T10:24:20.494 DEBUG:teuthology.orchestra.run.smithi106:> sudo service rsyslog restart 2024-04-25T10:24:20.496 DEBUG:teuthology.orchestra.run.smithi134:> sudo service rsyslog restart 2024-04-25T10:24:20.532 INFO:teuthology.orchestra.run.smithi106.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T10:24:20.572 INFO:teuthology.orchestra.run.smithi134.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T10:24:21.087 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T10:24:21.098 INFO:teuthology.task.internal:Starting timer... 2024-04-25T10:24:21.098 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T10:24:21.113 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T10:24:21.122 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-04-25T10:24:21.122 DEBUG:teuthology.orchestra.run.smithi106:> sudo service auditd rotate 2024-04-25T10:24:21.186 INFO:teuthology.orchestra.run.smithi106.stdout:Rotating logs: 2024-04-25T10:24:21.188 DEBUG:teuthology.orchestra.run.smithi134:> sudo service auditd rotate 2024-04-25T10:24:21.258 INFO:teuthology.orchestra.run.smithi134.stdout:Rotating logs: 2024-04-25T10:24:21.260 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T10:24:21.260 DEBUG:teuthology.orchestra.run.smithi106:> /usr/sbin/getenforce 2024-04-25T10:24:21.285 INFO:teuthology.orchestra.run.smithi106.stdout:Permissive 2024-04-25T10:24:21.285 DEBUG:teuthology.orchestra.run.smithi134:> /usr/sbin/getenforce 2024-04-25T10:24:21.317 INFO:teuthology.orchestra.run.smithi134.stdout:Permissive 2024-04-25T10:24:21.317 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi106.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi134.front.sepia.ceph.com': 'permissive'} 2024-04-25T10:24:21.318 DEBUG:teuthology.orchestra.run.smithi106:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T10:24:21.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:24:21.351 DEBUG:teuthology.orchestra.run.smithi134:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T10:24:21.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:24:21.383 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T10:24:21.383 DEBUG:teuthology.orchestra.run.smithi106:> sudo /usr/sbin/setenforce permissive 2024-04-25T10:24:21.421 DEBUG:teuthology.orchestra.run.smithi134:> sudo /usr/sbin/setenforce permissive 2024-04-25T10:24:21.453 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T10:24:21.463 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-25T10:24:21.468 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-25T10:24:21.592 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T10:24:21.609 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T10:24:21.613 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi106.front.sepia.ceph.com,smithi134.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T10:27:48.619 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi106.front.sepia.ceph.com'), Remote(name='ubuntu@smithi134.front.sepia.ceph.com')] 2024-04-25T10:27:48.621 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi106.front.sepia.ceph.com' 2024-04-25T10:27:48.622 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T10:27:48.705 DEBUG:teuthology.orchestra.run.smithi106:> true 2024-04-25T10:27:48.781 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi106.front.sepia.ceph.com' 2024-04-25T10:27:48.781 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-04-25T10:27:48.782 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T10:27:48.870 DEBUG:teuthology.orchestra.run.smithi134:> true 2024-04-25T10:27:48.950 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-04-25T10:27:48.951 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T10:27:48.965 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T10:27:48.966 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T10:27:48.967 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T10:27:48.969 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T10:27:48.969 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T10:27:48.995 INFO:teuthology.orchestra.run.smithi106.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T10:27:49.009 INFO:teuthology.orchestra.run.smithi106.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T10:27:49.021 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T10:27:49.031 INFO:teuthology.orchestra.run.smithi106.stderr:sudo: ntpd: command not found 2024-04-25T10:27:49.034 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T10:27:49.042 INFO:teuthology.orchestra.run.smithi106.stdout:506 Cannot talk to daemon 2024-04-25T10:27:49.055 INFO:teuthology.orchestra.run.smithi106.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T10:27:49.057 INFO:teuthology.orchestra.run.smithi134.stderr:sudo: ntpd: command not found 2024-04-25T10:27:49.068 INFO:teuthology.orchestra.run.smithi106.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T10:27:49.068 INFO:teuthology.orchestra.run.smithi134.stdout:506 Cannot talk to daemon 2024-04-25T10:27:49.081 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T10:27:49.095 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T10:27:49.120 INFO:teuthology.orchestra.run.smithi106.stderr:bash: line 1: ntpq: command not found 2024-04-25T10:27:49.123 INFO:teuthology.orchestra.run.smithi106.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T10:27:49.123 INFO:teuthology.orchestra.run.smithi106.stdout:=============================================================================== 2024-04-25T10:27:49.123 INFO:teuthology.orchestra.run.smithi106.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:27:49.123 INFO:teuthology.orchestra.run.smithi106.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:27:49.123 INFO:teuthology.orchestra.run.smithi106.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:27:49.124 INFO:teuthology.orchestra.run.smithi106.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:27:49.149 INFO:teuthology.orchestra.run.smithi134.stderr:bash: line 1: ntpq: command not found 2024-04-25T10:27:49.151 INFO:teuthology.orchestra.run.smithi134.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T10:27:49.152 INFO:teuthology.orchestra.run.smithi134.stdout:=============================================================================== 2024-04-25T10:27:49.152 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:27:49.152 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:27:49.152 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:27:49.152 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:27:49.153 INFO:teuthology.run_tasks:Running task install... 2024-04-25T10:27:49.161 DEBUG:teuthology.task.install:project ceph 2024-04-25T10:27:49.161 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}} 2024-04-25T10:27:49.162 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'} 2024-04-25T10:27:49.162 INFO:teuthology.task.install:Using flavor: default 2024-04-25T10:27:49.169 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T10:27:49.169 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-25T10:27:49.169 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'tag': None, 'wait_for_package': False} 2024-04-25T10:27:49.170 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T10:27:49.172 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'tag': None, 'wait_for_package': False} 2024-04-25T10:27:49.172 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T10:27:49.316 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/ 2024-04-25T10:27:49.316 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3291.g994e2f20 2024-04-25T10:27:49.324 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/ 2024-04-25T10:27:49.325 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3291.g994e2f20 2024-04-25T10:27:49.583 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T10:27:49.584 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-25T10:27:49.584 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T10:27:49.586 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T10:27:49.586 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T10:27:49.586 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T10:27:49.611 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T10:27:49.611 DEBUG:teuthology.orchestra.run.smithi106:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/994e2f20102a20fa9a27966c14b625cc01884df8/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T10:27:49.613 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T10:27:49.613 DEBUG:teuthology.orchestra.run.smithi134:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/994e2f20102a20fa9a27966c14b625cc01884df8/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T10:27:49.677 DEBUG:teuthology.orchestra.run.smithi106:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T10:27:49.679 DEBUG:teuthology.orchestra.run.smithi134:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T10:27:49.750 DEBUG:teuthology.orchestra.run.smithi106:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T10:27:49.754 DEBUG:teuthology.orchestra.run.smithi134:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T10:27:49.813 INFO:teuthology.orchestra.run.smithi106.stdout:check_obsoletes = 1 2024-04-25T10:27:49.815 DEBUG:teuthology.orchestra.run.smithi106:> sudo yum clean all 2024-04-25T10:27:49.818 INFO:teuthology.orchestra.run.smithi134.stdout:check_obsoletes = 1 2024-04-25T10:27:49.820 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum clean all 2024-04-25T10:27:50.038 INFO:teuthology.orchestra.run.smithi106.stdout:48 files removed 2024-04-25T10:27:50.045 INFO:teuthology.orchestra.run.smithi134.stdout:48 files removed 2024-04-25T10:27:50.063 DEBUG:teuthology.orchestra.run.smithi106:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T10:27:50.072 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T10:27:51.034 INFO:teuthology.orchestra.run.smithi134.stdout:ceph packages for x86_64 119 kB/s | 84 kB 00:00 2024-04-25T10:27:51.051 INFO:teuthology.orchestra.run.smithi106.stdout:ceph packages for x86_64 115 kB/s | 84 kB 00:00 2024-04-25T10:27:51.628 INFO:teuthology.orchestra.run.smithi106.stdout:ceph noarch packages 22 kB/s | 12 kB 00:00 2024-04-25T10:27:51.676 INFO:teuthology.orchestra.run.smithi134.stdout:ceph noarch packages 19 kB/s | 12 kB 00:00 2024-04-25T10:27:52.011 INFO:teuthology.orchestra.run.smithi106.stdout:ceph source packages 5.4 kB/s | 1.9 kB 00:00 2024-04-25T10:27:52.135 INFO:teuthology.orchestra.run.smithi134.stdout:ceph source packages 4.5 kB/s | 1.9 kB 00:00 2024-04-25T10:27:52.620 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.0 MB 00:00 2024-04-25T10:27:53.285 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - BaseOS 7.2 MB/s | 8.0 MB 00:01 2024-04-25T10:27:55.190 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - AppStream 23 MB/s | 19 MB 00:00 2024-04-25T10:27:58.196 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS Stream 9 - AppStream 4.2 MB/s | 19 MB 00:04 2024-04-25T10:27:59.421 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - CRB 9.0 MB/s | 6.2 MB 00:00 2024-04-25T10:28:00.738 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - Extras packages 94 kB/s | 16 kB 00:00 2024-04-25T10:28:02.255 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.2 MB 00:00 2024-04-25T10:28:02.690 INFO:teuthology.orchestra.run.smithi134.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-04-25T10:28:03.881 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS Stream 9 - Extras packages 33 kB/s | 16 kB 00:00 2024-04-25T10:28:06.483 INFO:teuthology.orchestra.run.smithi106.stdout:Extra Packages for Enterprise Linux 8.4 MB/s | 21 MB 00:02 2024-04-25T10:28:07.740 INFO:teuthology.orchestra.run.smithi134.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-04-25T10:28:09.891 INFO:teuthology.orchestra.run.smithi134.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T10:28:09.892 INFO:teuthology.orchestra.run.smithi134.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T10:28:09.902 INFO:teuthology.orchestra.run.smithi134.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T10:28:09.903 INFO:teuthology.orchestra.run.smithi134.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T10:28:09.903 INFO:teuthology.orchestra.run.smithi134.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T10:28:09.904 INFO:teuthology.orchestra.run.smithi134.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T10:28:09.906 INFO:teuthology.orchestra.run.smithi134.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T10:28:09.908 INFO:teuthology.orchestra.run.smithi134.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T10:28:09.909 INFO:teuthology.orchestra.run.smithi134.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T10:28:09.968 INFO:teuthology.orchestra.run.smithi134.stdout:Dependencies resolved. 2024-04-25T10:28:09.977 INFO:teuthology.orchestra.run.smithi134.stdout:======================================================================================= 2024-04-25T10:28:09.978 INFO:teuthology.orchestra.run.smithi134.stdout: Package Arch Version Repository Size 2024-04-25T10:28:09.978 INFO:teuthology.orchestra.run.smithi134.stdout:======================================================================================= 2024-04-25T10:28:09.978 INFO:teuthology.orchestra.run.smithi134.stdout:Installing: 2024-04-25T10:28:09.978 INFO:teuthology.orchestra.run.smithi134.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T10:28:09.978 INFO:teuthology.orchestra.run.smithi134.stdout: ceph x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 6.6 k 2024-04-25T10:28:09.978 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-base x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.4 M 2024-04-25T10:28:09.978 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 863 k 2024-04-25T10:28:09.978 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 147 k 2024-04-25T10:28:09.978 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 1.5 M 2024-04-25T10:28:09.978 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 142 k 2024-04-25T10:28:09.978 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 3.5 M 2024-04-25T10:28:09.978 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 7.4 M 2024-04-25T10:28:09.979 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-rook noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 49 k 2024-04-25T10:28:09.979 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-radosgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 10 M 2024-04-25T10:28:09.979 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-test x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 43 M 2024-04-25T10:28:09.979 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-volume noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 273 k 2024-04-25T10:28:09.979 INFO:teuthology.orchestra.run.smithi134.stdout: cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 762 k 2024-04-25T10:28:09.979 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 224 k 2024-04-25T10:28:09.979 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-shell x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 27 k 2024-04-25T10:28:09.979 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-top noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 18 k 2024-04-25T10:28:09.979 INFO:teuthology.orchestra.run.smithi134.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T10:28:09.979 INFO:teuthology.orchestra.run.smithi134.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T10:28:09.979 INFO:teuthology.orchestra.run.smithi134.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T10:28:09.979 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 32 k 2024-04-25T10:28:09.980 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 756 k 2024-04-25T10:28:09.980 INFO:teuthology.orchestra.run.smithi134.stdout: librados-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 127 k 2024-04-25T10:28:09.980 INFO:teuthology.orchestra.run.smithi134.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T10:28:09.980 INFO:teuthology.orchestra.run.smithi134.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T10:28:09.980 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T10:28:09.980 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T10:28:09.980 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T10:28:09.980 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T10:28:09.980 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cephfs x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 161 k 2024-04-25T10:28:09.980 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rados x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 321 k 2024-04-25T10:28:09.980 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 298 k 2024-04-25T10:28:09.980 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 100 k 2024-04-25T10:28:09.980 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 86 k 2024-04-25T10:28:09.981 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.0 M 2024-04-25T10:28:09.981 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-nbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 169 k 2024-04-25T10:28:09.981 INFO:teuthology.orchestra.run.smithi134.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T10:28:09.981 INFO:teuthology.orchestra.run.smithi134.stdout:Upgrading: 2024-04-25T10:28:09.981 INFO:teuthology.orchestra.run.smithi134.stdout: librados2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.3 M 2024-04-25T10:28:09.981 INFO:teuthology.orchestra.run.smithi134.stdout: librbd1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.1 M 2024-04-25T10:28:09.981 INFO:teuthology.orchestra.run.smithi134.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T10:28:09.981 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T10:28:09.981 INFO:teuthology.orchestra.run.smithi134.stdout:Installing dependencies: 2024-04-25T10:28:09.981 INFO:teuthology.orchestra.run.smithi134.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T10:28:09.981 INFO:teuthology.orchestra.run.smithi134.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T10:28:09.981 INFO:teuthology.orchestra.run.smithi134.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T10:28:09.981 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 20 M 2024-04-25T10:28:09.982 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 34 k 2024-04-25T10:28:09.982 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mds x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 2.3 M 2024-04-25T10:28:09.982 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 249 k 2024-04-25T10:28:09.982 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mon x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 4.9 M 2024-04-25T10:28:09.982 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-osd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 18 M 2024-04-25T10:28:09.982 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 16 k 2024-04-25T10:28:09.982 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-selinux x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 25 k 2024-04-25T10:28:09.982 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T10:28:09.982 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T10:28:09.983 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T10:28:09.983 INFO:teuthology.orchestra.run.smithi134.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T10:28:09.983 INFO:teuthology.orchestra.run.smithi134.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T10:28:09.983 INFO:teuthology.orchestra.run.smithi134.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T10:28:09.983 INFO:teuthology.orchestra.run.smithi134.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T10:28:09.983 INFO:teuthology.orchestra.run.smithi134.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T10:28:09.983 INFO:teuthology.orchestra.run.smithi134.stdout: libcephsqlite x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 158 k 2024-04-25T10:28:09.983 INFO:teuthology.orchestra.run.smithi134.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T10:28:09.983 INFO:teuthology.orchestra.run.smithi134.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T10:28:09.983 INFO:teuthology.orchestra.run.smithi134.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T10:28:09.983 INFO:teuthology.orchestra.run.smithi134.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T10:28:09.983 INFO:teuthology.orchestra.run.smithi134.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T10:28:09.983 INFO:teuthology.orchestra.run.smithi134.stdout: libradosstriper1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 497 k 2024-04-25T10:28:09.983 INFO:teuthology.orchestra.run.smithi134.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T10:28:09.984 INFO:teuthology.orchestra.run.smithi134.stdout: librgw2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.2 M 2024-04-25T10:28:09.984 INFO:teuthology.orchestra.run.smithi134.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T10:28:09.984 INFO:teuthology.orchestra.run.smithi134.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T10:28:09.984 INFO:teuthology.orchestra.run.smithi134.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T10:28:09.984 INFO:teuthology.orchestra.run.smithi134.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T10:28:09.984 INFO:teuthology.orchestra.run.smithi134.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T10:28:09.984 INFO:teuthology.orchestra.run.smithi134.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T10:28:09.984 INFO:teuthology.orchestra.run.smithi134.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T10:28:09.984 INFO:teuthology.orchestra.run.smithi134.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T10:28:09.984 INFO:teuthology.orchestra.run.smithi134.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T10:28:09.984 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T10:28:09.984 INFO:teuthology.orchestra.run.smithi134.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T10:28:09.984 INFO:teuthology.orchestra.run.smithi134.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T10:28:09.984 INFO:teuthology.orchestra.run.smithi134.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T10:28:09.985 INFO:teuthology.orchestra.run.smithi134.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T10:28:09.985 INFO:teuthology.orchestra.run.smithi134.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T10:28:09.985 INFO:teuthology.orchestra.run.smithi134.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T10:28:09.985 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T10:28:09.985 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-argparse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 45 k 2024-04-25T10:28:09.985 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 135 k 2024-04-25T10:28:09.985 INFO:teuthology.orchestra.run.smithi134.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T10:28:09.985 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T10:28:09.985 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T10:28:09.985 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T10:28:09.985 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T10:28:09.985 INFO:teuthology.orchestra.run.smithi134.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T10:28:09.986 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T10:28:09.986 INFO:teuthology.orchestra.run.smithi134.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T10:28:09.986 INFO:teuthology.orchestra.run.smithi134.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T10:28:09.986 INFO:teuthology.orchestra.run.smithi134.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T10:28:09.986 INFO:teuthology.orchestra.run.smithi134.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T10:28:09.986 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T10:28:09.986 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T10:28:09.986 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T10:28:09.986 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T10:28:09.986 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T10:28:09.986 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T10:28:09.986 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T10:28:09.987 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T10:28:09.987 INFO:teuthology.orchestra.run.smithi134.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T10:28:09.987 INFO:teuthology.orchestra.run.smithi134.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T10:28:09.987 INFO:teuthology.orchestra.run.smithi134.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T10:28:09.987 INFO:teuthology.orchestra.run.smithi134.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T10:28:09.987 INFO:teuthology.orchestra.run.smithi134.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T10:28:09.987 INFO:teuthology.orchestra.run.smithi134.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T10:28:09.987 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T10:28:09.987 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T10:28:09.987 INFO:teuthology.orchestra.run.smithi134.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T10:28:09.988 INFO:teuthology.orchestra.run.smithi134.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T10:28:09.988 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T10:28:09.988 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T10:28:09.988 INFO:teuthology.orchestra.run.smithi134.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T10:28:09.988 INFO:teuthology.orchestra.run.smithi134.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T10:28:09.988 INFO:teuthology.orchestra.run.smithi134.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T10:28:09.988 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T10:28:09.988 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T10:28:09.988 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T10:28:09.988 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T10:28:09.988 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T10:28:09.989 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T10:28:09.989 INFO:teuthology.orchestra.run.smithi134.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T10:28:09.989 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T10:28:09.989 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T10:28:09.989 INFO:teuthology.orchestra.run.smithi134.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T10:28:09.989 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T10:28:09.989 INFO:teuthology.orchestra.run.smithi134.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T10:28:09.989 INFO:teuthology.orchestra.run.smithi134.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T10:28:09.989 INFO:teuthology.orchestra.run.smithi134.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T10:28:09.989 INFO:teuthology.orchestra.run.smithi134.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T10:28:09.989 INFO:teuthology.orchestra.run.smithi134.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T10:28:09.989 INFO:teuthology.orchestra.run.smithi134.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T10:28:09.989 INFO:teuthology.orchestra.run.smithi134.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T10:28:09.989 INFO:teuthology.orchestra.run.smithi134.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T10:28:09.989 INFO:teuthology.orchestra.run.smithi134.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T10:28:09.990 INFO:teuthology.orchestra.run.smithi134.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T10:28:09.990 INFO:teuthology.orchestra.run.smithi134.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T10:28:09.990 INFO:teuthology.orchestra.run.smithi134.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T10:28:09.990 INFO:teuthology.orchestra.run.smithi134.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T10:28:09.990 INFO:teuthology.orchestra.run.smithi134.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T10:28:09.990 INFO:teuthology.orchestra.run.smithi134.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T10:28:09.990 INFO:teuthology.orchestra.run.smithi134.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T10:28:09.990 INFO:teuthology.orchestra.run.smithi134.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T10:28:09.990 INFO:teuthology.orchestra.run.smithi134.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T10:28:09.990 INFO:teuthology.orchestra.run.smithi134.stdout:Installing weak dependencies: 2024-04-25T10:28:09.990 INFO:teuthology.orchestra.run.smithi134.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T10:28:09.990 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T10:28:09.990 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction Summary 2024-04-25T10:28:09.990 INFO:teuthology.orchestra.run.smithi134.stdout:======================================================================================= 2024-04-25T10:28:09.991 INFO:teuthology.orchestra.run.smithi134.stdout:Install 148 Packages 2024-04-25T10:28:09.991 INFO:teuthology.orchestra.run.smithi134.stdout:Upgrade 4 Packages 2024-04-25T10:28:09.991 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T10:28:09.991 INFO:teuthology.orchestra.run.smithi134.stdout:Total download size: 214 M 2024-04-25T10:28:09.991 INFO:teuthology.orchestra.run.smithi134.stdout:Downloading Packages: 2024-04-25T10:28:11.257 INFO:teuthology.orchestra.run.smithi134.stdout:(1/152): ceph-19.0.0-3291.g994e2f20.el9.x86_64. 29 kB/s | 6.6 kB 00:00 2024-04-25T10:28:11.466 INFO:teuthology.orchestra.run.smithi106.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-04-25T10:28:11.582 INFO:teuthology.orchestra.run.smithi134.stdout:(2/152): ceph-fuse-19.0.0-3291.g994e2f20.el9.x8 2.6 MB/s | 863 kB 00:00 2024-04-25T10:28:11.676 INFO:teuthology.orchestra.run.smithi134.stdout:(3/152): ceph-base-19.0.0-3291.g994e2f20.el9.x8 8.3 MB/s | 5.4 MB 00:00 2024-04-25T10:28:11.701 INFO:teuthology.orchestra.run.smithi134.stdout:(4/152): ceph-immutable-object-cache-19.0.0-329 1.2 MB/s | 147 kB 00:00 2024-04-25T10:28:11.885 INFO:teuthology.orchestra.run.smithi134.stdout:(5/152): ceph-mds-19.0.0-3291.g994e2f20.el9.x86 11 MB/s | 2.3 MB 00:00 2024-04-25T10:28:12.027 INFO:teuthology.orchestra.run.smithi134.stdout:(6/152): ceph-mgr-19.0.0-3291.g994e2f20.el9.x86 4.6 MB/s | 1.5 MB 00:00 2024-04-25T10:28:12.286 INFO:teuthology.orchestra.run.smithi134.stdout:(7/152): ceph-mon-19.0.0-3291.g994e2f20.el9.x86 12 MB/s | 4.9 MB 00:00 2024-04-25T10:28:13.203 INFO:teuthology.orchestra.run.smithi134.stdout:(8/152): ceph-radosgw-19.0.0-3291.g994e2f20.el9 11 MB/s | 10 MB 00:00 2024-04-25T10:28:13.261 INFO:teuthology.orchestra.run.smithi134.stdout:(9/152): ceph-selinux-19.0.0-3291.g994e2f20.el9 433 kB/s | 25 kB 00:00 2024-04-25T10:28:13.578 INFO:teuthology.orchestra.run.smithi134.stdout:(10/152): ceph-common-19.0.0-3291.g994e2f20.el9 7.7 MB/s | 20 MB 00:02 2024-04-25T10:28:13.592 INFO:teuthology.orchestra.run.smithi106.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T10:28:13.592 INFO:teuthology.orchestra.run.smithi106.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T10:28:13.603 INFO:teuthology.orchestra.run.smithi106.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T10:28:13.604 INFO:teuthology.orchestra.run.smithi106.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T10:28:13.605 INFO:teuthology.orchestra.run.smithi106.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T10:28:13.606 INFO:teuthology.orchestra.run.smithi106.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T10:28:13.607 INFO:teuthology.orchestra.run.smithi106.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T10:28:13.609 INFO:teuthology.orchestra.run.smithi106.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T10:28:13.610 INFO:teuthology.orchestra.run.smithi106.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T10:28:13.669 INFO:teuthology.orchestra.run.smithi106.stdout:Dependencies resolved. 2024-04-25T10:28:13.678 INFO:teuthology.orchestra.run.smithi106.stdout:======================================================================================= 2024-04-25T10:28:13.678 INFO:teuthology.orchestra.run.smithi106.stdout: Package Arch Version Repository Size 2024-04-25T10:28:13.679 INFO:teuthology.orchestra.run.smithi106.stdout:======================================================================================= 2024-04-25T10:28:13.679 INFO:teuthology.orchestra.run.smithi106.stdout:Installing: 2024-04-25T10:28:13.679 INFO:teuthology.orchestra.run.smithi106.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T10:28:13.679 INFO:teuthology.orchestra.run.smithi106.stdout: ceph x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 6.6 k 2024-04-25T10:28:13.679 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-base x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.4 M 2024-04-25T10:28:13.679 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 863 k 2024-04-25T10:28:13.679 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 147 k 2024-04-25T10:28:13.679 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 1.5 M 2024-04-25T10:28:13.679 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 142 k 2024-04-25T10:28:13.679 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 3.5 M 2024-04-25T10:28:13.679 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 7.4 M 2024-04-25T10:28:13.679 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-rook noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 49 k 2024-04-25T10:28:13.679 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-radosgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 10 M 2024-04-25T10:28:13.680 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-test x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 43 M 2024-04-25T10:28:13.680 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-volume noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 273 k 2024-04-25T10:28:13.680 INFO:teuthology.orchestra.run.smithi106.stdout: cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 762 k 2024-04-25T10:28:13.680 INFO:teuthology.orchestra.run.smithi106.stdout: cephfs-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 224 k 2024-04-25T10:28:13.680 INFO:teuthology.orchestra.run.smithi106.stdout: cephfs-shell x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 27 k 2024-04-25T10:28:13.680 INFO:teuthology.orchestra.run.smithi106.stdout: cephfs-top noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 18 k 2024-04-25T10:28:13.680 INFO:teuthology.orchestra.run.smithi106.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T10:28:13.680 INFO:teuthology.orchestra.run.smithi106.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T10:28:13.680 INFO:teuthology.orchestra.run.smithi106.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T10:28:13.680 INFO:teuthology.orchestra.run.smithi106.stdout: libcephfs-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 32 k 2024-04-25T10:28:13.680 INFO:teuthology.orchestra.run.smithi106.stdout: libcephfs2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 756 k 2024-04-25T10:28:13.680 INFO:teuthology.orchestra.run.smithi106.stdout: librados-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 127 k 2024-04-25T10:28:13.680 INFO:teuthology.orchestra.run.smithi106.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T10:28:13.680 INFO:teuthology.orchestra.run.smithi106.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T10:28:13.681 INFO:teuthology.orchestra.run.smithi106.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T10:28:13.681 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T10:28:13.681 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T10:28:13.681 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T10:28:13.681 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cephfs x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 161 k 2024-04-25T10:28:13.681 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rados x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 321 k 2024-04-25T10:28:13.681 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 298 k 2024-04-25T10:28:13.681 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 100 k 2024-04-25T10:28:13.681 INFO:teuthology.orchestra.run.smithi106.stdout: rbd-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 86 k 2024-04-25T10:28:13.681 INFO:teuthology.orchestra.run.smithi106.stdout: rbd-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.0 M 2024-04-25T10:28:13.681 INFO:teuthology.orchestra.run.smithi106.stdout: rbd-nbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 169 k 2024-04-25T10:28:13.681 INFO:teuthology.orchestra.run.smithi106.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T10:28:13.681 INFO:teuthology.orchestra.run.smithi106.stdout:Upgrading: 2024-04-25T10:28:13.681 INFO:teuthology.orchestra.run.smithi106.stdout: librados2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.3 M 2024-04-25T10:28:13.682 INFO:teuthology.orchestra.run.smithi106.stdout: librbd1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.1 M 2024-04-25T10:28:13.682 INFO:teuthology.orchestra.run.smithi106.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T10:28:13.682 INFO:teuthology.orchestra.run.smithi106.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T10:28:13.682 INFO:teuthology.orchestra.run.smithi106.stdout:Installing dependencies: 2024-04-25T10:28:13.682 INFO:teuthology.orchestra.run.smithi106.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T10:28:13.682 INFO:teuthology.orchestra.run.smithi106.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T10:28:13.682 INFO:teuthology.orchestra.run.smithi106.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T10:28:13.682 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 20 M 2024-04-25T10:28:13.682 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 34 k 2024-04-25T10:28:13.682 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mds x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 2.3 M 2024-04-25T10:28:13.682 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 249 k 2024-04-25T10:28:13.682 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mon x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 4.9 M 2024-04-25T10:28:13.682 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-osd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 18 M 2024-04-25T10:28:13.683 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 16 k 2024-04-25T10:28:13.683 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-selinux x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 25 k 2024-04-25T10:28:13.683 INFO:teuthology.orchestra.run.smithi106.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T10:28:13.683 INFO:teuthology.orchestra.run.smithi106.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T10:28:13.683 INFO:teuthology.orchestra.run.smithi106.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T10:28:13.683 INFO:teuthology.orchestra.run.smithi106.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T10:28:13.683 INFO:teuthology.orchestra.run.smithi106.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T10:28:13.683 INFO:teuthology.orchestra.run.smithi106.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T10:28:13.683 INFO:teuthology.orchestra.run.smithi106.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T10:28:13.683 INFO:teuthology.orchestra.run.smithi106.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T10:28:13.683 INFO:teuthology.orchestra.run.smithi106.stdout: libcephsqlite x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 158 k 2024-04-25T10:28:13.683 INFO:teuthology.orchestra.run.smithi106.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T10:28:13.683 INFO:teuthology.orchestra.run.smithi106.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T10:28:13.683 INFO:teuthology.orchestra.run.smithi106.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T10:28:13.684 INFO:teuthology.orchestra.run.smithi106.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T10:28:13.684 INFO:teuthology.orchestra.run.smithi106.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T10:28:13.684 INFO:teuthology.orchestra.run.smithi106.stdout: libradosstriper1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 497 k 2024-04-25T10:28:13.684 INFO:teuthology.orchestra.run.smithi106.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T10:28:13.684 INFO:teuthology.orchestra.run.smithi106.stdout: librgw2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.2 M 2024-04-25T10:28:13.684 INFO:teuthology.orchestra.run.smithi106.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T10:28:13.684 INFO:teuthology.orchestra.run.smithi106.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T10:28:13.684 INFO:teuthology.orchestra.run.smithi106.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T10:28:13.684 INFO:teuthology.orchestra.run.smithi106.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T10:28:13.684 INFO:teuthology.orchestra.run.smithi106.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T10:28:13.684 INFO:teuthology.orchestra.run.smithi106.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T10:28:13.684 INFO:teuthology.orchestra.run.smithi106.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T10:28:13.684 INFO:teuthology.orchestra.run.smithi106.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T10:28:13.684 INFO:teuthology.orchestra.run.smithi106.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T10:28:13.685 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T10:28:13.685 INFO:teuthology.orchestra.run.smithi106.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T10:28:13.685 INFO:teuthology.orchestra.run.smithi106.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T10:28:13.685 INFO:teuthology.orchestra.run.smithi106.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T10:28:13.685 INFO:teuthology.orchestra.run.smithi106.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T10:28:13.685 INFO:teuthology.orchestra.run.smithi106.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T10:28:13.685 INFO:teuthology.orchestra.run.smithi106.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T10:28:13.685 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T10:28:13.685 INFO:teuthology.orchestra.run.smithi106.stdout: python3-ceph-argparse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 45 k 2024-04-25T10:28:13.685 INFO:teuthology.orchestra.run.smithi106.stdout: python3-ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 135 k 2024-04-25T10:28:13.685 INFO:teuthology.orchestra.run.smithi106.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T10:28:13.685 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T10:28:13.685 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T10:28:13.686 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T10:28:13.686 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T10:28:13.686 INFO:teuthology.orchestra.run.smithi106.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T10:28:13.686 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T10:28:13.686 INFO:teuthology.orchestra.run.smithi106.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T10:28:13.686 INFO:teuthology.orchestra.run.smithi106.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T10:28:13.686 INFO:teuthology.orchestra.run.smithi106.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T10:28:13.686 INFO:teuthology.orchestra.run.smithi106.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T10:28:13.686 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T10:28:13.686 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T10:28:13.686 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T10:28:13.686 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T10:28:13.686 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T10:28:13.686 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T10:28:13.687 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T10:28:13.687 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T10:28:13.687 INFO:teuthology.orchestra.run.smithi106.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T10:28:13.687 INFO:teuthology.orchestra.run.smithi106.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T10:28:13.687 INFO:teuthology.orchestra.run.smithi106.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T10:28:13.687 INFO:teuthology.orchestra.run.smithi106.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T10:28:13.687 INFO:teuthology.orchestra.run.smithi106.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T10:28:13.687 INFO:teuthology.orchestra.run.smithi106.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T10:28:13.687 INFO:teuthology.orchestra.run.smithi106.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T10:28:13.687 INFO:teuthology.orchestra.run.smithi106.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T10:28:13.687 INFO:teuthology.orchestra.run.smithi106.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T10:28:13.687 INFO:teuthology.orchestra.run.smithi106.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T10:28:13.687 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T10:28:13.687 INFO:teuthology.orchestra.run.smithi106.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T10:28:13.688 INFO:teuthology.orchestra.run.smithi106.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T10:28:13.688 INFO:teuthology.orchestra.run.smithi106.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T10:28:13.688 INFO:teuthology.orchestra.run.smithi106.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T10:28:13.688 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T10:28:13.688 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T10:28:13.688 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T10:28:13.688 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T10:28:13.688 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T10:28:13.688 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T10:28:13.688 INFO:teuthology.orchestra.run.smithi106.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T10:28:13.688 INFO:teuthology.orchestra.run.smithi106.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T10:28:13.688 INFO:teuthology.orchestra.run.smithi106.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T10:28:13.688 INFO:teuthology.orchestra.run.smithi106.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T10:28:13.688 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T10:28:13.689 INFO:teuthology.orchestra.run.smithi106.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T10:28:13.689 INFO:teuthology.orchestra.run.smithi106.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T10:28:13.689 INFO:teuthology.orchestra.run.smithi106.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T10:28:13.689 INFO:teuthology.orchestra.run.smithi106.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T10:28:13.689 INFO:teuthology.orchestra.run.smithi106.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T10:28:13.689 INFO:teuthology.orchestra.run.smithi106.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T10:28:13.689 INFO:teuthology.orchestra.run.smithi106.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T10:28:13.689 INFO:teuthology.orchestra.run.smithi106.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T10:28:13.689 INFO:teuthology.orchestra.run.smithi106.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T10:28:13.689 INFO:teuthology.orchestra.run.smithi106.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T10:28:13.689 INFO:teuthology.orchestra.run.smithi106.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T10:28:13.689 INFO:teuthology.orchestra.run.smithi106.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T10:28:13.689 INFO:teuthology.orchestra.run.smithi106.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T10:28:13.689 INFO:teuthology.orchestra.run.smithi106.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T10:28:13.690 INFO:teuthology.orchestra.run.smithi106.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T10:28:13.690 INFO:teuthology.orchestra.run.smithi106.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T10:28:13.690 INFO:teuthology.orchestra.run.smithi106.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T10:28:13.690 INFO:teuthology.orchestra.run.smithi106.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T10:28:13.690 INFO:teuthology.orchestra.run.smithi106.stdout:Installing weak dependencies: 2024-04-25T10:28:13.690 INFO:teuthology.orchestra.run.smithi106.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T10:28:13.690 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:28:13.690 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction Summary 2024-04-25T10:28:13.690 INFO:teuthology.orchestra.run.smithi106.stdout:======================================================================================= 2024-04-25T10:28:13.690 INFO:teuthology.orchestra.run.smithi106.stdout:Install 148 Packages 2024-04-25T10:28:13.690 INFO:teuthology.orchestra.run.smithi106.stdout:Upgrade 4 Packages 2024-04-25T10:28:13.690 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:28:13.691 INFO:teuthology.orchestra.run.smithi106.stdout:Total download size: 214 M 2024-04-25T10:28:13.691 INFO:teuthology.orchestra.run.smithi106.stdout:Downloading Packages: 2024-04-25T10:28:13.862 INFO:teuthology.orchestra.run.smithi134.stdout:(11/152): cephfs-mirror-19.0.0-3291.g994e2f20.e 792 kB/s | 224 kB 00:00 2024-04-25T10:28:13.962 INFO:teuthology.orchestra.run.smithi134.stdout:(12/152): cephfs-shell-19.0.0-3291.g994e2f20.el 271 kB/s | 27 kB 00:00 2024-04-25T10:28:14.162 INFO:teuthology.orchestra.run.smithi134.stdout:(13/152): ceph-osd-19.0.0-3291.g994e2f20.el9.x8 8.3 MB/s | 18 MB 00:02 2024-04-25T10:28:14.187 INFO:teuthology.orchestra.run.smithi134.stdout:(14/152): libcephfs-devel-19.0.0-3291.g994e2f20 141 kB/s | 32 kB 00:00 2024-04-25T10:28:14.246 INFO:teuthology.orchestra.run.smithi134.stdout:(15/152): libcephfs2-19.0.0-3291.g994e2f20.el9. 8.9 MB/s | 756 kB 00:00 2024-04-25T10:28:14.271 INFO:teuthology.orchestra.run.smithi134.stdout:(16/152): libcephsqlite-19.0.0-3291.g994e2f20.e 1.8 MB/s | 158 kB 00:00 2024-04-25T10:28:14.321 INFO:teuthology.orchestra.run.smithi134.stdout:(17/152): librados-devel-19.0.0-3291.g994e2f20. 1.7 MB/s | 127 kB 00:00 2024-04-25T10:28:14.438 INFO:teuthology.orchestra.run.smithi134.stdout:(18/152): libradosstriper1-19.0.0-3291.g994e2f2 2.9 MB/s | 497 kB 00:00 2024-04-25T10:28:14.547 INFO:teuthology.orchestra.run.smithi134.stdout:(19/152): python3-ceph-argparse-19.0.0-3291.g99 414 kB/s | 45 kB 00:00 2024-04-25T10:28:14.664 INFO:teuthology.orchestra.run.smithi134.stdout:(20/152): python3-ceph-common-19.0.0-3291.g994e 1.1 MB/s | 135 kB 00:00 2024-04-25T10:28:14.740 INFO:teuthology.orchestra.run.smithi134.stdout:(21/152): python3-cephfs-19.0.0-3291.g994e2f20. 2.1 MB/s | 161 kB 00:00 2024-04-25T10:28:14.831 INFO:teuthology.orchestra.run.smithi134.stdout:(22/152): python3-rados-19.0.0-3291.g994e2f20.e 3.4 MB/s | 321 kB 00:00 2024-04-25T10:28:14.924 INFO:teuthology.orchestra.run.smithi134.stdout:(23/152): python3-rbd-19.0.0-3291.g994e2f20.el9 3.2 MB/s | 298 kB 00:00 2024-04-25T10:28:14.925 INFO:teuthology.orchestra.run.smithi106.stdout:(1/152): ceph-19.0.0-3291.g994e2f20.el9.x86_64. 16 kB/s | 6.6 kB 00:00 2024-04-25T10:28:15.007 INFO:teuthology.orchestra.run.smithi134.stdout:(24/152): librgw2-19.0.0-3291.g994e2f20.el9.x86 7.5 MB/s | 5.2 MB 00:00 2024-04-25T10:28:15.034 INFO:teuthology.orchestra.run.smithi134.stdout:(25/152): python3-rgw-19.0.0-3291.g994e2f20.el9 918 kB/s | 100 kB 00:00 2024-04-25T10:28:15.074 INFO:teuthology.orchestra.run.smithi134.stdout:(26/152): rbd-fuse-19.0.0-3291.g994e2f20.el9.x8 1.3 MB/s | 86 kB 00:00 2024-04-25T10:28:15.174 INFO:teuthology.orchestra.run.smithi134.stdout:(27/152): rbd-nbd-19.0.0-3291.g994e2f20.el9.x86 1.7 MB/s | 169 kB 00:00 2024-04-25T10:28:15.275 INFO:teuthology.orchestra.run.smithi134.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3291.g 342 kB/s | 34 kB 00:00 2024-04-25T10:28:15.383 INFO:teuthology.orchestra.run.smithi134.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3291.g994e2f2 1.3 MB/s | 142 kB 00:00 2024-04-25T10:28:15.442 INFO:teuthology.orchestra.run.smithi134.stdout:(30/152): rbd-mirror-19.0.0-3291.g994e2f20.el9. 7.4 MB/s | 3.0 MB 00:00 2024-04-25T10:28:15.859 INFO:teuthology.orchestra.run.smithi134.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3291.g994e2 7.4 MB/s | 3.5 MB 00:00 2024-04-25T10:28:15.968 INFO:teuthology.orchestra.run.smithi134.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3291.g99 2.2 MB/s | 249 kB 00:00 2024-04-25T10:28:16.051 INFO:teuthology.orchestra.run.smithi106.stdout:(2/152): ceph-base-19.0.0-3291.g994e2f20.el9.x8 3.5 MB/s | 5.4 MB 00:01 2024-04-25T10:28:16.060 INFO:teuthology.orchestra.run.smithi134.stdout:(33/152): ceph-mgr-rook-19.0.0-3291.g994e2f20.e 537 kB/s | 49 kB 00:00 2024-04-25T10:28:16.101 INFO:teuthology.orchestra.run.smithi106.stdout:(3/152): ceph-immutable-object-cache-19.0.0-329 2.9 MB/s | 147 kB 00:00 2024-04-25T10:28:16.126 INFO:teuthology.orchestra.run.smithi106.stdout:(4/152): ceph-fuse-19.0.0-3291.g994e2f20.el9.x8 719 kB/s | 863 kB 00:01 2024-04-25T10:28:16.143 INFO:teuthology.orchestra.run.smithi134.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3291.g9 198 kB/s | 16 kB 00:00 2024-04-25T10:28:16.243 INFO:teuthology.orchestra.run.smithi134.stdout:(35/152): ceph-volume-19.0.0-3291.g994e2f20.el9 2.7 MB/s | 273 kB 00:00 2024-04-25T10:28:16.419 INFO:teuthology.orchestra.run.smithi134.stdout:(36/152): cephadm-19.0.0-3291.g994e2f20.el9.noa 4.3 MB/s | 762 kB 00:00 2024-04-25T10:28:16.511 INFO:teuthology.orchestra.run.smithi134.stdout:(37/152): ceph-mgr-diskprediction-local-19.0.0- 6.9 MB/s | 7.4 MB 00:01 2024-04-25T10:28:16.527 INFO:teuthology.orchestra.run.smithi134.stdout:(38/152): cephfs-top-19.0.0-3291.g994e2f20.el9. 168 kB/s | 18 kB 00:00 2024-04-25T10:28:16.551 INFO:teuthology.orchestra.run.smithi106.stdout:(5/152): ceph-mds-19.0.0-3291.g994e2f20.el9.x86 5.0 MB/s | 2.3 MB 00:00 2024-04-25T10:28:16.945 INFO:teuthology.orchestra.run.smithi134.stdout:(39/152): ceph-test-19.0.0-3291.g994e2f20.el9.x 12 MB/s | 43 MB 00:03 2024-04-25T10:28:17.018 INFO:teuthology.orchestra.run.smithi106.stdout:(6/152): ceph-mon-19.0.0-3291.g994e2f20.el9.x86 10 MB/s | 4.9 MB 00:00 2024-04-25T10:28:17.062 INFO:teuthology.orchestra.run.smithi134.stdout:(40/152): fuse-2.9.9-15.el9.x86_64.rpm 146 kB/s | 80 kB 00:00 2024-04-25T10:28:17.128 INFO:teuthology.orchestra.run.smithi134.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 195 kB 00:00 2024-04-25T10:28:17.135 INFO:teuthology.orchestra.run.smithi106.stdout:(7/152): ceph-mgr-19.0.0-3291.g994e2f20.el9.x86 1.5 MB/s | 1.5 MB 00:01 2024-04-25T10:28:17.179 INFO:teuthology.orchestra.run.smithi134.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.1 MB/s | 253 kB 00:00 2024-04-25T10:28:17.212 INFO:teuthology.orchestra.run.smithi134.stdout:(43/152): libgfortran-11.4.1-3.el9.x86_64.rpm 1.1 MB/s | 803 kB 00:00 2024-04-25T10:28:17.238 INFO:teuthology.orchestra.run.smithi134.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-04-25T10:28:17.271 INFO:teuthology.orchestra.run.smithi134.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-04-25T10:28:17.296 INFO:teuthology.orchestra.run.smithi134.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-04-25T10:28:17.330 INFO:teuthology.orchestra.run.smithi134.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-04-25T10:28:17.363 INFO:teuthology.orchestra.run.smithi134.stdout:(48/152): python3-cryptography-36.0.1-4.el9.x86 5.2 MB/s | 1.2 MB 00:00 2024-04-25T10:28:17.397 INFO:teuthology.orchestra.run.smithi134.stdout:(49/152): automake-1.16.2-8.el9.noarch.rpm 6.7 MB/s | 689 kB 00:00 2024-04-25T10:28:17.414 INFO:teuthology.orchestra.run.smithi134.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 107 kB 00:00 2024-04-25T10:28:17.456 INFO:teuthology.orchestra.run.smithi134.stdout:(51/152): bison-3.7.4-5.el9.x86_64.rpm 7.3 MB/s | 943 kB 00:00 2024-04-25T10:28:17.481 INFO:teuthology.orchestra.run.smithi134.stdout:(52/152): elfutils-libelf-devel-0.190-2.el9.x86 256 kB/s | 21 kB 00:00 2024-04-25T10:28:17.507 INFO:teuthology.orchestra.run.smithi134.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 3.3 MB/s | 313 kB 00:00 2024-04-25T10:28:17.532 INFO:teuthology.orchestra.run.smithi134.stdout:(54/152): flexiblas-3.0.4-8.el9.x86_64.rpm 434 kB/s | 33 kB 00:00 2024-04-25T10:28:17.565 INFO:teuthology.orchestra.run.smithi134.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 312 kB/s | 18 kB 00:00 2024-04-25T10:28:17.591 INFO:teuthology.orchestra.run.smithi134.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 190 kB/s | 11 kB 00:00 2024-04-25T10:28:17.624 INFO:teuthology.orchestra.run.smithi134.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-04-25T10:28:17.649 INFO:teuthology.orchestra.run.smithi134.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 772 kB/s | 45 kB 00:00 2024-04-25T10:28:17.717 INFO:teuthology.orchestra.run.smithi134.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 7.0 MB/s | 662 kB 00:00 2024-04-25T10:28:17.742 INFO:teuthology.orchestra.run.smithi134.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 6.2 MB/s | 582 kB 00:00 2024-04-25T10:28:17.792 INFO:teuthology.orchestra.run.smithi134.stdout:(61/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 9.6 MB/s | 3.0 MB 00:00 2024-04-25T10:28:17.818 INFO:teuthology.orchestra.run.smithi134.stdout:(62/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 253 kB/s | 25 kB 00:00 2024-04-25T10:28:17.843 INFO:teuthology.orchestra.run.smithi134.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 470 kB/s | 47 kB 00:00 2024-04-25T10:28:17.876 INFO:teuthology.orchestra.run.smithi134.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-04-25T10:28:17.902 INFO:teuthology.orchestra.run.smithi134.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 601 kB/s | 35 kB 00:00 2024-04-25T10:28:17.927 INFO:teuthology.orchestra.run.smithi134.stdout:(66/152): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-04-25T10:28:18.102 INFO:teuthology.orchestra.run.smithi134.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 9.1 MB/s | 1.6 MB 00:00 2024-04-25T10:28:18.203 INFO:teuthology.orchestra.run.smithi134.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 8.9 MB/s | 911 kB 00:00 2024-04-25T10:28:18.261 INFO:teuthology.orchestra.run.smithi134.stdout:(69/152): openblas-openmp-0.3.21-2.el9.x86_64.r 12 MB/s | 4.7 MB 00:00 2024-04-25T10:28:18.286 INFO:teuthology.orchestra.run.smithi134.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.6 MB/s | 133 kB 00:00 2024-04-25T10:28:18.345 INFO:teuthology.orchestra.run.smithi134.stdout:(71/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 10 MB/s | 4.4 MB 00:00 2024-04-25T10:28:18.404 INFO:teuthology.orchestra.run.smithi134.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-04-25T10:28:18.479 INFO:teuthology.orchestra.run.smithi134.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 5.3 MB/s | 1.0 MB 00:00 2024-04-25T10:28:18.546 INFO:teuthology.orchestra.run.smithi134.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 3.6 MB/s | 244 kB 00:00 2024-04-25T10:28:18.610 INFO:teuthology.orchestra.run.smithi106.stdout:(8/152): ceph-common-19.0.0-3291.g994e2f20.el9. 4.8 MB/s | 20 MB 00:04 2024-04-25T10:28:18.612 INFO:teuthology.orchestra.run.smithi134.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 3.6 MB/s | 248 kB 00:00 2024-04-25T10:28:18.652 INFO:teuthology.orchestra.run.smithi106.stdout:(9/152): ceph-selinux-19.0.0-3291.g994e2f20.el9 609 kB/s | 25 kB 00:00 2024-04-25T10:28:18.679 INFO:teuthology.orchestra.run.smithi134.stdout:(76/152): postgresql-server-13.14-1.el9.x86_64. 14 MB/s | 5.8 MB 00:00 2024-04-25T10:28:18.705 INFO:teuthology.orchestra.run.smithi134.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 278 kB/s | 26 kB 00:00 2024-04-25T10:28:18.738 INFO:teuthology.orchestra.run.smithi134.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 328 kB/s | 19 kB 00:00 2024-04-25T10:28:18.763 INFO:teuthology.orchestra.run.smithi134.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-04-25T10:28:18.797 INFO:teuthology.orchestra.run.smithi134.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 593 kB/s | 35 kB 00:00 2024-04-25T10:28:18.819 INFO:teuthology.orchestra.run.smithi106.stdout:(10/152): ceph-osd-19.0.0-3291.g994e2f20.el9.x8 9.9 MB/s | 18 MB 00:01 2024-04-25T10:28:18.864 INFO:teuthology.orchestra.run.smithi134.stdout:(81/152): python3-babel-2.9.1-2.el9.noarch.rpm 13 MB/s | 6.0 MB 00:00 2024-04-25T10:28:18.869 INFO:teuthology.orchestra.run.smithi106.stdout:(11/152): cephfs-mirror-19.0.0-3291.g994e2f20.e 4.4 MB/s | 224 kB 00:00 2024-04-25T10:28:18.889 INFO:teuthology.orchestra.run.smithi134.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.7 MB/s | 250 kB 00:00 2024-04-25T10:28:18.911 INFO:teuthology.orchestra.run.smithi106.stdout:(12/152): cephfs-shell-19.0.0-3291.g994e2f20.el 655 kB/s | 27 kB 00:00 2024-04-25T10:28:18.923 INFO:teuthology.orchestra.run.smithi134.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-04-25T10:28:18.947 INFO:teuthology.orchestra.run.smithi106.stdout:(13/152): libcephfs-devel-19.0.0-3291.g994e2f20 892 kB/s | 32 kB 00:00 2024-04-25T10:28:18.948 INFO:teuthology.orchestra.run.smithi134.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-04-25T10:28:18.982 INFO:teuthology.orchestra.run.smithi134.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 719 kB/s | 42 kB 00:00 2024-04-25T10:28:19.007 INFO:teuthology.orchestra.run.smithi134.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-04-25T10:28:19.039 INFO:teuthology.orchestra.run.smithi106.stdout:(14/152): libcephfs2-19.0.0-3291.g994e2f20.el9. 8.1 MB/s | 756 kB 00:00 2024-04-25T10:28:19.041 INFO:teuthology.orchestra.run.smithi134.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-04-25T10:28:19.066 INFO:teuthology.orchestra.run.smithi134.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 4.7 MB/s | 279 kB 00:00 2024-04-25T10:28:19.099 INFO:teuthology.orchestra.run.smithi134.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 873 kB/s | 51 kB 00:00 2024-04-25T10:28:19.106 INFO:teuthology.orchestra.run.smithi106.stdout:(15/152): libcephsqlite-19.0.0-3291.g994e2f20.e 2.3 MB/s | 158 kB 00:00 2024-04-25T10:28:19.125 INFO:teuthology.orchestra.run.smithi134.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 916 kB/s | 54 kB 00:00 2024-04-25T10:28:19.156 INFO:teuthology.orchestra.run.smithi106.stdout:(16/152): librados-devel-19.0.0-3291.g994e2f20. 2.5 MB/s | 127 kB 00:00 2024-04-25T10:28:19.209 INFO:teuthology.orchestra.run.smithi134.stdout:(91/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 11 MB/s | 5.1 MB 00:00 2024-04-25T10:28:19.231 INFO:teuthology.orchestra.run.smithi106.stdout:(17/152): libradosstriper1-19.0.0-3291.g994e2f2 6.5 MB/s | 497 kB 00:00 2024-04-25T10:28:19.233 INFO:teuthology.orchestra.run.smithi134.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 384 kB/s | 42 kB 00:00 2024-04-25T10:28:19.267 INFO:teuthology.orchestra.run.smithi134.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 757 kB/s | 44 kB 00:00 2024-04-25T10:28:19.301 INFO:teuthology.orchestra.run.smithi134.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 3.2 MB/s | 221 kB 00:00 2024-04-25T10:28:19.326 INFO:teuthology.orchestra.run.smithi134.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 612 kB/s | 36 kB 00:00 2024-04-25T10:28:19.360 INFO:teuthology.orchestra.run.smithi134.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 810 kB/s | 47 kB 00:00 2024-04-25T10:28:19.394 INFO:teuthology.orchestra.run.smithi134.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 4.4 MB/s | 305 kB 00:00 2024-04-25T10:28:19.419 INFO:teuthology.orchestra.run.smithi134.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 986 kB/s | 58 kB 00:00 2024-04-25T10:28:19.452 INFO:teuthology.orchestra.run.smithi134.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.6 MB/s | 94 kB 00:00 2024-04-25T10:28:19.478 INFO:teuthology.orchestra.run.smithi134.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-04-25T10:28:19.511 INFO:teuthology.orchestra.run.smithi134.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 764 kB/s | 45 kB 00:00 2024-04-25T10:28:19.578 INFO:teuthology.orchestra.run.smithi134.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 222 kB/s | 22 kB 00:00 2024-04-25T10:28:19.673 INFO:teuthology.orchestra.run.smithi106.stdout:(18/152): librgw2-19.0.0-3291.g994e2f20.el9.x86 12 MB/s | 5.2 MB 00:00 2024-04-25T10:28:19.723 INFO:teuthology.orchestra.run.smithi106.stdout:(19/152): python3-ceph-argparse-19.0.0-3291.g99 907 kB/s | 45 kB 00:00 2024-04-25T10:28:19.729 INFO:teuthology.orchestra.run.smithi134.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 3.9 MB/s | 863 kB 00:00 2024-04-25T10:28:19.774 INFO:teuthology.orchestra.run.smithi106.stdout:(20/152): python3-ceph-common-19.0.0-3291.g994e 2.6 MB/s | 135 kB 00:00 2024-04-25T10:28:19.824 INFO:teuthology.orchestra.run.smithi106.stdout:(21/152): python3-cephfs-19.0.0-3291.g994e2f20. 3.2 MB/s | 161 kB 00:00 2024-04-25T10:28:19.891 INFO:teuthology.orchestra.run.smithi106.stdout:(22/152): python3-rados-19.0.0-3291.g994e2f20.e 4.7 MB/s | 321 kB 00:00 2024-04-25T10:28:19.904 INFO:teuthology.orchestra.run.smithi134.stdout:(104/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 20 MB/s | 16 MB 00:00 2024-04-25T10:28:19.950 INFO:teuthology.orchestra.run.smithi106.stdout:(23/152): python3-rbd-19.0.0-3291.g994e2f20.el9 5.0 MB/s | 298 kB 00:00 2024-04-25T10:28:19.988 INFO:teuthology.orchestra.run.smithi134.stdout:(105/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.3 MB/s | 548 kB 00:00 2024-04-25T10:28:20.016 INFO:teuthology.orchestra.run.smithi134.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 180 kB/s | 19 kB 00:00 2024-04-25T10:28:20.025 INFO:teuthology.orchestra.run.smithi106.stdout:(24/152): python3-rgw-19.0.0-3291.g994e2f20.el9 1.3 MB/s | 100 kB 00:00 2024-04-25T10:28:20.067 INFO:teuthology.orchestra.run.smithi106.stdout:(25/152): rbd-fuse-19.0.0-3291.g994e2f20.el9.x8 2.0 MB/s | 86 kB 00:00 2024-04-25T10:28:20.071 INFO:teuthology.orchestra.run.smithi134.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 445 kB/s | 26 kB 00:00 2024-04-25T10:28:20.121 INFO:teuthology.orchestra.run.smithi134.stdout:(108/152): gperftools-libs-2.9.1-2.el9.x86_64.r 787 kB/s | 309 kB 00:00 2024-04-25T10:28:20.147 INFO:teuthology.orchestra.run.smithi134.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 647 kB/s | 49 kB 00:00 2024-04-25T10:28:20.189 INFO:teuthology.orchestra.run.smithi134.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-04-25T10:28:20.247 INFO:teuthology.orchestra.run.smithi134.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.5 MB/s | 151 kB 00:00 2024-04-25T10:28:20.373 INFO:teuthology.orchestra.run.smithi134.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 4.3 MB/s | 548 kB 00:00 2024-04-25T10:28:20.375 INFO:teuthology.orchestra.run.smithi106.stdout:(26/152): rbd-mirror-19.0.0-3291.g994e2f20.el9. 9.9 MB/s | 3.0 MB 00:00 2024-04-25T10:28:20.426 INFO:teuthology.orchestra.run.smithi106.stdout:(27/152): rbd-nbd-19.0.0-3291.g994e2f20.el9.x86 3.3 MB/s | 169 kB 00:00 2024-04-25T10:28:20.431 INFO:teuthology.orchestra.run.smithi134.stdout:(113/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.4 MB/s | 838 kB 00:00 2024-04-25T10:28:20.457 INFO:teuthology.orchestra.run.smithi134.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 520 kB/s | 43 kB 00:00 2024-04-25T10:28:20.467 INFO:teuthology.orchestra.run.smithi106.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3291.g 824 kB/s | 34 kB 00:00 2024-04-25T10:28:20.518 INFO:teuthology.orchestra.run.smithi106.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3291.g994e2f2 2.8 MB/s | 142 kB 00:00 2024-04-25T10:28:20.540 INFO:teuthology.orchestra.run.smithi134.stdout:(115/152): libarrow-9.0.0-10.el9.x86_64.rpm 8.0 MB/s | 4.4 MB 00:00 2024-04-25T10:28:20.557 INFO:teuthology.orchestra.run.smithi134.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 149 kB/s | 15 kB 00:00 2024-04-25T10:28:20.574 INFO:teuthology.orchestra.run.smithi134.stdout:(117/152): python3-cachetools-4.2.4-1.el9.noarc 226 kB/s | 32 kB 00:00 2024-04-25T10:28:20.616 INFO:teuthology.orchestra.run.smithi134.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-04-25T10:28:20.641 INFO:teuthology.orchestra.run.smithi134.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-04-25T10:28:20.658 INFO:teuthology.orchestra.run.smithi134.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 850 kB/s | 35 kB 00:00 2024-04-25T10:28:20.683 INFO:teuthology.orchestra.run.smithi134.stdout:(121/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.7 MB/s | 303 kB 00:00 2024-04-25T10:28:20.708 INFO:teuthology.orchestra.run.smithi134.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 3.0 MB/s | 206 kB 00:00 2024-04-25T10:28:20.759 INFO:teuthology.orchestra.run.smithi134.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-04-25T10:28:20.784 INFO:teuthology.orchestra.run.smithi134.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 142 kB/s | 11 kB 00:00 2024-04-25T10:28:20.810 INFO:teuthology.orchestra.run.smithi134.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 350 kB/s | 18 kB 00:00 2024-04-25T10:28:20.835 INFO:teuthology.orchestra.run.smithi134.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 461 kB/s | 23 kB 00:00 2024-04-25T10:28:20.852 INFO:teuthology.orchestra.run.smithi106.stdout:(30/152): ceph-mgr-dashboard-19.0.0-3291.g994e2 11 MB/s | 3.5 MB 00:00 2024-04-25T10:28:20.860 INFO:teuthology.orchestra.run.smithi134.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 386 kB/s | 19 kB 00:00 2024-04-25T10:28:20.885 INFO:teuthology.orchestra.run.smithi134.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-04-25T10:28:20.927 INFO:teuthology.orchestra.run.smithi134.stdout:(129/152): python3-grpcio-1.46.7-10.el9.x86_64. 7.6 MB/s | 2.0 MB 00:00 2024-04-25T10:28:20.953 INFO:teuthology.orchestra.run.smithi134.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 692 kB/s | 46 kB 00:00 2024-04-25T10:28:20.978 INFO:teuthology.orchestra.run.smithi134.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-04-25T10:28:21.003 INFO:teuthology.orchestra.run.smithi134.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-04-25T10:28:21.053 INFO:teuthology.orchestra.run.smithi134.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-04-25T10:28:21.070 INFO:teuthology.orchestra.run.smithi134.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 245 kB/s | 16 kB 00:00 2024-04-25T10:28:21.104 INFO:teuthology.orchestra.run.smithi134.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-04-25T10:28:21.138 INFO:teuthology.orchestra.run.smithi134.stdout:(136/152): python3-kubernetes-26.1.0-2.el9.noar 3.7 MB/s | 1.0 MB 00:00 2024-04-25T10:28:21.154 INFO:teuthology.orchestra.run.smithi134.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 337 kB/s | 28 kB 00:00 2024-04-25T10:28:21.169 INFO:teuthology.orchestra.run.smithi106.stdout:(31/152): ceph-radosgw-19.0.0-3291.g994e2f20.el 2.5 MB/s | 10 MB 00:04 2024-04-25T10:28:21.173 INFO:teuthology.orchestra.run.smithi134.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 460 kB/s | 31 kB 00:00 2024-04-25T10:28:21.222 INFO:teuthology.orchestra.run.smithi134.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-04-25T10:28:21.238 INFO:teuthology.orchestra.run.smithi134.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 536 kB/s | 36 kB 00:00 2024-04-25T10:28:21.263 INFO:teuthology.orchestra.run.smithi134.stdout:(141/152): python3-rsa-4.9-2.el9.noarch.rpm 643 kB/s | 59 kB 00:00 2024-04-25T10:28:21.286 INFO:teuthology.orchestra.run.smithi106.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3291.g99 2.1 MB/s | 249 kB 00:00 2024-04-25T10:28:21.289 INFO:teuthology.orchestra.run.smithi134.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 762 kB/s | 51 kB 00:00 2024-04-25T10:28:21.314 INFO:teuthology.orchestra.run.smithi134.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-04-25T10:28:21.339 INFO:teuthology.orchestra.run.smithi134.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-04-25T10:28:21.361 INFO:teuthology.orchestra.run.smithi106.stdout:(33/152): ceph-mgr-rook-19.0.0-3291.g994e2f20.e 659 kB/s | 49 kB 00:00 2024-04-25T10:28:21.365 INFO:teuthology.orchestra.run.smithi134.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-04-25T10:28:21.390 INFO:teuthology.orchestra.run.smithi134.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-04-25T10:28:21.415 INFO:teuthology.orchestra.run.smithi134.stdout:(147/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-04-25T10:28:21.436 INFO:teuthology.orchestra.run.smithi106.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3291.g9 219 kB/s | 16 kB 00:00 2024-04-25T10:28:21.490 INFO:teuthology.orchestra.run.smithi134.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-04-25T10:28:21.562 INFO:teuthology.orchestra.run.smithi106.stdout:(35/152): ceph-volume-19.0.0-3291.g994e2f20.el9 2.1 MB/s | 273 kB 00:00 2024-04-25T10:28:21.646 INFO:teuthology.orchestra.run.smithi106.stdout:(36/152): ceph-mgr-diskprediction-local-19.0.0- 9.3 MB/s | 7.4 MB 00:00 2024-04-25T10:28:21.712 INFO:teuthology.orchestra.run.smithi106.stdout:(37/152): cephfs-top-19.0.0-3291.g994e2f20.el9. 275 kB/s | 18 kB 00:00 2024-04-25T10:28:21.741 INFO:teuthology.orchestra.run.smithi134.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 5.4 MB/s | 1.3 MB 00:00 2024-04-25T10:28:21.791 INFO:teuthology.orchestra.run.smithi134.stdout:(150/152): librbd1-19.0.0-3291.g994e2f20.el9.x8 8.2 MB/s | 3.1 MB 00:00 2024-04-25T10:28:21.933 INFO:teuthology.orchestra.run.smithi134.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 12 MB/s | 2.4 MB 00:00 2024-04-25T10:28:21.979 INFO:teuthology.orchestra.run.smithi106.stdout:(38/152): cephadm-19.0.0-3291.g994e2f20.el9.noa 1.8 MB/s | 762 kB 00:00 2024-04-25T10:28:21.983 INFO:teuthology.orchestra.run.smithi134.stdout:(152/152): librados2-19.0.0-3291.g994e2f20.el9. 5.6 MB/s | 3.3 MB 00:00 2024-04-25T10:28:21.986 INFO:teuthology.orchestra.run.smithi134.stdout:-------------------------------------------------------------------------------- 2024-04-25T10:28:21.986 INFO:teuthology.orchestra.run.smithi134.stdout:Total 18 MB/s | 214 MB 00:12 2024-04-25T10:28:22.129 INFO:teuthology.orchestra.run.smithi106.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 193 kB/s | 80 kB 00:00 2024-04-25T10:28:22.572 INFO:teuthology.orchestra.run.smithi106.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 442 kB/s | 195 kB 00:00 2024-04-25T10:28:22.872 INFO:teuthology.orchestra.run.smithi106.stdout:(41/152): libgfortran-11.4.1-3.el9.x86_64.rpm 900 kB/s | 803 kB 00:00 2024-04-25T10:28:23.072 INFO:teuthology.orchestra.run.smithi106.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 506 kB/s | 253 kB 00:00 2024-04-25T10:28:23.094 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction check 2024-04-25T10:28:23.237 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction check succeeded. 2024-04-25T10:28:23.237 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction test 2024-04-25T10:28:23.489 INFO:teuthology.orchestra.run.smithi106.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 256 kB/s | 106 kB 00:00 2024-04-25T10:28:23.948 INFO:teuthology.orchestra.run.smithi106.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 295 kB/s | 135 kB 00:00 2024-04-25T10:28:24.356 INFO:teuthology.orchestra.run.smithi106.stdout:(45/152): python3-cryptography-36.0.1-4.el9.x86 846 kB/s | 1.2 MB 00:01 2024-04-25T10:28:24.382 INFO:teuthology.orchestra.run.smithi106.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 288 kB/s | 125 kB 00:00 2024-04-25T10:28:24.640 INFO:teuthology.orchestra.run.smithi106.stdout:(47/152): automake-1.16.2-8.el9.noarch.rpm 2.6 MB/s | 689 kB 00:00 2024-04-25T10:28:24.948 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction test succeeded. 2024-04-25T10:28:24.948 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction 2024-04-25T10:28:25.082 INFO:teuthology.orchestra.run.smithi106.stdout:(48/152): ceph-test-19.0.0-3291.g994e2f20.el9.x 6.7 MB/s | 43 MB 00:06 2024-04-25T10:28:25.107 INFO:teuthology.orchestra.run.smithi106.stdout:(49/152): python3-urllib3-1.26.5-5.el9.noarch.r 286 kB/s | 215 kB 00:00 2024-04-25T10:28:25.132 INFO:teuthology.orchestra.run.smithi106.stdout:(50/152): bison-3.7.4-5.el9.x86_64.rpm 1.9 MB/s | 943 kB 00:00 2024-04-25T10:28:25.199 INFO:teuthology.orchestra.run.smithi106.stdout:(51/152): flex-2.6.4-9.el9.x86_64.rpm 4.6 MB/s | 313 kB 00:00 2024-04-25T10:28:25.242 INFO:teuthology.orchestra.run.smithi106.stdout:(52/152): elfutils-libelf-devel-0.190-2.el9.x86 160 kB/s | 21 kB 00:00 2024-04-25T10:28:25.267 INFO:teuthology.orchestra.run.smithi106.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 489 kB/s | 33 kB 00:00 2024-04-25T10:28:25.325 INFO:teuthology.orchestra.run.smithi106.stdout:(54/152): boost-program-options-1.75.0-8.el9.x8 440 kB/s | 107 kB 00:00 2024-04-25T10:28:25.350 INFO:teuthology.orchestra.run.smithi106.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 218 kB/s | 18 kB 00:00 2024-04-25T10:28:25.426 INFO:teuthology.orchestra.run.smithi106.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 111 kB/s | 11 kB 00:00 2024-04-25T10:28:25.451 INFO:teuthology.orchestra.run.smithi106.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.6 MB/s | 160 kB 00:00 2024-04-25T10:28:25.493 INFO:teuthology.orchestra.run.smithi106.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 677 kB/s | 45 kB 00:00 2024-04-25T10:28:25.543 INFO:teuthology.orchestra.run.smithi106.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 7.0 MB/s | 662 kB 00:00 2024-04-25T10:28:25.594 INFO:teuthology.orchestra.run.smithi106.stdout:(60/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 8.5 MB/s | 3.0 MB 00:00 2024-04-25T10:28:25.619 INFO:teuthology.orchestra.run.smithi106.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 338 kB/s | 25 kB 00:00 2024-04-25T10:28:25.669 INFO:teuthology.orchestra.run.smithi106.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 629 kB/s | 47 kB 00:00 2024-04-25T10:28:25.703 INFO:teuthology.orchestra.run.smithi106.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-04-25T10:28:25.728 INFO:teuthology.orchestra.run.smithi106.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.2 MB/s | 188 kB 00:00 2024-04-25T10:28:25.770 INFO:teuthology.orchestra.run.smithi106.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 527 kB/s | 35 kB 00:00 2024-04-25T10:28:25.912 INFO:teuthology.orchestra.run.smithi106.stdout:(66/152): libtool-2.4.6-45.el9.x86_64.rpm 1.4 MB/s | 582 kB 00:00 2024-04-25T10:28:26.046 INFO:teuthology.orchestra.run.smithi106.stdout:(67/152): openblas-openmp-0.3.21-2.el9.x86_64.r 15 MB/s | 4.7 MB 00:00 2024-04-25T10:28:26.146 INFO:teuthology.orchestra.run.smithi106.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 8.9 MB/s | 911 kB 00:00 2024-04-25T10:28:26.205 INFO:teuthology.orchestra.run.smithi106.stdout:(69/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 10 MB/s | 4.4 MB 00:00 2024-04-25T10:28:26.230 INFO:teuthology.orchestra.run.smithi106.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.6 MB/s | 133 kB 00:00 2024-04-25T10:28:26.305 INFO:teuthology.orchestra.run.smithi106.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 13 MB/s | 1.0 MB 00:00 2024-04-25T10:28:26.364 INFO:teuthology.orchestra.run.smithi106.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-04-25T10:28:26.631 INFO:teuthology.orchestra.run.smithi106.stdout:(73/152): postgresql-server-13.14-1.el9.x86_64. 14 MB/s | 5.8 MB 00:00 2024-04-25T10:28:26.709 INFO:teuthology.orchestra.run.smithi106.stdout:(74/152): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 6.0 MB 00:00 2024-04-25T10:28:26.731 INFO:teuthology.orchestra.run.smithi106.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 2.4 MB/s | 244 kB 00:00 2024-04-25T10:28:26.781 INFO:teuthology.orchestra.run.smithi106.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 3.2 MB/s | 248 kB 00:00 2024-04-25T10:28:26.798 INFO:teuthology.orchestra.run.smithi106.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 383 kB/s | 26 kB 00:00 2024-04-25T10:28:26.849 INFO:teuthology.orchestra.run.smithi106.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 286 kB/s | 19 kB 00:00 2024-04-25T10:28:26.874 INFO:teuthology.orchestra.run.smithi106.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-04-25T10:28:26.924 INFO:teuthology.orchestra.run.smithi106.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 463 kB/s | 35 kB 00:00 2024-04-25T10:28:27.025 INFO:teuthology.orchestra.run.smithi106.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.4 MB/s | 250 kB 00:00 2024-04-25T10:28:27.117 INFO:teuthology.orchestra.run.smithi106.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.4 MB/s | 222 kB 00:00 2024-04-25T10:28:27.217 INFO:teuthology.orchestra.run.smithi106.stdout:(83/152): postgresql-13.14-1.el9.x86_64.rpm 1.2 MB/s | 1.6 MB 00:01 2024-04-25T10:28:27.242 INFO:teuthology.orchestra.run.smithi106.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 617 kB/s | 77 kB 00:00 2024-04-25T10:28:27.317 INFO:teuthology.orchestra.run.smithi106.stdout:(85/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-04-25T10:28:27.342 INFO:teuthology.orchestra.run.smithi106.stdout:(86/152): python3-prettytable-0.7.2-27.el9.noar 337 kB/s | 42 kB 00:00 2024-04-25T10:28:27.368 INFO:teuthology.orchestra.run.smithi106.stdout:(87/152): python3-protobuf-3.14.0-13.el9.noarch 2.1 MB/s | 269 kB 00:00 2024-04-25T10:28:27.393 INFO:teuthology.orchestra.run.smithi106.stdout:(88/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-04-25T10:28:27.477 INFO:teuthology.orchestra.run.smithi106.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 470 kB/s | 51 kB 00:00 2024-04-25T10:28:27.502 INFO:teuthology.orchestra.run.smithi106.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 494 kB/s | 54 kB 00:00 2024-04-25T10:28:27.586 INFO:teuthology.orchestra.run.smithi106.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 499 kB/s | 42 kB 00:00 2024-04-25T10:28:27.611 INFO:teuthology.orchestra.run.smithi106.stdout:(92/152): python3-pyasn1-modules-0.4.8-6.el9.no 1.0 MB/s | 279 kB 00:00 2024-04-25T10:28:27.645 INFO:teuthology.orchestra.run.smithi106.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 757 kB/s | 44 kB 00:00 2024-04-25T10:28:27.703 INFO:teuthology.orchestra.run.smithi106.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 618 kB/s | 36 kB 00:00 2024-04-25T10:28:27.762 INFO:teuthology.orchestra.run.smithi106.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 812 kB/s | 47 kB 00:00 2024-04-25T10:28:27.787 INFO:teuthology.orchestra.run.smithi106.stdout:(96/152): qatlib-24.02.0-1.el9.x86_64.rpm 1.2 MB/s | 221 kB 00:00 2024-04-25T10:28:27.821 INFO:teuthology.orchestra.run.smithi106.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.1 MB/s | 305 kB 00:00 2024-04-25T10:28:27.846 INFO:teuthology.orchestra.run.smithi106.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 987 kB/s | 58 kB 00:00 2024-04-25T10:28:27.879 INFO:teuthology.orchestra.run.smithi106.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.6 MB/s | 94 kB 00:00 2024-04-25T10:28:27.913 INFO:teuthology.orchestra.run.smithi106.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 945 kB/s | 64 kB 00:00 2024-04-25T10:28:27.947 INFO:teuthology.orchestra.run.smithi106.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 671 kB/s | 45 kB 00:00 2024-04-25T10:28:28.139 INFO:teuthology.orchestra.run.smithi106.stdout:(102/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 25 MB/s | 16 MB 00:00 2024-04-25T10:28:28.164 INFO:teuthology.orchestra.run.smithi106.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 89 kB/s | 22 kB 00:00 2024-04-25T10:28:28.298 INFO:teuthology.orchestra.run.smithi106.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 2.4 MB/s | 863 kB 00:00 2024-04-25T10:28:28.531 INFO:teuthology.orchestra.run.smithi106.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 84 kB/s | 19 kB 00:00 2024-04-25T10:28:28.598 INFO:teuthology.orchestra.run.smithi106.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 712 kB/s | 309 kB 00:00 2024-04-25T10:28:28.624 INFO:teuthology.orchestra.run.smithi106.stdout:(107/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.1 MB/s | 548 kB 00:00 2024-04-25T10:28:28.782 INFO:teuthology.orchestra.run.smithi106.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 142 kB/s | 26 kB 00:00 2024-04-25T10:28:28.799 INFO:teuthology.orchestra.run.smithi106.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 278 kB/s | 49 kB 00:00 2024-04-25T10:28:28.866 INFO:teuthology.orchestra.run.smithi106.stdout:(110/152): libarrow-9.0.0-10.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-04-25T10:28:28.891 INFO:teuthology.orchestra.run.smithi106.stdout:(111/152): libunwind-1.6.2-1.el9.x86_64.rpm 619 kB/s | 67 kB 00:00 2024-04-25T10:28:28.916 INFO:teuthology.orchestra.run.smithi106.stdout:(112/152): luarocks-3.9.2-1.el9.noarch.rpm 1.3 MB/s | 151 kB 00:00 2024-04-25T10:28:28.958 INFO:teuthology.orchestra.run.smithi106.stdout:(113/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 8.9 MB/s | 838 kB 00:00 2024-04-25T10:28:28.992 INFO:teuthology.orchestra.run.smithi106.stdout:(114/152): python3-asyncssh-2.13.2-5.el9.noarch 7.3 MB/s | 548 kB 00:00 2024-04-25T10:28:29.017 INFO:teuthology.orchestra.run.smithi106.stdout:(115/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 439 kB/s | 43 kB 00:00 2024-04-25T10:28:29.042 INFO:teuthology.orchestra.run.smithi106.stdout:(116/152): python3-cachetools-4.2.4-1.el9.noarc 384 kB/s | 32 kB 00:00 2024-04-25T10:28:29.068 INFO:teuthology.orchestra.run.smithi106.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 199 kB/s | 15 kB 00:00 2024-04-25T10:28:29.101 INFO:teuthology.orchestra.run.smithi106.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.0 MB/s | 172 kB 00:00 2024-04-25T10:28:29.127 INFO:teuthology.orchestra.run.smithi106.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-04-25T10:28:29.152 INFO:teuthology.orchestra.run.smithi106.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.6 MB/s | 303 kB 00:00 2024-04-25T10:28:29.177 INFO:teuthology.orchestra.run.smithi106.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 471 kB/s | 35 kB 00:00 2024-04-25T10:28:29.203 INFO:teuthology.orchestra.run.smithi106.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 2.7 MB/s | 206 kB 00:00 2024-04-25T10:28:29.280 INFO:teuthology.orchestra.run.smithi106.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 16 MB/s | 2.0 MB 00:00 2024-04-25T10:28:29.311 INFO:teuthology.orchestra.run.smithi106.stdout:(124/152): python3-grpcio-tools-1.46.7-10.el9.x 1.1 MB/s | 144 kB 00:00 2024-04-25T10:28:29.336 INFO:teuthology.orchestra.run.smithi106.stdout:(125/152): python3-jaraco-8.2.1-3.el9.noarch.rp 80 kB/s | 11 kB 00:00 2024-04-25T10:28:29.362 INFO:teuthology.orchestra.run.smithi106.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 218 kB/s | 18 kB 00:00 2024-04-25T10:28:29.404 INFO:teuthology.orchestra.run.smithi106.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 347 kB/s | 23 kB 00:00 2024-04-25T10:28:29.429 INFO:teuthology.orchestra.run.smithi106.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 212 kB/s | 19 kB 00:00 2024-04-25T10:28:29.454 INFO:teuthology.orchestra.run.smithi106.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 213 kB/s | 20 kB 00:00 2024-04-25T10:28:29.521 INFO:teuthology.orchestra.run.smithi106.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 11 MB/s | 1.0 MB 00:00 2024-04-25T10:28:29.546 INFO:teuthology.orchestra.run.smithi106.stdout:(131/152): python3-logutils-0.3.5-21.el9.noarch 396 kB/s | 46 kB 00:00 2024-04-25T10:28:29.572 INFO:teuthology.orchestra.run.smithi106.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 672 kB/s | 79 kB 00:00 2024-04-25T10:28:29.597 INFO:teuthology.orchestra.run.smithi106.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 765 kB/s | 58 kB 00:00 2024-04-25T10:28:29.647 INFO:teuthology.orchestra.run.smithi106.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-04-25T10:28:29.672 INFO:teuthology.orchestra.run.smithi106.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 164 kB/s | 16 kB 00:00 2024-04-25T10:28:29.689 INFO:teuthology.orchestra.run.smithi106.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 977 kB/s | 90 kB 00:00 2024-04-25T10:28:29.714 INFO:teuthology.orchestra.run.smithi106.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 424 kB/s | 28 kB 00:00 2024-04-25T10:28:29.757 INFO:teuthology.orchestra.run.smithi106.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 460 kB/s | 31 kB 00:00 2024-04-25T10:28:29.782 INFO:teuthology.orchestra.run.smithi106.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 2.0 MB/s | 188 kB 00:00 2024-04-25T10:28:29.807 INFO:teuthology.orchestra.run.smithi106.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 641 kB/s | 59 kB 00:00 2024-04-25T10:28:29.832 INFO:teuthology.orchestra.run.smithi106.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 476 kB/s | 36 kB 00:00 2024-04-25T10:28:29.858 INFO:teuthology.orchestra.run.smithi106.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 677 kB/s | 51 kB 00:00 2024-04-25T10:28:29.883 INFO:teuthology.orchestra.run.smithi106.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-04-25T10:28:29.908 INFO:teuthology.orchestra.run.smithi106.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-04-25T10:28:29.933 INFO:teuthology.orchestra.run.smithi106.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 5.6 MB/s | 427 kB 00:00 2024-04-25T10:28:29.959 INFO:teuthology.orchestra.run.smithi106.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 265 kB/s | 20 kB 00:00 2024-04-25T10:28:29.984 INFO:teuthology.orchestra.run.smithi106.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-04-25T10:28:30.034 INFO:teuthology.orchestra.run.smithi106.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-04-25T10:28:30.326 INFO:teuthology.orchestra.run.smithi106.stdout:(149/152): librbd1-19.0.0-3291.g994e2f20.el9.x8 9.1 MB/s | 3.1 MB 00:00 2024-04-25T10:28:30.818 INFO:teuthology.orchestra.run.smithi134.stdout: Preparing : 1/1 2024-04-25T10:28:30.903 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T10:28:30.942 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T10:28:30.960 INFO:teuthology.orchestra.run.smithi106.stdout:(150/152): librados2-19.0.0-3291.g994e2f20.el9. 3.3 MB/s | 3.3 MB 00:01 2024-04-25T10:28:30.995 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T10:28:31.029 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T10:28:31.055 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T10:28:31.097 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T10:28:31.410 INFO:teuthology.orchestra.run.smithi106.stdout:(151/152): openssl-3.2.1-1.el9.x86_64.rpm 1.0 MB/s | 1.3 MB 00:01 2024-04-25T10:28:31.432 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T10:28:31.547 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T10:28:31.672 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T10:28:31.706 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T10:28:31.806 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T10:28:31.836 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T10:28:31.982 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T10:28:32.033 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T10:28:32.061 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T10:28:32.166 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T10:28:32.204 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T10:28:32.258 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T10:28:32.279 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T10:28:32.298 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T10:28:32.446 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T10:28:32.487 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T10:28:32.594 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T10:28:32.689 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T10:28:32.728 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T10:28:32.762 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T10:28:32.827 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T10:28:32.870 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T10:28:32.914 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T10:28:32.946 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T10:28:32.974 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T10:28:33.042 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T10:28:33.095 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T10:28:33.127 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T10:28:33.162 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T10:28:33.176 INFO:teuthology.orchestra.run.smithi106.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 860 kB/s | 2.4 MB 00:02 2024-04-25T10:28:33.183 INFO:teuthology.orchestra.run.smithi106.stdout:-------------------------------------------------------------------------------- 2024-04-25T10:28:33.183 INFO:teuthology.orchestra.run.smithi106.stdout:Total 11 MB/s | 214 MB 00:19 2024-04-25T10:28:33.238 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T10:28:33.273 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 31/156 2024-04-25T10:28:33.308 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 32/156 2024-04-25T10:28:33.345 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 33/156 2024-04-25T10:28:33.414 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T10:28:33.459 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T10:28:33.533 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T10:28:33.665 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T10:28:33.720 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T10:28:34.233 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T10:28:34.257 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction check 2024-04-25T10:28:34.315 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T10:28:34.345 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T10:28:34.387 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T10:28:34.401 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction check succeeded. 2024-04-25T10:28:34.402 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction test 2024-04-25T10:28:34.426 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 43/156 2024-04-25T10:28:34.484 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T10:28:34.628 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T10:28:34.678 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T10:28:34.734 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T10:28:34.805 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T10:28:34.837 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T10:28:34.874 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T10:28:34.921 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T10:28:34.965 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T10:28:35.007 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T10:28:35.050 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T10:28:35.185 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T10:28:35.231 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T10:28:35.261 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T10:28:35.301 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T10:28:35.395 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T10:28:35.427 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T10:28:35.467 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T10:28:35.493 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T10:28:35.517 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T10:28:35.591 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T10:28:35.717 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T10:28:35.749 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T10:28:35.778 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T10:28:35.812 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T10:28:35.851 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T10:28:35.882 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T10:28:35.912 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T10:28:35.947 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T10:28:35.980 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T10:28:36.021 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T10:28:36.064 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T10:28:36.118 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction test succeeded. 2024-04-25T10:28:36.119 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction 2024-04-25T10:28:36.150 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T10:28:36.195 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T10:28:36.234 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T10:28:36.281 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T10:28:36.379 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T10:28:36.561 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T10:28:37.276 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T10:28:37.349 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T10:28:37.383 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T10:28:37.481 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T10:28:37.533 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T10:28:37.619 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T10:28:37.660 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T10:28:37.837 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T10:28:37.866 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T10:28:37.953 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T10:28:38.582 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T10:28:38.696 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T10:28:39.915 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T10:28:40.003 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T10:28:40.030 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T10:28:40.065 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T10:28:40.383 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T10:28:40.419 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T10:28:40.501 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T10:28:40.532 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 100/156 2024-04-25T10:28:40.567 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T10:28:41.058 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T10:28:41.213 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T10:28:41.284 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T10:28:41.306 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 104/156 2024-04-25T10:28:42.034 INFO:teuthology.orchestra.run.smithi106.stdout: Preparing : 1/1 2024-04-25T10:28:42.114 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T10:28:42.151 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T10:28:42.205 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T10:28:42.248 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T10:28:42.273 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T10:28:42.316 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T10:28:42.642 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T10:28:42.758 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T10:28:42.883 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T10:28:42.909 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T10:28:42.983 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T10:28:43.013 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T10:28:43.160 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T10:28:43.210 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T10:28:43.238 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T10:28:43.343 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T10:28:43.372 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T10:28:43.435 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T10:28:43.457 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T10:28:43.481 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T10:28:43.481 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T10:28:43.528 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T10:28:43.674 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T10:28:43.706 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T10:28:43.821 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T10:28:43.913 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T10:28:43.948 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T10:28:44.000 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T10:28:44.074 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T10:28:44.084 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T10:28:44.116 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T10:28:44.131 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T10:28:44.160 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T10:28:44.192 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T10:28:44.220 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T10:28:44.220 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T10:28:44.220 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T10:28:44.220 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T10:28:44.221 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T10:28:44.249 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T10:28:44.290 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T10:28:44.342 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T10:28:44.373 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T10:28:44.416 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T10:28:44.493 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T10:28:44.527 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 31/156 2024-04-25T10:28:44.570 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 32/156 2024-04-25T10:28:44.599 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 33/156 2024-04-25T10:28:44.669 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T10:28:44.714 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T10:28:44.789 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T10:28:44.925 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T10:28:44.974 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T10:28:45.492 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T10:28:45.570 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T10:28:45.598 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T10:28:45.633 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T10:28:45.672 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 43/156 2024-04-25T10:28:45.714 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T10:28:45.868 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T10:28:45.916 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T10:28:45.971 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T10:28:46.044 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T10:28:46.075 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T10:28:46.111 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T10:28:46.158 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T10:28:46.202 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T10:28:46.263 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T10:28:46.305 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T10:28:46.442 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T10:28:46.485 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T10:28:46.516 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T10:28:46.557 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T10:28:46.651 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T10:28:46.690 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T10:28:46.722 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T10:28:46.747 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T10:28:46.772 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T10:28:46.847 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T10:28:46.974 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T10:28:47.005 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T10:28:47.032 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T10:28:47.067 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T10:28:47.106 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T10:28:47.146 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T10:28:47.178 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T10:28:47.211 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T10:28:47.244 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T10:28:47.277 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T10:28:47.312 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T10:28:47.399 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T10:28:47.492 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T10:28:47.523 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T10:28:47.571 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T10:28:47.669 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T10:28:47.725 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T10:28:48.461 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T10:28:48.505 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T10:28:48.539 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T10:28:48.636 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T10:28:48.688 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T10:28:48.786 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T10:28:48.824 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T10:28:49.000 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T10:28:49.030 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T10:28:49.107 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T10:28:49.744 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T10:28:49.862 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T10:28:51.085 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T10:28:51.159 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T10:28:51.187 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T10:28:51.220 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T10:28:51.539 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T10:28:51.583 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T10:28:51.674 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T10:28:51.704 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 100/156 2024-04-25T10:28:51.740 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T10:28:52.218 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T10:28:52.244 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T10:28:52.307 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T10:28:52.329 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 104/156 2024-04-25T10:28:54.561 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T10:28:54.592 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T10:28:55.154 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T10:28:55.196 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T10:28:55.282 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T10:28:55.282 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T10:28:55.283 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T10:28:55.283 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:28:55.306 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T10:28:55.633 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T10:28:55.633 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /sys 2024-04-25T10:28:55.633 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /proc 2024-04-25T10:28:55.633 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /mnt 2024-04-25T10:28:55.633 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /var/tmp 2024-04-25T10:28:55.633 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /home 2024-04-25T10:28:55.633 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /root 2024-04-25T10:28:55.633 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /tmp 2024-04-25T10:28:55.633 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T10:28:55.802 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T10:28:55.818 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T10:28:55.818 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:28:55.818 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T10:28:55.818 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T10:28:57.389 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T10:28:57.414 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T10:28:57.414 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:28:57.414 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T10:28:57.414 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T10:28:57.414 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T10:28:57.414 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T10:28:57.639 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T10:28:57.661 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T10:28:57.661 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:28:57.662 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T10:28:57.662 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T10:28:57.662 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T10:28:57.662 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T10:28:58.123 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T10:28:58.145 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T10:28:58.145 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:28:58.145 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T10:28:58.146 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T10:28:58.146 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T10:28:58.146 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T10:28:58.261 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T10:28:58.318 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T10:28:58.373 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T10:28:58.558 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T10:28:58.607 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T10:28:58.676 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T10:28:58.764 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T10:28:58.894 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T10:28:58.940 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T10:28:58.972 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T10:28:59.000 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T10:28:59.438 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T10:28:59.465 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T10:28:59.497 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 124/156 2024-04-25T10:28:59.558 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 125/156 2024-04-25T10:28:59.596 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T10:29:00.639 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T10:29:00.681 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T10:29:01.653 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T10:29:01.681 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T10:29:01.783 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T10:29:01.948 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 129/156 2024-04-25T10:29:01.975 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T10:29:01.998 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T10:29:01.999 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:29:01.999 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T10:29:01.999 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T10:29:01.999 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T10:29:01.999 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T10:29:02.043 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T10:29:02.053 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T10:29:02.152 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 132/156 2024-04-25T10:29:02.199 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 133/156 2024-04-25T10:29:03.135 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T10:29:03.162 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T10:29:03.185 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T10:29:03.185 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:29:03.185 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T10:29:03.185 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T10:29:03.185 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T10:29:03.185 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T10:29:03.229 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T10:29:03.250 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T10:29:03.251 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:29:03.251 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T10:29:03.251 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T10:29:03.296 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T10:29:03.318 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T10:29:03.318 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:29:03.318 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T10:29:03.319 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T10:29:03.319 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T10:29:03.319 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T10:29:03.660 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T10:29:03.682 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T10:29:03.683 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:29:03.683 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T10:29:03.683 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T10:29:03.683 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T10:29:03.683 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T10:29:06.788 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T10:29:06.788 INFO:teuthology.orchestra.run.smithi106.stdout:skipping the directory /sys 2024-04-25T10:29:06.788 INFO:teuthology.orchestra.run.smithi106.stdout:skipping the directory /proc 2024-04-25T10:29:06.789 INFO:teuthology.orchestra.run.smithi106.stdout:skipping the directory /mnt 2024-04-25T10:29:06.789 INFO:teuthology.orchestra.run.smithi106.stdout:skipping the directory /var/tmp 2024-04-25T10:29:06.789 INFO:teuthology.orchestra.run.smithi106.stdout:skipping the directory /home 2024-04-25T10:29:06.789 INFO:teuthology.orchestra.run.smithi106.stdout:skipping the directory /root 2024-04-25T10:29:06.789 INFO:teuthology.orchestra.run.smithi106.stdout:skipping the directory /tmp 2024-04-25T10:29:06.789 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:29:06.959 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T10:29:06.974 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T10:29:06.974 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:29:06.974 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T10:29:06.974 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:29:07.934 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 139/156 2024-04-25T10:29:07.975 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 140/156 2024-04-25T10:29:08.015 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 141/156 2024-04-25T10:29:08.048 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T10:29:08.075 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T10:29:08.250 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 144/156 2024-04-25T10:29:08.288 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T10:29:08.326 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 146/156 2024-04-25T10:29:08.501 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T10:29:08.527 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T10:29:08.527 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:29:08.527 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T10:29:08.527 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T10:29:08.527 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T10:29:08.527 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:29:08.752 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T10:29:08.774 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T10:29:08.774 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:29:08.774 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T10:29:08.774 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T10:29:08.775 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T10:29:08.775 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:29:08.899 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T10:29:08.940 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T10:29:08.959 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T10:29:09.236 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T10:29:09.258 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T10:29:09.258 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:29:09.258 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T10:29:09.258 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T10:29:09.258 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T10:29:09.258 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:29:09.366 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T10:29:09.414 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T10:29:09.470 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T10:29:09.664 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T10:29:09.712 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T10:29:09.781 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 148/156 2024-04-25T10:29:09.788 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T10:29:09.878 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T10:29:09.878 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T10:29:09.919 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T10:29:10.008 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T10:29:10.015 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T10:29:10.048 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T10:29:10.049 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T10:29:10.061 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T10:29:10.094 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T10:29:10.132 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T10:29:10.146 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T10:29:10.165 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T10:29:10.165 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T10:29:10.254 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T10:29:10.322 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T10:29:10.597 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T10:29:10.637 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T10:29:10.669 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 124/156 2024-04-25T10:29:10.731 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 125/156 2024-04-25T10:29:10.760 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T10:29:11.790 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T10:29:11.828 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T10:29:12.785 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T10:29:12.814 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T10:29:12.912 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T10:29:13.077 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 129/156 2024-04-25T10:29:13.105 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T10:29:13.128 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T10:29:13.128 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:29:13.128 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T10:29:13.128 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T10:29:13.128 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T10:29:13.128 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:29:13.173 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T10:29:13.184 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T10:29:13.282 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 132/156 2024-04-25T10:29:13.330 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 133/156 2024-04-25T10:29:14.315 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T10:29:14.353 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T10:29:14.377 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T10:29:14.377 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:29:14.377 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T10:29:14.377 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T10:29:14.377 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T10:29:14.377 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:29:14.420 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T10:29:14.442 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T10:29:14.442 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:29:14.442 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T10:29:14.442 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:29:14.496 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T10:29:14.519 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T10:29:14.519 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:29:14.519 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T10:29:14.519 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T10:29:14.519 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T10:29:14.519 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:29:14.840 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T10:29:14.862 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T10:29:14.862 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:29:14.862 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T10:29:14.863 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T10:29:14.863 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T10:29:14.863 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:29:16.212 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T10:29:16.212 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 1/156 2024-04-25T10:29:16.212 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2/156 2024-04-25T10:29:16.212 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 3/156 2024-04-25T10:29:16.212 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 4/156 2024-04-25T10:29:16.212 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 5/156 2024-04-25T10:29:16.212 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 6/156 2024-04-25T10:29:16.213 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 7/156 2024-04-25T10:29:16.213 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T10:29:16.213 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T10:29:16.213 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T10:29:16.213 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 11/156 2024-04-25T10:29:16.213 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 12/156 2024-04-25T10:29:16.213 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T10:29:16.213 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 14/156 2024-04-25T10:29:16.213 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 15/156 2024-04-25T10:29:16.213 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 16/156 2024-04-25T10:29:16.213 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 17/156 2024-04-25T10:29:16.213 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 18/156 2024-04-25T10:29:16.213 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 19/156 2024-04-25T10:29:16.213 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 20/156 2024-04-25T10:29:16.214 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 21/156 2024-04-25T10:29:16.214 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 22/156 2024-04-25T10:29:16.214 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 23/156 2024-04-25T10:29:16.214 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 24/156 2024-04-25T10:29:16.214 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 25/156 2024-04-25T10:29:16.214 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 26/156 2024-04-25T10:29:16.214 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 27/156 2024-04-25T10:29:16.214 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 28/156 2024-04-25T10:29:16.214 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 29/156 2024-04-25T10:29:16.227 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 30/156 2024-04-25T10:29:16.227 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 31/156 2024-04-25T10:29:16.227 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 32/156 2024-04-25T10:29:16.228 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 33/156 2024-04-25T10:29:16.228 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 34/156 2024-04-25T10:29:16.228 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 35/156 2024-04-25T10:29:16.228 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 36/156 2024-04-25T10:29:16.228 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 37/156 2024-04-25T10:29:16.228 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 38/156 2024-04-25T10:29:16.228 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 39/156 2024-04-25T10:29:16.228 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T10:29:16.228 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T10:29:16.228 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T10:29:16.228 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T10:29:16.229 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T10:29:16.229 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T10:29:16.229 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T10:29:16.229 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T10:29:16.229 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T10:29:16.229 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T10:29:16.229 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T10:29:16.229 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T10:29:16.229 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T10:29:16.229 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T10:29:16.229 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T10:29:16.229 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T10:29:16.229 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T10:29:16.230 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T10:29:16.230 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T10:29:16.230 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T10:29:16.230 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T10:29:16.230 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T10:29:16.230 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T10:29:16.230 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T10:29:16.230 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T10:29:16.230 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T10:29:16.231 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T10:29:16.231 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T10:29:16.231 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T10:29:16.231 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T10:29:16.231 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T10:29:16.231 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T10:29:16.231 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T10:29:16.231 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T10:29:16.231 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T10:29:16.231 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T10:29:16.231 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T10:29:16.231 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T10:29:16.232 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T10:29:16.232 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T10:29:16.232 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T10:29:16.232 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T10:29:16.232 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T10:29:16.232 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T10:29:16.232 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T10:29:16.232 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T10:29:16.232 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T10:29:16.232 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T10:29:16.232 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T10:29:16.232 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T10:29:16.232 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T10:29:16.233 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T10:29:16.233 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T10:29:16.233 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T10:29:16.233 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T10:29:16.233 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T10:29:16.233 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T10:29:16.233 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T10:29:16.233 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T10:29:16.233 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T10:29:16.233 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T10:29:16.233 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T10:29:16.233 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T10:29:16.233 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T10:29:16.234 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T10:29:16.234 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T10:29:16.234 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T10:29:16.234 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T10:29:16.234 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T10:29:16.234 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T10:29:16.234 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T10:29:16.234 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T10:29:16.234 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T10:29:16.234 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T10:29:16.234 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T10:29:16.235 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T10:29:16.235 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T10:29:16.235 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T10:29:16.235 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T10:29:16.235 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T10:29:16.235 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T10:29:16.235 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T10:29:16.235 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T10:29:16.235 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T10:29:16.235 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T10:29:16.235 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T10:29:16.235 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T10:29:16.235 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T10:29:16.236 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T10:29:16.236 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T10:29:16.236 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T10:29:16.236 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T10:29:16.236 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T10:29:16.236 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T10:29:16.236 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T10:29:16.236 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T10:29:16.236 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T10:29:16.236 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T10:29:16.236 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T10:29:16.236 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T10:29:16.236 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T10:29:16.236 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T10:29:16.236 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T10:29:16.237 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T10:29:16.237 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T10:29:16.237 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T10:29:16.237 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T10:29:16.237 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T10:29:16.237 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T10:29:16.237 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 149/156 2024-04-25T10:29:16.237 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T10:29:16.237 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 151/156 2024-04-25T10:29:16.237 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T10:29:16.237 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T10:29:16.237 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T10:29:16.237 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T10:29:18.264 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T10:29:18.264 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T10:29:18.264 INFO:teuthology.orchestra.run.smithi134.stdout:Upgraded: 2024-04-25T10:29:18.264 INFO:teuthology.orchestra.run.smithi134.stdout: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:29:18.264 INFO:teuthology.orchestra.run.smithi134.stdout: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:29:18.264 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T10:29:18.264 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T10:29:18.264 INFO:teuthology.orchestra.run.smithi134.stdout:Installed: 2024-04-25T10:29:18.264 INFO:teuthology.orchestra.run.smithi134.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T10:29:18.264 INFO:teuthology.orchestra.run.smithi134.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T10:29:18.264 INFO:teuthology.orchestra.run.smithi134.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T10:29:18.264 INFO:teuthology.orchestra.run.smithi134.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T10:29:18.265 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:29:18.265 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:29:18.265 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:29:18.265 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:29:18.265 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:29:18.265 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-immutable-object-cache-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:29:18.265 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:29:18.265 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:29:18.265 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:29:18.265 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:29:18.265 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:29:18.265 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:29:18.265 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:29:18.265 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:29:18.265 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:29:18.266 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:29:18.266 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:29:18.266 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:29:18.266 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:29:18.266 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:29:18.266 INFO:teuthology.orchestra.run.smithi134.stdout: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:29:18.266 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:29:18.266 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:29:18.266 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:29:18.266 INFO:teuthology.orchestra.run.smithi134.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T10:29:18.266 INFO:teuthology.orchestra.run.smithi134.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T10:29:18.266 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T10:29:18.266 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T10:29:18.267 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T10:29:18.267 INFO:teuthology.orchestra.run.smithi134.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T10:29:18.267 INFO:teuthology.orchestra.run.smithi134.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T10:29:18.267 INFO:teuthology.orchestra.run.smithi134.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T10:29:18.267 INFO:teuthology.orchestra.run.smithi134.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T10:29:18.267 INFO:teuthology.orchestra.run.smithi134.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T10:29:18.267 INFO:teuthology.orchestra.run.smithi134.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T10:29:18.267 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:29:18.267 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:29:18.267 INFO:teuthology.orchestra.run.smithi134.stdout: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:29:18.267 INFO:teuthology.orchestra.run.smithi134.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T10:29:18.267 INFO:teuthology.orchestra.run.smithi134.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T10:29:18.267 INFO:teuthology.orchestra.run.smithi134.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T10:29:18.267 INFO:teuthology.orchestra.run.smithi134.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T10:29:18.267 INFO:teuthology.orchestra.run.smithi134.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T10:29:18.268 INFO:teuthology.orchestra.run.smithi134.stdout: librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:29:18.268 INFO:teuthology.orchestra.run.smithi134.stdout: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:29:18.268 INFO:teuthology.orchestra.run.smithi134.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T10:29:18.268 INFO:teuthology.orchestra.run.smithi134.stdout: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:29:18.268 INFO:teuthology.orchestra.run.smithi134.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T10:29:18.268 INFO:teuthology.orchestra.run.smithi134.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T10:29:18.268 INFO:teuthology.orchestra.run.smithi134.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T10:29:18.268 INFO:teuthology.orchestra.run.smithi134.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T10:29:18.268 INFO:teuthology.orchestra.run.smithi134.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T10:29:18.268 INFO:teuthology.orchestra.run.smithi134.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T10:29:18.268 INFO:teuthology.orchestra.run.smithi134.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T10:29:18.268 INFO:teuthology.orchestra.run.smithi134.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T10:29:18.268 INFO:teuthology.orchestra.run.smithi134.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T10:29:18.269 INFO:teuthology.orchestra.run.smithi134.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T10:29:18.269 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T10:29:18.269 INFO:teuthology.orchestra.run.smithi134.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T10:29:18.269 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T10:29:18.269 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T10:29:18.269 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T10:29:18.269 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T10:29:18.269 INFO:teuthology.orchestra.run.smithi134.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T10:29:18.269 INFO:teuthology.orchestra.run.smithi134.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T10:29:18.269 INFO:teuthology.orchestra.run.smithi134.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T10:29:18.269 INFO:teuthology.orchestra.run.smithi134.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T10:29:18.269 INFO:teuthology.orchestra.run.smithi134.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T10:29:18.269 INFO:teuthology.orchestra.run.smithi134.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T10:29:18.269 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T10:29:18.269 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:29:18.269 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:29:18.270 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:29:18.270 INFO:teuthology.orchestra.run.smithi134.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T10:29:18.270 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T10:29:18.270 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T10:29:18.270 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T10:29:18.270 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T10:29:18.270 INFO:teuthology.orchestra.run.smithi134.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T10:29:18.270 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T10:29:18.270 INFO:teuthology.orchestra.run.smithi134.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T10:29:18.270 INFO:teuthology.orchestra.run.smithi134.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T10:29:18.270 INFO:teuthology.orchestra.run.smithi134.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T10:29:18.270 INFO:teuthology.orchestra.run.smithi134.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T10:29:18.270 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T10:29:18.270 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T10:29:18.271 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T10:29:18.271 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T10:29:18.271 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T10:29:18.271 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T10:29:18.271 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T10:29:18.271 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T10:29:18.271 INFO:teuthology.orchestra.run.smithi134.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T10:29:18.271 INFO:teuthology.orchestra.run.smithi134.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T10:29:18.271 INFO:teuthology.orchestra.run.smithi134.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T10:29:18.271 INFO:teuthology.orchestra.run.smithi134.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T10:29:18.271 INFO:teuthology.orchestra.run.smithi134.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T10:29:18.271 INFO:teuthology.orchestra.run.smithi134.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T10:29:18.271 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T10:29:18.271 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T10:29:18.272 INFO:teuthology.orchestra.run.smithi134.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T10:29:18.272 INFO:teuthology.orchestra.run.smithi134.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T10:29:18.272 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T10:29:18.272 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T10:29:18.272 INFO:teuthology.orchestra.run.smithi134.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T10:29:18.272 INFO:teuthology.orchestra.run.smithi134.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T10:29:18.272 INFO:teuthology.orchestra.run.smithi134.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T10:29:18.272 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T10:29:18.272 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T10:29:18.272 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T10:29:18.272 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T10:29:18.272 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T10:29:18.273 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T10:29:18.273 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:29:18.273 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:29:18.273 INFO:teuthology.orchestra.run.smithi134.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T10:29:18.273 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T10:29:18.273 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T10:29:18.273 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:29:18.273 INFO:teuthology.orchestra.run.smithi134.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T10:29:18.273 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T10:29:18.273 INFO:teuthology.orchestra.run.smithi134.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T10:29:18.273 INFO:teuthology.orchestra.run.smithi134.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T10:29:18.273 INFO:teuthology.orchestra.run.smithi134.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T10:29:18.273 INFO:teuthology.orchestra.run.smithi134.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T10:29:18.273 INFO:teuthology.orchestra.run.smithi134.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T10:29:18.274 INFO:teuthology.orchestra.run.smithi134.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T10:29:18.274 INFO:teuthology.orchestra.run.smithi134.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T10:29:18.274 INFO:teuthology.orchestra.run.smithi134.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T10:29:18.274 INFO:teuthology.orchestra.run.smithi134.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T10:29:18.274 INFO:teuthology.orchestra.run.smithi134.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T10:29:18.274 INFO:teuthology.orchestra.run.smithi134.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T10:29:18.274 INFO:teuthology.orchestra.run.smithi134.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T10:29:18.274 INFO:teuthology.orchestra.run.smithi134.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T10:29:18.274 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:29:18.274 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:29:18.274 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:29:18.274 INFO:teuthology.orchestra.run.smithi134.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T10:29:18.274 INFO:teuthology.orchestra.run.smithi134.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T10:29:18.275 INFO:teuthology.orchestra.run.smithi134.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T10:29:18.275 INFO:teuthology.orchestra.run.smithi134.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T10:29:18.275 INFO:teuthology.orchestra.run.smithi134.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T10:29:18.275 INFO:teuthology.orchestra.run.smithi134.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T10:29:18.275 INFO:teuthology.orchestra.run.smithi134.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T10:29:18.275 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T10:29:18.275 INFO:teuthology.orchestra.run.smithi134.stdout:Complete! 2024-04-25T10:29:18.718 DEBUG:teuthology.parallel:result is None 2024-04-25T10:29:19.110 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 139/156 2024-04-25T10:29:19.152 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 140/156 2024-04-25T10:29:19.191 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 141/156 2024-04-25T10:29:19.224 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T10:29:19.251 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T10:29:19.431 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 144/156 2024-04-25T10:29:19.472 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T10:29:19.502 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 146/156 2024-04-25T10:29:20.088 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T10:29:20.125 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T10:29:20.145 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T10:29:20.985 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 148/156 2024-04-25T10:29:21.055 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T10:29:21.095 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T10:29:21.177 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T10:29:21.217 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T10:29:21.217 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T10:29:21.301 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T10:29:21.334 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T10:29:21.334 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T10:29:21.420 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T10:29:21.491 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T10:29:27.468 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T10:29:27.468 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 1/156 2024-04-25T10:29:27.468 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2/156 2024-04-25T10:29:27.468 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 3/156 2024-04-25T10:29:27.468 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 4/156 2024-04-25T10:29:27.468 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 5/156 2024-04-25T10:29:27.469 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 6/156 2024-04-25T10:29:27.469 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 7/156 2024-04-25T10:29:27.469 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T10:29:27.469 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T10:29:27.469 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T10:29:27.469 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 11/156 2024-04-25T10:29:27.469 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 12/156 2024-04-25T10:29:27.469 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T10:29:27.469 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 14/156 2024-04-25T10:29:27.469 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 15/156 2024-04-25T10:29:27.469 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 16/156 2024-04-25T10:29:27.469 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 17/156 2024-04-25T10:29:27.469 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 18/156 2024-04-25T10:29:27.469 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 19/156 2024-04-25T10:29:27.469 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 20/156 2024-04-25T10:29:27.470 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 21/156 2024-04-25T10:29:27.470 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 22/156 2024-04-25T10:29:27.470 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 23/156 2024-04-25T10:29:27.470 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 24/156 2024-04-25T10:29:27.470 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 25/156 2024-04-25T10:29:27.470 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 26/156 2024-04-25T10:29:27.470 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 27/156 2024-04-25T10:29:27.470 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 28/156 2024-04-25T10:29:27.470 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 29/156 2024-04-25T10:29:27.470 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 30/156 2024-04-25T10:29:27.481 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 31/156 2024-04-25T10:29:27.481 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 32/156 2024-04-25T10:29:27.481 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 33/156 2024-04-25T10:29:27.481 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 34/156 2024-04-25T10:29:27.481 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 35/156 2024-04-25T10:29:27.481 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 36/156 2024-04-25T10:29:27.481 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 37/156 2024-04-25T10:29:27.482 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 38/156 2024-04-25T10:29:27.482 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 39/156 2024-04-25T10:29:27.482 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T10:29:27.482 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T10:29:27.482 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T10:29:27.482 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T10:29:27.482 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T10:29:27.482 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T10:29:27.482 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T10:29:27.482 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T10:29:27.482 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T10:29:27.482 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T10:29:27.482 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T10:29:27.482 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T10:29:27.483 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T10:29:27.483 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T10:29:27.483 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T10:29:27.483 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T10:29:27.483 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T10:29:27.483 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T10:29:27.483 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T10:29:27.483 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T10:29:27.483 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T10:29:27.483 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T10:29:27.483 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T10:29:27.483 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T10:29:27.483 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T10:29:27.483 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T10:29:27.484 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T10:29:27.484 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T10:29:27.484 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T10:29:27.484 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T10:29:27.484 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T10:29:27.484 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T10:29:27.484 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T10:29:27.484 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T10:29:27.484 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T10:29:27.484 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T10:29:27.484 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T10:29:27.484 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T10:29:27.484 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T10:29:27.484 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T10:29:27.485 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T10:29:27.485 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T10:29:27.485 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T10:29:27.485 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T10:29:27.485 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T10:29:27.485 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T10:29:27.485 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T10:29:27.485 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T10:29:27.485 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T10:29:27.485 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T10:29:27.485 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T10:29:27.485 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T10:29:27.485 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T10:29:27.485 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T10:29:27.485 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T10:29:27.486 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T10:29:27.486 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T10:29:27.486 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T10:29:27.486 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T10:29:27.486 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T10:29:27.486 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T10:29:27.486 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T10:29:27.486 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T10:29:27.486 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T10:29:27.486 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T10:29:27.486 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T10:29:27.486 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T10:29:27.486 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T10:29:27.486 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T10:29:27.487 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T10:29:27.487 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T10:29:27.487 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T10:29:27.487 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T10:29:27.487 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T10:29:27.487 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T10:29:27.487 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T10:29:27.487 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T10:29:27.487 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T10:29:27.487 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T10:29:27.487 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T10:29:27.487 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T10:29:27.487 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T10:29:27.487 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T10:29:27.487 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T10:29:27.488 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T10:29:27.488 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T10:29:27.488 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T10:29:27.488 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T10:29:27.488 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T10:29:27.488 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T10:29:27.488 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T10:29:27.488 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T10:29:27.488 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T10:29:27.488 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T10:29:27.488 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T10:29:27.488 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T10:29:27.488 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T10:29:27.488 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T10:29:27.488 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T10:29:27.489 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T10:29:27.489 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T10:29:27.489 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T10:29:27.489 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T10:29:27.489 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T10:29:27.489 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T10:29:27.489 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T10:29:27.489 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T10:29:27.489 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T10:29:27.489 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T10:29:27.489 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 149/156 2024-04-25T10:29:27.489 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T10:29:27.489 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 151/156 2024-04-25T10:29:27.489 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T10:29:27.489 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T10:29:27.490 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T10:29:27.490 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T10:29:29.566 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T10:29:29.566 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:29:29.566 INFO:teuthology.orchestra.run.smithi106.stdout:Upgraded: 2024-04-25T10:29:29.566 INFO:teuthology.orchestra.run.smithi106.stdout: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:29:29.566 INFO:teuthology.orchestra.run.smithi106.stdout: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:29:29.566 INFO:teuthology.orchestra.run.smithi106.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T10:29:29.567 INFO:teuthology.orchestra.run.smithi106.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T10:29:29.567 INFO:teuthology.orchestra.run.smithi106.stdout:Installed: 2024-04-25T10:29:29.567 INFO:teuthology.orchestra.run.smithi106.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T10:29:29.567 INFO:teuthology.orchestra.run.smithi106.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T10:29:29.567 INFO:teuthology.orchestra.run.smithi106.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T10:29:29.567 INFO:teuthology.orchestra.run.smithi106.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T10:29:29.567 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:29:29.567 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:29:29.567 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:29:29.567 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:29:29.567 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:29:29.567 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-immutable-object-cache-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:29:29.567 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:29:29.567 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:29:29.567 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:29:29.568 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:29:29.568 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:29:29.568 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:29:29.568 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:29:29.568 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:29:29.568 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:29:29.568 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:29:29.568 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:29:29.568 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:29:29.568 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:29:29.568 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:29:29.568 INFO:teuthology.orchestra.run.smithi106.stdout: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:29:29.568 INFO:teuthology.orchestra.run.smithi106.stdout: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:29:29.568 INFO:teuthology.orchestra.run.smithi106.stdout: cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:29:29.569 INFO:teuthology.orchestra.run.smithi106.stdout: cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:29:29.569 INFO:teuthology.orchestra.run.smithi106.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T10:29:29.569 INFO:teuthology.orchestra.run.smithi106.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T10:29:29.569 INFO:teuthology.orchestra.run.smithi106.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T10:29:29.569 INFO:teuthology.orchestra.run.smithi106.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T10:29:29.569 INFO:teuthology.orchestra.run.smithi106.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T10:29:29.569 INFO:teuthology.orchestra.run.smithi106.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T10:29:29.569 INFO:teuthology.orchestra.run.smithi106.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T10:29:29.569 INFO:teuthology.orchestra.run.smithi106.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T10:29:29.569 INFO:teuthology.orchestra.run.smithi106.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T10:29:29.569 INFO:teuthology.orchestra.run.smithi106.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T10:29:29.569 INFO:teuthology.orchestra.run.smithi106.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T10:29:29.569 INFO:teuthology.orchestra.run.smithi106.stdout: libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:29:29.569 INFO:teuthology.orchestra.run.smithi106.stdout: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:29:29.569 INFO:teuthology.orchestra.run.smithi106.stdout: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:29:29.570 INFO:teuthology.orchestra.run.smithi106.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T10:29:29.570 INFO:teuthology.orchestra.run.smithi106.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T10:29:29.570 INFO:teuthology.orchestra.run.smithi106.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T10:29:29.570 INFO:teuthology.orchestra.run.smithi106.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T10:29:29.570 INFO:teuthology.orchestra.run.smithi106.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T10:29:29.570 INFO:teuthology.orchestra.run.smithi106.stdout: librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:29:29.570 INFO:teuthology.orchestra.run.smithi106.stdout: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:29:29.570 INFO:teuthology.orchestra.run.smithi106.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T10:29:29.570 INFO:teuthology.orchestra.run.smithi106.stdout: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:29:29.570 INFO:teuthology.orchestra.run.smithi106.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T10:29:29.570 INFO:teuthology.orchestra.run.smithi106.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T10:29:29.570 INFO:teuthology.orchestra.run.smithi106.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T10:29:29.570 INFO:teuthology.orchestra.run.smithi106.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T10:29:29.570 INFO:teuthology.orchestra.run.smithi106.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T10:29:29.571 INFO:teuthology.orchestra.run.smithi106.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T10:29:29.571 INFO:teuthology.orchestra.run.smithi106.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T10:29:29.571 INFO:teuthology.orchestra.run.smithi106.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T10:29:29.571 INFO:teuthology.orchestra.run.smithi106.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T10:29:29.571 INFO:teuthology.orchestra.run.smithi106.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T10:29:29.571 INFO:teuthology.orchestra.run.smithi106.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T10:29:29.571 INFO:teuthology.orchestra.run.smithi106.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T10:29:29.571 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T10:29:29.571 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T10:29:29.571 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T10:29:29.571 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T10:29:29.571 INFO:teuthology.orchestra.run.smithi106.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T10:29:29.571 INFO:teuthology.orchestra.run.smithi106.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T10:29:29.571 INFO:teuthology.orchestra.run.smithi106.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T10:29:29.572 INFO:teuthology.orchestra.run.smithi106.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T10:29:29.572 INFO:teuthology.orchestra.run.smithi106.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T10:29:29.572 INFO:teuthology.orchestra.run.smithi106.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T10:29:29.572 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T10:29:29.572 INFO:teuthology.orchestra.run.smithi106.stdout: python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:29:29.572 INFO:teuthology.orchestra.run.smithi106.stdout: python3-ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:29:29.572 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:29:29.572 INFO:teuthology.orchestra.run.smithi106.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T10:29:29.572 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T10:29:29.572 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T10:29:29.572 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T10:29:29.572 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T10:29:29.572 INFO:teuthology.orchestra.run.smithi106.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T10:29:29.572 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T10:29:29.572 INFO:teuthology.orchestra.run.smithi106.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T10:29:29.573 INFO:teuthology.orchestra.run.smithi106.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T10:29:29.573 INFO:teuthology.orchestra.run.smithi106.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T10:29:29.573 INFO:teuthology.orchestra.run.smithi106.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T10:29:29.573 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T10:29:29.573 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T10:29:29.573 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T10:29:29.573 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T10:29:29.573 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T10:29:29.573 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T10:29:29.573 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T10:29:29.573 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T10:29:29.573 INFO:teuthology.orchestra.run.smithi106.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T10:29:29.573 INFO:teuthology.orchestra.run.smithi106.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T10:29:29.573 INFO:teuthology.orchestra.run.smithi106.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T10:29:29.573 INFO:teuthology.orchestra.run.smithi106.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T10:29:29.574 INFO:teuthology.orchestra.run.smithi106.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T10:29:29.574 INFO:teuthology.orchestra.run.smithi106.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T10:29:29.574 INFO:teuthology.orchestra.run.smithi106.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T10:29:29.574 INFO:teuthology.orchestra.run.smithi106.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T10:29:29.574 INFO:teuthology.orchestra.run.smithi106.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T10:29:29.574 INFO:teuthology.orchestra.run.smithi106.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T10:29:29.574 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T10:29:29.574 INFO:teuthology.orchestra.run.smithi106.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T10:29:29.574 INFO:teuthology.orchestra.run.smithi106.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T10:29:29.574 INFO:teuthology.orchestra.run.smithi106.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T10:29:29.574 INFO:teuthology.orchestra.run.smithi106.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T10:29:29.574 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T10:29:29.574 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T10:29:29.574 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T10:29:29.575 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T10:29:29.575 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T10:29:29.575 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T10:29:29.575 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:29:29.575 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:29:29.575 INFO:teuthology.orchestra.run.smithi106.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T10:29:29.575 INFO:teuthology.orchestra.run.smithi106.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T10:29:29.576 INFO:teuthology.orchestra.run.smithi106.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T10:29:29.576 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:29:29.576 INFO:teuthology.orchestra.run.smithi106.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T10:29:29.576 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T10:29:29.576 INFO:teuthology.orchestra.run.smithi106.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T10:29:29.576 INFO:teuthology.orchestra.run.smithi106.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T10:29:29.576 INFO:teuthology.orchestra.run.smithi106.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T10:29:29.576 INFO:teuthology.orchestra.run.smithi106.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T10:29:29.576 INFO:teuthology.orchestra.run.smithi106.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T10:29:29.576 INFO:teuthology.orchestra.run.smithi106.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T10:29:29.576 INFO:teuthology.orchestra.run.smithi106.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T10:29:29.576 INFO:teuthology.orchestra.run.smithi106.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T10:29:29.576 INFO:teuthology.orchestra.run.smithi106.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T10:29:29.576 INFO:teuthology.orchestra.run.smithi106.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T10:29:29.577 INFO:teuthology.orchestra.run.smithi106.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T10:29:29.577 INFO:teuthology.orchestra.run.smithi106.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T10:29:29.577 INFO:teuthology.orchestra.run.smithi106.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T10:29:29.577 INFO:teuthology.orchestra.run.smithi106.stdout: rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:29:29.577 INFO:teuthology.orchestra.run.smithi106.stdout: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:29:29.577 INFO:teuthology.orchestra.run.smithi106.stdout: rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:29:29.577 INFO:teuthology.orchestra.run.smithi106.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T10:29:29.577 INFO:teuthology.orchestra.run.smithi106.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T10:29:29.577 INFO:teuthology.orchestra.run.smithi106.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T10:29:29.578 INFO:teuthology.orchestra.run.smithi106.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T10:29:29.578 INFO:teuthology.orchestra.run.smithi106.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T10:29:29.578 INFO:teuthology.orchestra.run.smithi106.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T10:29:29.578 INFO:teuthology.orchestra.run.smithi106.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T10:29:29.579 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:29:29.579 INFO:teuthology.orchestra.run.smithi106.stdout:Complete! 2024-04-25T10:29:29.961 DEBUG:teuthology.parallel:result is None 2024-04-25T10:29:29.962 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T10:29:30.183 DEBUG:teuthology.orchestra.run.smithi106:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T10:29:30.206 INFO:teuthology.orchestra.run.smithi106.stdout:19.0.0-3291.g994e2f20.el9 2024-04-25T10:29:30.206 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3291.g994e2f20.el9 2024-04-25T10:29:30.206 INFO:teuthology.task.install:The correct ceph version 19.0.0-3291.g994e2f20 is installed. 2024-04-25T10:29:30.208 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T10:29:30.348 DEBUG:teuthology.orchestra.run.smithi134:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T10:29:30.373 INFO:teuthology.orchestra.run.smithi134.stdout:19.0.0-3291.g994e2f20.el9 2024-04-25T10:29:30.374 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3291.g994e2f20.el9 2024-04-25T10:29:30.374 INFO:teuthology.task.install:The correct ceph version 19.0.0-3291.g994e2f20 is installed. 2024-04-25T10:29:30.376 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T10:29:30.376 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T10:29:30.376 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T10:29:30.401 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-25T10:29:30.401 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T10:29:30.443 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T10:29:30.443 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T10:29:30.443 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T10:29:30.467 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T10:29:30.532 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-25T10:29:30.532 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T10:29:30.558 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T10:29:30.624 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T10:29:30.624 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T10:29:30.624 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T10:29:30.648 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T10:29:30.712 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-25T10:29:30.713 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T10:29:30.738 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T10:29:30.803 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T10:29:30.804 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T10:29:30.804 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T10:29:30.828 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T10:29:30.891 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-25T10:29:30.892 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T10:29:30.916 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T10:29:30.982 INFO:teuthology.run_tasks:Running task ceph... 2024-04-25T10:29:31.094 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-25T10:29:31.094 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 777 /var/log/ceph 2024-04-25T10:29:31.096 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 777 /var/log/ceph 2024-04-25T10:29:31.120 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-25T10:29:31.120 DEBUG:teuthology.orchestra.run.smithi106:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T10:29:31.161 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T10:29:31.186 INFO:tasks.ceph:Creating extra log directories... 2024-04-25T10:29:31.186 DEBUG:teuthology.orchestra.run.smithi106:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T10:29:31.225 DEBUG:teuthology.orchestra.run.smithi134:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T10:29:31.256 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-25T10:29:31.257 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-25T10:29:31.257 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-25T10:29:31.257 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T10:29:31.257 DEBUG:teuthology.orchestra.run.smithi106:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T10:29:31.301 DEBUG:teuthology.orchestra.run.smithi106:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T10:29:31.416 DEBUG:teuthology.orchestra.run.smithi106:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-25T10:29:31.439 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-25T10:29:31.439 DEBUG:teuthology.orchestra.run.smithi134:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T10:29:31.454 DEBUG:teuthology.orchestra.run.smithi134:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T10:29:31.563 DEBUG:teuthology.orchestra.run.smithi134:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-25T10:29:31.593 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-25T10:29:31.593 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'overall HEALTH_', '\\(MON_DOWN\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-25T10:29:31.593 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673079', 'branch': 'wip-pdonnell-testing-20240425.015853-debug', 'description': 'fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_dirfrags/no prefetch_dirfrags/yes prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mon 2-workunit/suites/iozone}}', 'email': 'pdonnell@redhat.com', 'first_in_suite': False, 'job_id': '7673079', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-pdonnell-testing-20240425.015853-debug'}, 'ceph': {'cephfs': {'max_mds': 5, 'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'overall HEALTH_', '\\(MON_DOWN\\)'], 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'check-counter': {'counters': {'mds': ['mds.exported', 'mds.imported']}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-pdonnell-testing-20240425.015853-debug', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}}, 'owner': 'scheduled_pdonnell@teuthology', 'priority': 85, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7']], 'seed': 1880, 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'sleep_before_teardown': 0, 'subset': '178/512', 'suite': 'fs', 'suite_branch': 'wip-pdonnell-testing-20240425.015853-debug', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi106.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBETD64HyMKMQLS3kViFb0IHUZamIJNGGh6L6TVEKRWbRk8LgIQs3RtuGal8vlw4QMBYO4JewP8pqw0tojZ11xo=', 'smithi134.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDLZPaQIpOulwMGfS8SDt9ONjvD4gK6qOuiIK2YqaJES2XKgT+OCIlH7sU/VFic6a0EkZg26PLttrOZfoV+6Mxc='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'overall HEALTH_', '\\(MON_DOWN\\)'], 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'ceph-fuse': None}, {'mon_thrash': {'check_mds_failover': True, 'revive_delay': 20, 'thrash_delay': 10}}, {'workunit': {'clients': {'all': ['suites/iozone.sh']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-25_10:00:23', 'tube': 'smithi', 'user': 'pdonnell', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-25T10:29:31.594 DEBUG:teuthology.orchestra.run.smithi106:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T10:29:31.596 DEBUG:teuthology.orchestra.run.smithi134:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T10:29:31.641 DEBUG:teuthology.orchestra.run.smithi106:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T10:29:31.650 DEBUG:teuthology.orchestra.run.smithi134:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T10:29:31.705 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T10:29:31.705 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T10:29:31.726 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T10:29:31.726 DEBUG:teuthology.orchestra.run.smithi106:> stat /dev/vg_nvme/lv_1 2024-04-25T10:29:31.780 INFO:teuthology.orchestra.run.smithi106.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T10:29:31.781 INFO:teuthology.orchestra.run.smithi106.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:29:31.781 INFO:teuthology.orchestra.run.smithi106.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-25T10:29:31.781 INFO:teuthology.orchestra.run.smithi106.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:29:31.781 INFO:teuthology.orchestra.run.smithi106.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:29:31.781 INFO:teuthology.orchestra.run.smithi106.stdout:Access: 2024-04-25 10:29:21.847385439 +0000 2024-04-25T10:29:31.781 INFO:teuthology.orchestra.run.smithi106.stdout:Modify: 2024-04-25 10:27:29.739742055 +0000 2024-04-25T10:29:31.781 INFO:teuthology.orchestra.run.smithi106.stdout:Change: 2024-04-25 10:27:29.739742055 +0000 2024-04-25T10:29:31.781 INFO:teuthology.orchestra.run.smithi106.stdout: Birth: 2024-04-25 10:27:29.739742055 +0000 2024-04-25T10:29:31.781 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T10:29:31.843 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records in 2024-04-25T10:29:31.843 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records out 2024-04-25T10:29:31.843 INFO:teuthology.orchestra.run.smithi106.stderr:512 bytes copied, 0.000192237 s, 2.7 MB/s 2024-04-25T10:29:31.844 DEBUG:teuthology.orchestra.run.smithi106:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T10:29:31.900 DEBUG:teuthology.orchestra.run.smithi106:> stat /dev/vg_nvme/lv_2 2024-04-25T10:29:31.955 INFO:teuthology.orchestra.run.smithi106.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T10:29:31.955 INFO:teuthology.orchestra.run.smithi106.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:29:31.956 INFO:teuthology.orchestra.run.smithi106.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-25T10:29:31.956 INFO:teuthology.orchestra.run.smithi106.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:29:31.956 INFO:teuthology.orchestra.run.smithi106.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:29:31.956 INFO:teuthology.orchestra.run.smithi106.stdout:Access: 2024-04-25 10:29:21.848385417 +0000 2024-04-25T10:29:31.956 INFO:teuthology.orchestra.run.smithi106.stdout:Modify: 2024-04-25 10:27:30.076733429 +0000 2024-04-25T10:29:31.956 INFO:teuthology.orchestra.run.smithi106.stdout:Change: 2024-04-25 10:27:30.076733429 +0000 2024-04-25T10:29:31.956 INFO:teuthology.orchestra.run.smithi106.stdout: Birth: 2024-04-25 10:27:30.076733429 +0000 2024-04-25T10:29:31.956 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T10:29:32.019 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records in 2024-04-25T10:29:32.019 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records out 2024-04-25T10:29:32.019 INFO:teuthology.orchestra.run.smithi106.stderr:512 bytes copied, 0.000191111 s, 2.7 MB/s 2024-04-25T10:29:32.020 DEBUG:teuthology.orchestra.run.smithi106:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T10:29:32.076 DEBUG:teuthology.orchestra.run.smithi106:> stat /dev/vg_nvme/lv_3 2024-04-25T10:29:32.130 INFO:teuthology.orchestra.run.smithi106.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T10:29:32.131 INFO:teuthology.orchestra.run.smithi106.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:29:32.131 INFO:teuthology.orchestra.run.smithi106.stdout:Device: 5h/5d Inode: 701 Links: 1 2024-04-25T10:29:32.131 INFO:teuthology.orchestra.run.smithi106.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:29:32.131 INFO:teuthology.orchestra.run.smithi106.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:29:32.131 INFO:teuthology.orchestra.run.smithi106.stdout:Access: 2024-04-25 10:29:21.848385417 +0000 2024-04-25T10:29:32.131 INFO:teuthology.orchestra.run.smithi106.stdout:Modify: 2024-04-25 10:27:30.408724931 +0000 2024-04-25T10:29:32.131 INFO:teuthology.orchestra.run.smithi106.stdout:Change: 2024-04-25 10:27:30.408724931 +0000 2024-04-25T10:29:32.131 INFO:teuthology.orchestra.run.smithi106.stdout: Birth: 2024-04-25 10:27:30.408724931 +0000 2024-04-25T10:29:32.132 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T10:29:32.193 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records in 2024-04-25T10:29:32.193 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records out 2024-04-25T10:29:32.193 INFO:teuthology.orchestra.run.smithi106.stderr:512 bytes copied, 0.0001896 s, 2.7 MB/s 2024-04-25T10:29:32.194 DEBUG:teuthology.orchestra.run.smithi106:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T10:29:32.249 DEBUG:teuthology.orchestra.run.smithi106:> stat /dev/vg_nvme/lv_4 2024-04-25T10:29:32.305 INFO:teuthology.orchestra.run.smithi106.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T10:29:32.305 INFO:teuthology.orchestra.run.smithi106.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:29:32.305 INFO:teuthology.orchestra.run.smithi106.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-25T10:29:32.305 INFO:teuthology.orchestra.run.smithi106.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:29:32.305 INFO:teuthology.orchestra.run.smithi106.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:29:32.305 INFO:teuthology.orchestra.run.smithi106.stdout:Access: 2024-04-25 10:29:21.848385417 +0000 2024-04-25T10:29:32.305 INFO:teuthology.orchestra.run.smithi106.stdout:Modify: 2024-04-25 10:27:30.742716382 +0000 2024-04-25T10:29:32.305 INFO:teuthology.orchestra.run.smithi106.stdout:Change: 2024-04-25 10:27:30.742716382 +0000 2024-04-25T10:29:32.305 INFO:teuthology.orchestra.run.smithi106.stdout: Birth: 2024-04-25 10:27:30.742716382 +0000 2024-04-25T10:29:32.305 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T10:29:32.367 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records in 2024-04-25T10:29:32.367 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records out 2024-04-25T10:29:32.367 INFO:teuthology.orchestra.run.smithi106.stderr:512 bytes copied, 0.0001886 s, 2.7 MB/s 2024-04-25T10:29:32.368 DEBUG:teuthology.orchestra.run.smithi106:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T10:29:32.423 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T10:29:32.424 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-25T10:29:32.424 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T10:29:32.441 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T10:29:32.441 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_1 2024-04-25T10:29:32.498 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T10:29:32.498 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:29:32.498 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-25T10:29:32.498 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:29:32.498 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:29:32.498 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-04-25 10:29:10.689342549 +0000 2024-04-25T10:29:32.498 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-04-25 10:27:29.752502571 +0000 2024-04-25T10:29:32.498 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-04-25 10:27:29.752502571 +0000 2024-04-25T10:29:32.498 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: 2024-04-25 10:27:29.752502571 +0000 2024-04-25T10:29:32.499 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T10:29:32.563 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-04-25T10:29:32.563 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-04-25T10:29:32.563 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000285778 s, 1.8 MB/s 2024-04-25T10:29:32.564 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T10:29:32.622 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_2 2024-04-25T10:29:32.678 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T10:29:32.678 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:29:32.678 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 682 Links: 1 2024-04-25T10:29:32.678 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:29:32.678 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:29:32.678 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-04-25 10:29:10.689342549 +0000 2024-04-25T10:29:32.678 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-04-25 10:27:30.104493782 +0000 2024-04-25T10:29:32.678 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-04-25 10:27:30.104493782 +0000 2024-04-25T10:29:32.678 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: 2024-04-25 10:27:30.104493782 +0000 2024-04-25T10:29:32.679 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T10:29:32.742 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-04-25T10:29:32.742 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-04-25T10:29:32.742 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000193205 s, 2.7 MB/s 2024-04-25T10:29:32.743 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T10:29:32.799 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_3 2024-04-25T10:29:32.854 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T10:29:32.854 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:29:32.855 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-25T10:29:32.855 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:29:32.855 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:29:32.855 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-04-25 10:29:10.690342525 +0000 2024-04-25T10:29:32.855 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-04-25 10:27:30.433485567 +0000 2024-04-25T10:29:32.855 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-04-25 10:27:30.433485567 +0000 2024-04-25T10:29:32.855 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: 2024-04-25 10:27:30.433485567 +0000 2024-04-25T10:29:32.855 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T10:29:32.917 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-04-25T10:29:32.917 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-04-25T10:29:32.917 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000177005 s, 2.9 MB/s 2024-04-25T10:29:32.918 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T10:29:32.974 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_4 2024-04-25T10:29:33.029 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T10:29:33.030 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:29:33.030 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-04-25T10:29:33.030 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:29:33.030 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:29:33.030 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-04-25 10:29:10.690342525 +0000 2024-04-25T10:29:33.030 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-04-25 10:27:30.777476977 +0000 2024-04-25T10:29:33.030 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-04-25 10:27:30.777476977 +0000 2024-04-25T10:29:33.030 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: 2024-04-25 10:27:30.777476977 +0000 2024-04-25T10:29:33.030 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T10:29:33.094 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-04-25T10:29:33.094 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-04-25T10:29:33.094 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000206194 s, 2.5 MB/s 2024-04-25T10:29:33.096 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T10:29:33.153 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T10:29:33.153 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi106.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi134.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T10:29:33.153 INFO:tasks.ceph:Generating config... 2024-04-25T10:29:33.154 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-25T10:29:33.154 INFO:tasks.ceph:[client] client_shutdown_timeout = 120 2024-04-25T10:29:33.154 INFO:tasks.ceph:[client] debug client = 20 2024-04-25T10:29:33.154 INFO:tasks.ceph:[client] debug ms = 1 2024-04-25T10:29:33.154 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-25T10:29:33.155 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-25T10:29:33.155 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-25T10:29:33.155 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-25T10:29:33.155 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-25T10:29:33.155 INFO:tasks.ceph:[mds] mds bal fragment size max = 10000 2024-04-25T10:29:33.155 INFO:tasks.ceph:[mds] mds bal merge size = 5 2024-04-25T10:29:33.155 INFO:tasks.ceph:[mds] mds bal split bits = 3 2024-04-25T10:29:33.155 INFO:tasks.ceph:[mds] mds bal split size = 100 2024-04-25T10:29:33.155 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-25T10:29:33.155 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-25T10:29:33.155 INFO:tasks.ceph:[mds] mds oft prefetch dirfrags = True 2024-04-25T10:29:33.155 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-25T10:29:33.155 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-25T10:29:33.155 INFO:tasks.ceph:[mds] mds_dir_prefetch = True 2024-04-25T10:29:33.155 INFO:tasks.ceph:[mds] mds_sleep_rank_change = 5000000.0 2024-04-25T10:29:33.156 INFO:tasks.ceph:[mds] mon client directed command retry = 5 2024-04-25T10:29:33.156 INFO:tasks.ceph:[mds] ms inject delay max = 1 2024-04-25T10:29:33.156 INFO:tasks.ceph:[mds] ms inject delay probability = 0.005 2024-04-25T10:29:33.156 INFO:tasks.ceph:[mds] ms inject delay type = client mds osd 2024-04-25T10:29:33.156 INFO:tasks.ceph:[mds] ms inject socket failures = 2500 2024-04-25T10:29:33.156 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-25T10:29:33.156 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-25T10:29:33.156 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-25T10:29:33.156 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-25T10:29:33.156 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-25T10:29:33.156 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-25T10:29:33.157 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-25T10:29:33.157 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-25T10:29:33.157 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-25T10:29:33.157 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-25T10:29:33.157 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-25T10:29:33.157 INFO:tasks.ceph:[osd] bdev async discard = True 2024-04-25T10:29:33.157 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-04-25T10:29:33.157 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-04-25T10:29:33.157 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-25T10:29:33.157 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-25T10:29:33.157 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-04-25T10:29:33.157 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-04-25T10:29:33.157 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-25T10:29:33.157 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-25T10:29:33.158 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-04-25T10:29:33.158 INFO:tasks.ceph:[osd] mon client directed command retry = 5 2024-04-25T10:29:33.158 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-25T10:29:33.158 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-25T10:29:33.158 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-25T10:29:33.158 INFO:tasks.ceph:[osd] ms inject delay max = 1 2024-04-25T10:29:33.158 INFO:tasks.ceph:[osd] ms inject delay probability = 0.005 2024-04-25T10:29:33.158 INFO:tasks.ceph:[osd] ms inject delay type = client mds 2024-04-25T10:29:33.158 INFO:tasks.ceph:[osd] ms inject socket failures = 2500 2024-04-25T10:29:33.158 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-04-25T10:29:33.158 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-25T10:29:33.158 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-04-25T10:29:33.158 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-25T10:29:33.158 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-25T10:29:33.159 INFO:tasks.ceph:Setting up mon.a... 2024-04-25T10:29:33.159 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-25T10:29:33.218 INFO:teuthology.orchestra.run.smithi106.stdout:creating /etc/ceph/ceph.keyring 2024-04-25T10:29:33.221 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-25T10:29:33.273 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T10:29:33.296 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.106'), ('mon.b', '172.21.15.134'), ('mon.c', '[v2:172.21.15.134:3301,v1:172.21.15.134:6790]')] 2024-04-25T10:29:33.296 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.106,172.21.15.134,[v2:172.21.15.134:3301,v1:172.21.15.134:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.e': {}, 'mds.b': {}, 'mds.d': {}, 'mds.f': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-25T10:29:33.297 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T10:29:33.297 DEBUG:teuthology.orchestra.run.smithi106:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T10:29:33.350 DEBUG:teuthology.orchestra.run.smithi106:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.106 --add b 172.21.15.134 --addv c '[v2:172.21.15.134:3301,v1:172.21.15.134:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:29:33.447 INFO:teuthology.orchestra.run.smithi106.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:29:33.447 INFO:teuthology.orchestra.run.smithi106.stdout:monmaptool: generated fsid 3ee77985-e8e4-41cf-81e9-7c774cf7891f 2024-04-25T10:29:33.447 INFO:teuthology.orchestra.run.smithi106.stdout:setting min_mon_release = quincy 2024-04-25T10:29:33.447 INFO:teuthology.orchestra.run.smithi106.stdout:epoch 0 2024-04-25T10:29:33.447 INFO:teuthology.orchestra.run.smithi106.stdout:fsid 3ee77985-e8e4-41cf-81e9-7c774cf7891f 2024-04-25T10:29:33.447 INFO:teuthology.orchestra.run.smithi106.stdout:last_changed 2024-04-25T10:29:33.446228+0000 2024-04-25T10:29:33.447 INFO:teuthology.orchestra.run.smithi106.stdout:created 2024-04-25T10:29:33.446228+0000 2024-04-25T10:29:33.447 INFO:teuthology.orchestra.run.smithi106.stdout:min_mon_release 17 (quincy) 2024-04-25T10:29:33.448 INFO:teuthology.orchestra.run.smithi106.stdout:election_strategy: 1 2024-04-25T10:29:33.448 INFO:teuthology.orchestra.run.smithi106.stdout:0: [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] mon.a 2024-04-25T10:29:33.448 INFO:teuthology.orchestra.run.smithi106.stdout:1: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.b 2024-04-25T10:29:33.448 INFO:teuthology.orchestra.run.smithi106.stdout:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.c 2024-04-25T10:29:33.448 INFO:teuthology.orchestra.run.smithi106.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-25T10:29:33.449 DEBUG:teuthology.orchestra.run.smithi106:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T10:29:33.463 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 3ee77985-e8e4-41cf-81e9-7c774cf7891f... 2024-04-25T10:29:33.464 DEBUG:teuthology.orchestra.run.smithi106:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:29:33.507 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:29:33.544 INFO:teuthology.orchestra.run.smithi106.stdout:[global] 2024-04-25T10:29:33.544 INFO:teuthology.orchestra.run.smithi106.stdout: chdir = "" 2024-04-25T10:29:33.544 INFO:teuthology.orchestra.run.smithi106.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:29:33.544 INFO:teuthology.orchestra.run.smithi106.stdout: auth supported = cephx 2024-04-25T10:29:33.545 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:29:33.545 INFO:teuthology.orchestra.run.smithi106.stdout: filestore xattr use omap = true 2024-04-25T10:29:33.545 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:29:33.545 INFO:teuthology.orchestra.run.smithi106.stdout: mon clock drift allowed = 1.000 2024-04-25T10:29:33.545 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:29:33.545 INFO:teuthology.orchestra.run.smithi106.stdout: osd crush chooseleaf type = 0 2024-04-25T10:29:33.545 INFO:teuthology.orchestra.run.smithi106.stdout: auth debug = true 2024-04-25T10:29:33.545 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:29:33.545 INFO:teuthology.orchestra.run.smithi106.stdout: ms die on old message = true 2024-04-25T10:29:33.545 INFO:teuthology.orchestra.run.smithi106.stdout: ms die on bug = true 2024-04-25T10:29:33.545 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:29:33.545 INFO:teuthology.orchestra.run.smithi106.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:29:33.545 INFO:teuthology.orchestra.run.smithi106.stdout: mon pg warn max object skew = 0 2024-04-25T10:29:33.545 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:29:33.545 INFO:teuthology.orchestra.run.smithi106.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:29:33.546 INFO:teuthology.orchestra.run.smithi106.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:29:33.546 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:29:33.546 INFO:teuthology.orchestra.run.smithi106.stdout: osd pool default size = 2 2024-04-25T10:29:33.546 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:29:33.546 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd allow primary affinity = true 2024-04-25T10:29:33.546 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd allow pg remap = true 2024-04-25T10:29:33.546 INFO:teuthology.orchestra.run.smithi106.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:29:33.546 INFO:teuthology.orchestra.run.smithi106.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:29:33.546 INFO:teuthology.orchestra.run.smithi106.stdout: mon warn on no sortbitwise = false 2024-04-25T10:29:33.546 INFO:teuthology.orchestra.run.smithi106.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:29:33.546 INFO:teuthology.orchestra.run.smithi106.stdout: mon warn on too few osds = false 2024-04-25T10:29:33.546 INFO:teuthology.orchestra.run.smithi106.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:29:33.546 INFO:teuthology.orchestra.run.smithi106.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:29:33.546 INFO:teuthology.orchestra.run.smithi106.stdout: mon_allow_pool_size_one = true 2024-04-25T10:29:33.547 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:29:33.547 INFO:teuthology.orchestra.run.smithi106.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:29:33.547 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:29:33.547 INFO:teuthology.orchestra.run.smithi106.stdout: osd default data pool replay window = 5 2024-04-25T10:29:33.547 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:29:33.547 INFO:teuthology.orchestra.run.smithi106.stdout: mon allow pool delete = true 2024-04-25T10:29:33.547 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:29:33.547 INFO:teuthology.orchestra.run.smithi106.stdout: mon cluster log file level = debug 2024-04-25T10:29:33.547 INFO:teuthology.orchestra.run.smithi106.stdout: debug asserts on shutdown = true 2024-04-25T10:29:33.547 INFO:teuthology.orchestra.run.smithi106.stdout: mon health detail to clog = false 2024-04-25T10:29:33.547 INFO:teuthology.orchestra.run.smithi106.stdout: mon host = "172.21.15.106,172.21.15.134,[v2:172.21.15.134:3301,v1:172.21.15.134:6790]" 2024-04-25T10:29:33.547 INFO:teuthology.orchestra.run.smithi106.stdout: fsid = 3ee77985-e8e4-41cf-81e9-7c774cf7891f 2024-04-25T10:29:33.547 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:29:33.547 INFO:teuthology.orchestra.run.smithi106.stdout:[osd] 2024-04-25T10:29:33.547 INFO:teuthology.orchestra.run.smithi106.stdout: osd journal size = 100 2024-04-25T10:29:33.548 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:29:33.548 INFO:teuthology.orchestra.run.smithi106.stdout: osd scrub load threshold = 5.0 2024-04-25T10:29:33.548 INFO:teuthology.orchestra.run.smithi106.stdout: osd scrub max interval = 600 2024-04-25T10:29:33.548 INFO:teuthology.orchestra.run.smithi106.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:29:33.548 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:29:33.548 INFO:teuthology.orchestra.run.smithi106.stdout: osd recover clone overlap = true 2024-04-25T10:29:33.548 INFO:teuthology.orchestra.run.smithi106.stdout: osd recovery max chunk = 1048576 2024-04-25T10:29:33.548 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:29:33.548 INFO:teuthology.orchestra.run.smithi106.stdout: osd debug shutdown = true 2024-04-25T10:29:33.548 INFO:teuthology.orchestra.run.smithi106.stdout: osd debug op order = true 2024-04-25T10:29:33.548 INFO:teuthology.orchestra.run.smithi106.stdout: osd debug verify stray on activate = true 2024-04-25T10:29:33.548 INFO:teuthology.orchestra.run.smithi106.stdout: osd debug trim objects = true 2024-04-25T10:29:33.548 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:29:33.548 INFO:teuthology.orchestra.run.smithi106.stdout: osd open classes on start = true 2024-04-25T10:29:33.548 INFO:teuthology.orchestra.run.smithi106.stdout: osd debug pg log writeout = true 2024-04-25T10:29:33.549 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:29:33.549 INFO:teuthology.orchestra.run.smithi106.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:29:33.549 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:29:33.549 INFO:teuthology.orchestra.run.smithi106.stdout: osd map max advance = 10 2024-04-25T10:29:33.549 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:29:33.549 INFO:teuthology.orchestra.run.smithi106.stdout: journal zero on create = true 2024-04-25T10:29:33.549 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:29:33.549 INFO:teuthology.orchestra.run.smithi106.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:29:33.549 INFO:teuthology.orchestra.run.smithi106.stdout: filestore apply finisher threads = 3 2024-04-25T10:29:33.549 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:29:33.549 INFO:teuthology.orchestra.run.smithi106.stdout: bdev debug aio = true 2024-04-25T10:29:33.549 INFO:teuthology.orchestra.run.smithi106.stdout: osd debug misdirected ops = true 2024-04-25T10:29:33.549 INFO:teuthology.orchestra.run.smithi106.stdout: bdev async discard = True 2024-04-25T10:29:33.549 INFO:teuthology.orchestra.run.smithi106.stdout: bdev enable discard = True 2024-04-25T10:29:33.549 INFO:teuthology.orchestra.run.smithi106.stdout: bluestore allocator = bitmap 2024-04-25T10:29:33.550 INFO:teuthology.orchestra.run.smithi106.stdout: bluestore block size = 96636764160 2024-04-25T10:29:33.550 INFO:teuthology.orchestra.run.smithi106.stdout: bluestore fsck on mount = True 2024-04-25T10:29:33.550 INFO:teuthology.orchestra.run.smithi106.stdout: debug bluefs = 1/20 2024-04-25T10:29:33.550 INFO:teuthology.orchestra.run.smithi106.stdout: debug bluestore = 1/20 2024-04-25T10:29:33.550 INFO:teuthology.orchestra.run.smithi106.stdout: debug ms = 1 2024-04-25T10:29:33.550 INFO:teuthology.orchestra.run.smithi106.stdout: debug osd = 20 2024-04-25T10:29:33.550 INFO:teuthology.orchestra.run.smithi106.stdout: debug rocksdb = 4/10 2024-04-25T10:29:33.550 INFO:teuthology.orchestra.run.smithi106.stdout: mon client directed command retry = 5 2024-04-25T10:29:33.550 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:29:33.550 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd full ratio = 0.9 2024-04-25T10:29:33.550 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:29:33.550 INFO:teuthology.orchestra.run.smithi106.stdout: ms inject delay max = 1 2024-04-25T10:29:33.550 INFO:teuthology.orchestra.run.smithi106.stdout: ms inject delay probability = 0.005 2024-04-25T10:29:33.550 INFO:teuthology.orchestra.run.smithi106.stdout: ms inject delay type = client mds 2024-04-25T10:29:33.551 INFO:teuthology.orchestra.run.smithi106.stdout: ms inject socket failures = 2500 2024-04-25T10:29:33.551 INFO:teuthology.orchestra.run.smithi106.stdout: osd blocked scrub grace period = 3600 2024-04-25T10:29:33.551 INFO:teuthology.orchestra.run.smithi106.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:29:33.551 INFO:teuthology.orchestra.run.smithi106.stdout: osd max markdown count = 1000 2024-04-25T10:29:33.551 INFO:teuthology.orchestra.run.smithi106.stdout: osd objectstore = bluestore 2024-04-25T10:29:33.551 INFO:teuthology.orchestra.run.smithi106.stdout: osd op complaint time = 180 2024-04-25T10:29:33.551 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:29:33.551 INFO:teuthology.orchestra.run.smithi106.stdout:[mgr] 2024-04-25T10:29:33.551 INFO:teuthology.orchestra.run.smithi106.stdout: debug ms = 1 2024-04-25T10:29:33.551 INFO:teuthology.orchestra.run.smithi106.stdout: debug mgr = 20 2024-04-25T10:29:33.551 INFO:teuthology.orchestra.run.smithi106.stdout: debug mon = 20 2024-04-25T10:29:33.551 INFO:teuthology.orchestra.run.smithi106.stdout: debug auth = 20 2024-04-25T10:29:33.551 INFO:teuthology.orchestra.run.smithi106.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:29:33.551 INFO:teuthology.orchestra.run.smithi106.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:29:33.552 INFO:teuthology.orchestra.run.smithi106.stdout: mgr/telemetry/nag = false 2024-04-25T10:29:33.552 INFO:teuthology.orchestra.run.smithi106.stdout: debug client = 20 2024-04-25T10:29:33.552 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:29:33.552 INFO:teuthology.orchestra.run.smithi106.stdout:[mon] 2024-04-25T10:29:33.552 INFO:teuthology.orchestra.run.smithi106.stdout: debug ms = 1 2024-04-25T10:29:33.552 INFO:teuthology.orchestra.run.smithi106.stdout: debug mon = 20 2024-04-25T10:29:33.552 INFO:teuthology.orchestra.run.smithi106.stdout: debug paxos = 20 2024-04-25T10:29:33.552 INFO:teuthology.orchestra.run.smithi106.stdout: debug auth = 20 2024-04-25T10:29:33.552 INFO:teuthology.orchestra.run.smithi106.stdout: mon data avail warn = 5 2024-04-25T10:29:33.552 INFO:teuthology.orchestra.run.smithi106.stdout: mon mgr mkfs grace = 240 2024-04-25T10:29:33.552 INFO:teuthology.orchestra.run.smithi106.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:29:33.552 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd reporter subtree level = osd 2024-04-25T10:29:33.552 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd prime pg temp = true 2024-04-25T10:29:33.552 INFO:teuthology.orchestra.run.smithi106.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:29:33.552 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:29:33.553 INFO:teuthology.orchestra.run.smithi106.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:29:33.553 INFO:teuthology.orchestra.run.smithi106.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:29:33.553 INFO:teuthology.orchestra.run.smithi106.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:29:33.553 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:29:33.553 INFO:teuthology.orchestra.run.smithi106.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:29:33.553 INFO:teuthology.orchestra.run.smithi106.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:29:33.553 INFO:teuthology.orchestra.run.smithi106.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:29:33.553 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:29:33.553 INFO:teuthology.orchestra.run.smithi106.stdout: # 1m isn't quite enough 2024-04-25T10:29:33.553 INFO:teuthology.orchestra.run.smithi106.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:29:33.553 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:29:33.553 INFO:teuthology.orchestra.run.smithi106.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:29:33.553 INFO:teuthology.orchestra.run.smithi106.stdout: mon down mkfs grace = 300 2024-04-25T10:29:33.554 INFO:teuthology.orchestra.run.smithi106.stdout: mon op complaint time = 120 2024-04-25T10:29:33.554 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:29:33.554 INFO:teuthology.orchestra.run.smithi106.stdout:[client] 2024-04-25T10:29:33.554 INFO:teuthology.orchestra.run.smithi106.stdout: rgw cache enabled = true 2024-04-25T10:29:33.554 INFO:teuthology.orchestra.run.smithi106.stdout: rgw enable ops log = true 2024-04-25T10:29:33.554 INFO:teuthology.orchestra.run.smithi106.stdout: rgw enable usage log = true 2024-04-25T10:29:33.554 INFO:teuthology.orchestra.run.smithi106.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:29:33.554 INFO:teuthology.orchestra.run.smithi106.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:29:33.554 INFO:teuthology.orchestra.run.smithi106.stdout: client mount timeout = 600 2024-04-25T10:29:33.554 INFO:teuthology.orchestra.run.smithi106.stdout: client_shutdown_timeout = 120 2024-04-25T10:29:33.554 INFO:teuthology.orchestra.run.smithi106.stdout: debug client = 20 2024-04-25T10:29:33.554 INFO:teuthology.orchestra.run.smithi106.stdout: debug ms = 1 2024-04-25T10:29:33.554 INFO:teuthology.orchestra.run.smithi106.stdout: rados mon op timeout = 900 2024-04-25T10:29:33.554 INFO:teuthology.orchestra.run.smithi106.stdout: rados osd op timeout = 900 2024-04-25T10:29:33.554 INFO:teuthology.orchestra.run.smithi106.stdout:[mon.a] 2024-04-25T10:29:33.555 INFO:teuthology.orchestra.run.smithi106.stdout:[mon.b] 2024-04-25T10:29:33.555 INFO:teuthology.orchestra.run.smithi106.stdout:[mon.c] 2024-04-25T10:29:33.555 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.a] 2024-04-25T10:29:33.555 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.c] 2024-04-25T10:29:33.555 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.e] 2024-04-25T10:29:33.555 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.b] 2024-04-25T10:29:33.555 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.d] 2024-04-25T10:29:33.555 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.f] 2024-04-25T10:29:33.555 INFO:teuthology.orchestra.run.smithi106.stdout:[mds] 2024-04-25T10:29:33.555 INFO:teuthology.orchestra.run.smithi106.stdout: debug mds = 20 2024-04-25T10:29:33.555 INFO:teuthology.orchestra.run.smithi106.stdout: debug mds balancer = 20 2024-04-25T10:29:33.555 INFO:teuthology.orchestra.run.smithi106.stdout: debug ms = 1 2024-04-25T10:29:33.555 INFO:teuthology.orchestra.run.smithi106.stdout: mds bal fragment size max = 10000 2024-04-25T10:29:33.555 INFO:teuthology.orchestra.run.smithi106.stdout: mds bal merge size = 5 2024-04-25T10:29:33.556 INFO:teuthology.orchestra.run.smithi106.stdout: mds bal split bits = 3 2024-04-25T10:29:33.556 INFO:teuthology.orchestra.run.smithi106.stdout: mds bal split size = 100 2024-04-25T10:29:33.556 INFO:teuthology.orchestra.run.smithi106.stdout: mds debug frag = True 2024-04-25T10:29:33.556 INFO:teuthology.orchestra.run.smithi106.stdout: mds debug scatterstat = True 2024-04-25T10:29:33.556 INFO:teuthology.orchestra.run.smithi106.stdout: mds oft prefetch dirfrags = True 2024-04-25T10:29:33.556 INFO:teuthology.orchestra.run.smithi106.stdout: mds op complaint time = 180 2024-04-25T10:29:33.556 INFO:teuthology.orchestra.run.smithi106.stdout: mds verify scatter = True 2024-04-25T10:29:33.556 INFO:teuthology.orchestra.run.smithi106.stdout: mds_dir_prefetch = True 2024-04-25T10:29:33.556 INFO:teuthology.orchestra.run.smithi106.stdout: mds_sleep_rank_change = 5000000.0 2024-04-25T10:29:33.556 INFO:teuthology.orchestra.run.smithi106.stdout: mon client directed command retry = 5 2024-04-25T10:29:33.556 INFO:teuthology.orchestra.run.smithi106.stdout: ms inject delay max = 1 2024-04-25T10:29:33.556 INFO:teuthology.orchestra.run.smithi106.stdout: ms inject delay probability = 0.005 2024-04-25T10:29:33.556 INFO:teuthology.orchestra.run.smithi106.stdout: ms inject delay type = client mds osd 2024-04-25T10:29:33.556 INFO:teuthology.orchestra.run.smithi106.stdout: ms inject socket failures = 2500 2024-04-25T10:29:33.556 INFO:teuthology.orchestra.run.smithi106.stdout: osd op complaint time = 180 2024-04-25T10:29:33.556 INFO:teuthology.orchestra.run.smithi106.stdout: rados mon op timeout = 900 2024-04-25T10:29:33.557 INFO:teuthology.orchestra.run.smithi106.stdout: rados osd op timeout = 900 2024-04-25T10:29:33.557 INFO:teuthology.orchestra.run.smithi134.stdout:[global] 2024-04-25T10:29:33.557 INFO:teuthology.orchestra.run.smithi134.stdout: chdir = "" 2024-04-25T10:29:33.557 INFO:teuthology.orchestra.run.smithi134.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:29:33.557 INFO:teuthology.orchestra.run.smithi134.stdout: auth supported = cephx 2024-04-25T10:29:33.558 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T10:29:33.558 INFO:teuthology.orchestra.run.smithi134.stdout: filestore xattr use omap = true 2024-04-25T10:29:33.558 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T10:29:33.558 INFO:teuthology.orchestra.run.smithi134.stdout: mon clock drift allowed = 1.000 2024-04-25T10:29:33.558 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T10:29:33.558 INFO:teuthology.orchestra.run.smithi134.stdout: osd crush chooseleaf type = 0 2024-04-25T10:29:33.558 INFO:teuthology.orchestra.run.smithi134.stdout: auth debug = true 2024-04-25T10:29:33.558 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T10:29:33.558 INFO:teuthology.orchestra.run.smithi134.stdout: ms die on old message = true 2024-04-25T10:29:33.558 INFO:teuthology.orchestra.run.smithi134.stdout: ms die on bug = true 2024-04-25T10:29:33.558 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T10:29:33.558 INFO:teuthology.orchestra.run.smithi134.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:29:33.558 INFO:teuthology.orchestra.run.smithi134.stdout: mon pg warn max object skew = 0 2024-04-25T10:29:33.558 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T10:29:33.559 INFO:teuthology.orchestra.run.smithi134.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:29:33.559 INFO:teuthology.orchestra.run.smithi134.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:29:33.559 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T10:29:33.559 INFO:teuthology.orchestra.run.smithi134.stdout: osd pool default size = 2 2024-04-25T10:29:33.559 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T10:29:33.559 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd allow primary affinity = true 2024-04-25T10:29:33.559 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd allow pg remap = true 2024-04-25T10:29:33.559 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:29:33.559 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:29:33.559 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on no sortbitwise = false 2024-04-25T10:29:33.559 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:29:33.559 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on too few osds = false 2024-04-25T10:29:33.559 INFO:teuthology.orchestra.run.smithi134.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:29:33.559 INFO:teuthology.orchestra.run.smithi134.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:29:33.560 INFO:teuthology.orchestra.run.smithi134.stdout: mon_allow_pool_size_one = true 2024-04-25T10:29:33.560 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T10:29:33.560 INFO:teuthology.orchestra.run.smithi134.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:29:33.560 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T10:29:33.560 INFO:teuthology.orchestra.run.smithi134.stdout: osd default data pool replay window = 5 2024-04-25T10:29:33.560 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T10:29:33.560 INFO:teuthology.orchestra.run.smithi134.stdout: mon allow pool delete = true 2024-04-25T10:29:33.560 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T10:29:33.560 INFO:teuthology.orchestra.run.smithi134.stdout: mon cluster log file level = debug 2024-04-25T10:29:33.560 INFO:teuthology.orchestra.run.smithi134.stdout: debug asserts on shutdown = true 2024-04-25T10:29:33.560 INFO:teuthology.orchestra.run.smithi134.stdout: mon health detail to clog = false 2024-04-25T10:29:33.560 INFO:teuthology.orchestra.run.smithi134.stdout: mon host = "172.21.15.106,172.21.15.134,[v2:172.21.15.134:3301,v1:172.21.15.134:6790]" 2024-04-25T10:29:33.560 INFO:teuthology.orchestra.run.smithi134.stdout: fsid = 3ee77985-e8e4-41cf-81e9-7c774cf7891f 2024-04-25T10:29:33.560 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T10:29:33.560 INFO:teuthology.orchestra.run.smithi134.stdout:[osd] 2024-04-25T10:29:33.561 INFO:teuthology.orchestra.run.smithi134.stdout: osd journal size = 100 2024-04-25T10:29:33.561 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T10:29:33.561 INFO:teuthology.orchestra.run.smithi134.stdout: osd scrub load threshold = 5.0 2024-04-25T10:29:33.561 INFO:teuthology.orchestra.run.smithi134.stdout: osd scrub max interval = 600 2024-04-25T10:29:33.561 INFO:teuthology.orchestra.run.smithi134.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:29:33.561 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T10:29:33.561 INFO:teuthology.orchestra.run.smithi134.stdout: osd recover clone overlap = true 2024-04-25T10:29:33.561 INFO:teuthology.orchestra.run.smithi134.stdout: osd recovery max chunk = 1048576 2024-04-25T10:29:33.561 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T10:29:33.561 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug shutdown = true 2024-04-25T10:29:33.561 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug op order = true 2024-04-25T10:29:33.561 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug verify stray on activate = true 2024-04-25T10:29:33.561 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug trim objects = true 2024-04-25T10:29:33.561 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T10:29:33.561 INFO:teuthology.orchestra.run.smithi134.stdout: osd open classes on start = true 2024-04-25T10:29:33.562 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug pg log writeout = true 2024-04-25T10:29:33.562 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T10:29:33.562 INFO:teuthology.orchestra.run.smithi134.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:29:33.562 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T10:29:33.562 INFO:teuthology.orchestra.run.smithi134.stdout: osd map max advance = 10 2024-04-25T10:29:33.562 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T10:29:33.562 INFO:teuthology.orchestra.run.smithi134.stdout: journal zero on create = true 2024-04-25T10:29:33.562 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T10:29:33.562 INFO:teuthology.orchestra.run.smithi134.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:29:33.562 INFO:teuthology.orchestra.run.smithi134.stdout: filestore apply finisher threads = 3 2024-04-25T10:29:33.562 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T10:29:33.562 INFO:teuthology.orchestra.run.smithi134.stdout: bdev debug aio = true 2024-04-25T10:29:33.562 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug misdirected ops = true 2024-04-25T10:29:33.562 INFO:teuthology.orchestra.run.smithi134.stdout: bdev async discard = True 2024-04-25T10:29:33.563 INFO:teuthology.orchestra.run.smithi134.stdout: bdev enable discard = True 2024-04-25T10:29:33.563 INFO:teuthology.orchestra.run.smithi134.stdout: bluestore allocator = bitmap 2024-04-25T10:29:33.563 INFO:teuthology.orchestra.run.smithi134.stdout: bluestore block size = 96636764160 2024-04-25T10:29:33.563 INFO:teuthology.orchestra.run.smithi134.stdout: bluestore fsck on mount = True 2024-04-25T10:29:33.563 INFO:teuthology.orchestra.run.smithi134.stdout: debug bluefs = 1/20 2024-04-25T10:29:33.563 INFO:teuthology.orchestra.run.smithi134.stdout: debug bluestore = 1/20 2024-04-25T10:29:33.563 INFO:teuthology.orchestra.run.smithi134.stdout: debug ms = 1 2024-04-25T10:29:33.563 INFO:teuthology.orchestra.run.smithi134.stdout: debug osd = 20 2024-04-25T10:29:33.563 INFO:teuthology.orchestra.run.smithi134.stdout: debug rocksdb = 4/10 2024-04-25T10:29:33.563 INFO:teuthology.orchestra.run.smithi134.stdout: mon client directed command retry = 5 2024-04-25T10:29:33.563 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:29:33.563 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd full ratio = 0.9 2024-04-25T10:29:33.563 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:29:33.563 INFO:teuthology.orchestra.run.smithi134.stdout: ms inject delay max = 1 2024-04-25T10:29:33.563 INFO:teuthology.orchestra.run.smithi134.stdout: ms inject delay probability = 0.005 2024-04-25T10:29:33.564 INFO:teuthology.orchestra.run.smithi134.stdout: ms inject delay type = client mds 2024-04-25T10:29:33.564 INFO:teuthology.orchestra.run.smithi134.stdout: ms inject socket failures = 2500 2024-04-25T10:29:33.564 INFO:teuthology.orchestra.run.smithi134.stdout: osd blocked scrub grace period = 3600 2024-04-25T10:29:33.564 INFO:teuthology.orchestra.run.smithi134.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:29:33.564 INFO:teuthology.orchestra.run.smithi134.stdout: osd max markdown count = 1000 2024-04-25T10:29:33.564 INFO:teuthology.orchestra.run.smithi134.stdout: osd objectstore = bluestore 2024-04-25T10:29:33.564 INFO:teuthology.orchestra.run.smithi134.stdout: osd op complaint time = 180 2024-04-25T10:29:33.564 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T10:29:33.564 INFO:teuthology.orchestra.run.smithi134.stdout:[mgr] 2024-04-25T10:29:33.564 INFO:teuthology.orchestra.run.smithi134.stdout: debug ms = 1 2024-04-25T10:29:33.564 INFO:teuthology.orchestra.run.smithi134.stdout: debug mgr = 20 2024-04-25T10:29:33.564 INFO:teuthology.orchestra.run.smithi134.stdout: debug mon = 20 2024-04-25T10:29:33.564 INFO:teuthology.orchestra.run.smithi134.stdout: debug auth = 20 2024-04-25T10:29:33.564 INFO:teuthology.orchestra.run.smithi134.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:29:33.564 INFO:teuthology.orchestra.run.smithi134.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:29:33.565 INFO:teuthology.orchestra.run.smithi134.stdout: mgr/telemetry/nag = false 2024-04-25T10:29:33.565 INFO:teuthology.orchestra.run.smithi134.stdout: debug client = 20 2024-04-25T10:29:33.565 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T10:29:33.565 INFO:teuthology.orchestra.run.smithi134.stdout:[mon] 2024-04-25T10:29:33.565 INFO:teuthology.orchestra.run.smithi134.stdout: debug ms = 1 2024-04-25T10:29:33.565 INFO:teuthology.orchestra.run.smithi134.stdout: debug mon = 20 2024-04-25T10:29:33.565 INFO:teuthology.orchestra.run.smithi134.stdout: debug paxos = 20 2024-04-25T10:29:33.565 INFO:teuthology.orchestra.run.smithi134.stdout: debug auth = 20 2024-04-25T10:29:33.565 INFO:teuthology.orchestra.run.smithi134.stdout: mon data avail warn = 5 2024-04-25T10:29:33.565 INFO:teuthology.orchestra.run.smithi134.stdout: mon mgr mkfs grace = 240 2024-04-25T10:29:33.565 INFO:teuthology.orchestra.run.smithi134.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:29:33.565 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd reporter subtree level = osd 2024-04-25T10:29:33.565 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd prime pg temp = true 2024-04-25T10:29:33.565 INFO:teuthology.orchestra.run.smithi134.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:29:33.565 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T10:29:33.566 INFO:teuthology.orchestra.run.smithi134.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:29:33.566 INFO:teuthology.orchestra.run.smithi134.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:29:33.566 INFO:teuthology.orchestra.run.smithi134.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:29:33.566 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T10:29:33.566 INFO:teuthology.orchestra.run.smithi134.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:29:33.566 INFO:teuthology.orchestra.run.smithi134.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:29:33.566 INFO:teuthology.orchestra.run.smithi134.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:29:33.566 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T10:29:33.566 INFO:teuthology.orchestra.run.smithi134.stdout: # 1m isn't quite enough 2024-04-25T10:29:33.566 INFO:teuthology.orchestra.run.smithi134.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:29:33.566 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T10:29:33.566 INFO:teuthology.orchestra.run.smithi134.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:29:33.566 INFO:teuthology.orchestra.run.smithi134.stdout: mon down mkfs grace = 300 2024-04-25T10:29:33.566 INFO:teuthology.orchestra.run.smithi134.stdout: mon op complaint time = 120 2024-04-25T10:29:33.567 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-25T10:29:33.567 INFO:teuthology.orchestra.run.smithi134.stdout:[client] 2024-04-25T10:29:33.567 INFO:teuthology.orchestra.run.smithi134.stdout: rgw cache enabled = true 2024-04-25T10:29:33.567 INFO:teuthology.orchestra.run.smithi134.stdout: rgw enable ops log = true 2024-04-25T10:29:33.567 INFO:teuthology.orchestra.run.smithi134.stdout: rgw enable usage log = true 2024-04-25T10:29:33.567 INFO:teuthology.orchestra.run.smithi134.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:29:33.567 INFO:teuthology.orchestra.run.smithi134.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:29:33.567 INFO:teuthology.orchestra.run.smithi134.stdout: client mount timeout = 600 2024-04-25T10:29:33.567 INFO:teuthology.orchestra.run.smithi134.stdout: client_shutdown_timeout = 120 2024-04-25T10:29:33.567 INFO:teuthology.orchestra.run.smithi134.stdout: debug client = 20 2024-04-25T10:29:33.567 INFO:teuthology.orchestra.run.smithi134.stdout: debug ms = 1 2024-04-25T10:29:33.567 INFO:teuthology.orchestra.run.smithi134.stdout: rados mon op timeout = 900 2024-04-25T10:29:33.567 INFO:teuthology.orchestra.run.smithi134.stdout: rados osd op timeout = 900 2024-04-25T10:29:33.567 INFO:teuthology.orchestra.run.smithi134.stdout:[mon.a] 2024-04-25T10:29:33.567 INFO:teuthology.orchestra.run.smithi134.stdout:[mon.b] 2024-04-25T10:29:33.568 INFO:teuthology.orchestra.run.smithi134.stdout:[mon.c] 2024-04-25T10:29:33.568 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.a] 2024-04-25T10:29:33.568 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.c] 2024-04-25T10:29:33.568 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.e] 2024-04-25T10:29:33.568 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.b] 2024-04-25T10:29:33.568 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.d] 2024-04-25T10:29:33.568 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.f] 2024-04-25T10:29:33.568 INFO:teuthology.orchestra.run.smithi134.stdout:[mds] 2024-04-25T10:29:33.568 INFO:teuthology.orchestra.run.smithi134.stdout: debug mds = 20 2024-04-25T10:29:33.568 INFO:teuthology.orchestra.run.smithi134.stdout: debug mds balancer = 20 2024-04-25T10:29:33.568 INFO:teuthology.orchestra.run.smithi134.stdout: debug ms = 1 2024-04-25T10:29:33.568 INFO:teuthology.orchestra.run.smithi134.stdout: mds bal fragment size max = 10000 2024-04-25T10:29:33.568 INFO:teuthology.orchestra.run.smithi134.stdout: mds bal merge size = 5 2024-04-25T10:29:33.568 INFO:teuthology.orchestra.run.smithi134.stdout: mds bal split bits = 3 2024-04-25T10:29:33.568 INFO:teuthology.orchestra.run.smithi134.stdout: mds bal split size = 100 2024-04-25T10:29:33.569 INFO:teuthology.orchestra.run.smithi134.stdout: mds debug frag = True 2024-04-25T10:29:33.569 INFO:teuthology.orchestra.run.smithi134.stdout: mds debug scatterstat = True 2024-04-25T10:29:33.569 INFO:teuthology.orchestra.run.smithi134.stdout: mds oft prefetch dirfrags = True 2024-04-25T10:29:33.569 INFO:teuthology.orchestra.run.smithi134.stdout: mds op complaint time = 180 2024-04-25T10:29:33.569 INFO:teuthology.orchestra.run.smithi134.stdout: mds verify scatter = True 2024-04-25T10:29:33.569 INFO:teuthology.orchestra.run.smithi134.stdout: mds_dir_prefetch = True 2024-04-25T10:29:33.569 INFO:teuthology.orchestra.run.smithi134.stdout: mds_sleep_rank_change = 5000000.0 2024-04-25T10:29:33.569 INFO:teuthology.orchestra.run.smithi134.stdout: mon client directed command retry = 5 2024-04-25T10:29:33.569 INFO:teuthology.orchestra.run.smithi134.stdout: ms inject delay max = 1 2024-04-25T10:29:33.569 INFO:teuthology.orchestra.run.smithi134.stdout: ms inject delay probability = 0.005 2024-04-25T10:29:33.569 INFO:teuthology.orchestra.run.smithi134.stdout: ms inject delay type = client mds osd 2024-04-25T10:29:33.569 INFO:teuthology.orchestra.run.smithi134.stdout: ms inject socket failures = 2500 2024-04-25T10:29:33.569 INFO:teuthology.orchestra.run.smithi134.stdout: osd op complaint time = 180 2024-04-25T10:29:33.569 INFO:teuthology.orchestra.run.smithi134.stdout: rados mon op timeout = 900 2024-04-25T10:29:33.569 INFO:teuthology.orchestra.run.smithi134.stdout: rados osd op timeout = 900 2024-04-25T10:29:33.571 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-25T10:29:33.571 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-25T10:29:33.674 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-25T10:29:33.675 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T10:29:33.675 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-25T10:29:33.689 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T10:29:33.689 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-25T10:29:33.744 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi106.front.sepia.ceph.com 2024-04-25T10:29:33.744 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T10:29:33.744 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T10:29:33.744 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T10:29:33.816 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T10:29:33.816 DEBUG:teuthology.orchestra.run.smithi106:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:29:33.871 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi134.front.sepia.ceph.com 2024-04-25T10:29:33.871 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-25T10:29:33.871 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T10:29:33.871 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T10:29:33.904 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-25T10:29:33.904 DEBUG:teuthology.orchestra.run.smithi134:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:29:33.959 INFO:tasks.ceph:Setting up mon nodes... 2024-04-25T10:29:33.960 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-25T10:29:33.960 DEBUG:teuthology.orchestra.run.smithi106:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T10:29:34.017 INFO:teuthology.orchestra.run.smithi106.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T10:29:34.020 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T10:29:34.106 INFO:teuthology.orchestra.run.smithi134.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T10:29:34.109 INFO:tasks.ceph:Setting up mds nodes... 2024-04-25T10:29:34.110 DEBUG:teuthology.orchestra.run.smithi106:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T10:29:34.169 INFO:teuthology.orchestra.run.smithi106.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T10:29:34.172 DEBUG:teuthology.orchestra.run.smithi106:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-25T10:29:34.195 DEBUG:teuthology.orchestra.run.smithi106:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T10:29:34.292 INFO:teuthology.orchestra.run.smithi106.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T10:29:34.295 DEBUG:teuthology.orchestra.run.smithi106:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-25T10:29:34.318 DEBUG:teuthology.orchestra.run.smithi106:> sudo mkdir -p /var/lib/ceph/mds/ceph-e && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.e /var/lib/ceph/mds/ceph-e/keyring 2024-04-25T10:29:34.416 INFO:teuthology.orchestra.run.smithi106.stdout:creating /var/lib/ceph/mds/ceph-e/keyring 2024-04-25T10:29:34.419 DEBUG:teuthology.orchestra.run.smithi106:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-e 2024-04-25T10:29:34.443 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T10:29:34.502 INFO:teuthology.orchestra.run.smithi134.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T10:29:34.505 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-25T10:29:34.528 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T10:29:34.625 INFO:teuthology.orchestra.run.smithi134.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T10:29:34.628 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-25T10:29:34.651 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/mds/ceph-f && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.f /var/lib/ceph/mds/ceph-f/keyring 2024-04-25T10:29:34.749 INFO:teuthology.orchestra.run.smithi134.stdout:creating /var/lib/ceph/mds/ceph-f/keyring 2024-04-25T10:29:34.752 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-f 2024-04-25T10:29:34.776 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-25T10:29:34.777 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T10:29:34.826 INFO:teuthology.orchestra.run.smithi106.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-25T10:29:34.838 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-25T10:29:34.838 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi106.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi134.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T10:29:34.839 DEBUG:teuthology.orchestra.run.smithi106:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-25T10:29:34.862 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T10:29:34.862 INFO:tasks.ceph:role: osd.0 2024-04-25T10:29:34.862 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi106.front.sepia.ceph.com 2024-04-25T10:29:34.862 DEBUG:teuthology.orchestra.run.smithi106:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T10:29:34.932 INFO:teuthology.orchestra.run.smithi106.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:29:34.933 INFO:teuthology.orchestra.run.smithi106.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:29:34.933 INFO:teuthology.orchestra.run.smithi106.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:29:34.933 INFO:teuthology.orchestra.run.smithi106.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T10:29:34.933 INFO:teuthology.orchestra.run.smithi106.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:29:34.933 INFO:teuthology.orchestra.run.smithi106.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:29:34.933 INFO:teuthology.orchestra.run.smithi106.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:29:34.933 INFO:teuthology.orchestra.run.smithi106.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T10:29:34.933 INFO:teuthology.orchestra.run.smithi106.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:29:34.933 INFO:teuthology.orchestra.run.smithi106.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:29:35.452 INFO:teuthology.orchestra.run.smithi106.stdout:Discarding blocks...Done. 2024-04-25T10:29:35.454 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi106.front.sepia.ceph.com -o noatime 2024-04-25T10:29:35.454 DEBUG:teuthology.orchestra.run.smithi106:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-25T10:29:35.592 DEBUG:teuthology.orchestra.run.smithi106:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-25T10:29:35.622 DEBUG:teuthology.orchestra.run.smithi106:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-25T10:29:35.686 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T10:29:35.686 INFO:tasks.ceph:role: osd.1 2024-04-25T10:29:35.686 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi106.front.sepia.ceph.com 2024-04-25T10:29:35.686 DEBUG:teuthology.orchestra.run.smithi106:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T10:29:35.756 INFO:teuthology.orchestra.run.smithi106.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:29:35.756 INFO:teuthology.orchestra.run.smithi106.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:29:35.756 INFO:teuthology.orchestra.run.smithi106.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:29:35.757 INFO:teuthology.orchestra.run.smithi106.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T10:29:35.757 INFO:teuthology.orchestra.run.smithi106.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:29:35.757 INFO:teuthology.orchestra.run.smithi106.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:29:35.757 INFO:teuthology.orchestra.run.smithi106.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:29:35.757 INFO:teuthology.orchestra.run.smithi106.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T10:29:35.757 INFO:teuthology.orchestra.run.smithi106.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:29:35.757 INFO:teuthology.orchestra.run.smithi106.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:29:36.205 INFO:teuthology.orchestra.run.smithi106.stdout:Discarding blocks...Done. 2024-04-25T10:29:36.207 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi106.front.sepia.ceph.com -o noatime 2024-04-25T10:29:36.207 DEBUG:teuthology.orchestra.run.smithi106:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-25T10:29:36.308 DEBUG:teuthology.orchestra.run.smithi106:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-25T10:29:36.336 DEBUG:teuthology.orchestra.run.smithi106:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-25T10:29:36.399 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T10:29:36.400 INFO:tasks.ceph:role: osd.2 2024-04-25T10:29:36.400 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi106.front.sepia.ceph.com 2024-04-25T10:29:36.400 DEBUG:teuthology.orchestra.run.smithi106:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T10:29:36.471 INFO:teuthology.orchestra.run.smithi106.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:29:36.471 INFO:teuthology.orchestra.run.smithi106.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:29:36.471 INFO:teuthology.orchestra.run.smithi106.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:29:36.472 INFO:teuthology.orchestra.run.smithi106.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T10:29:36.472 INFO:teuthology.orchestra.run.smithi106.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:29:36.472 INFO:teuthology.orchestra.run.smithi106.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:29:36.472 INFO:teuthology.orchestra.run.smithi106.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:29:36.472 INFO:teuthology.orchestra.run.smithi106.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T10:29:36.472 INFO:teuthology.orchestra.run.smithi106.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:29:36.472 INFO:teuthology.orchestra.run.smithi106.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:29:36.957 INFO:teuthology.orchestra.run.smithi106.stdout:Discarding blocks...Done. 2024-04-25T10:29:36.959 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi106.front.sepia.ceph.com -o noatime 2024-04-25T10:29:36.959 DEBUG:teuthology.orchestra.run.smithi106:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-25T10:29:37.106 DEBUG:teuthology.orchestra.run.smithi106:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-25T10:29:37.134 DEBUG:teuthology.orchestra.run.smithi106:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-25T10:29:37.198 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T10:29:37.198 INFO:tasks.ceph:role: osd.3 2024-04-25T10:29:37.198 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi106.front.sepia.ceph.com 2024-04-25T10:29:37.198 DEBUG:teuthology.orchestra.run.smithi106:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T10:29:37.269 INFO:teuthology.orchestra.run.smithi106.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:29:37.269 INFO:teuthology.orchestra.run.smithi106.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:29:37.269 INFO:teuthology.orchestra.run.smithi106.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:29:37.270 INFO:teuthology.orchestra.run.smithi106.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T10:29:37.270 INFO:teuthology.orchestra.run.smithi106.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:29:37.270 INFO:teuthology.orchestra.run.smithi106.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:29:37.270 INFO:teuthology.orchestra.run.smithi106.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:29:37.270 INFO:teuthology.orchestra.run.smithi106.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T10:29:37.270 INFO:teuthology.orchestra.run.smithi106.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:29:37.270 INFO:teuthology.orchestra.run.smithi106.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:29:37.791 INFO:teuthology.orchestra.run.smithi106.stdout:Discarding blocks...Done. 2024-04-25T10:29:37.793 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi106.front.sepia.ceph.com -o noatime 2024-04-25T10:29:37.793 DEBUG:teuthology.orchestra.run.smithi106:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-25T10:29:37.938 DEBUG:teuthology.orchestra.run.smithi106:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-25T10:29:37.968 DEBUG:teuthology.orchestra.run.smithi106:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:29:38.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:29:38.072+0000 7f399b40c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-25T10:29:38.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:29:38.073+0000 7f399b40c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-25T10:29:38.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:29:38.073+0000 7f399b40c640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T10:29:38.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:29:38.073+0000 7f399b40c640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T10:29:38.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:29:38.073+0000 7f399b40c640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T10:29:38.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:29:38.073+0000 7f399b40c640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-25T10:29:40.927 DEBUG:teuthology.orchestra.run.smithi106:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-25T10:29:40.956 DEBUG:teuthology.orchestra.run.smithi106:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:29:41.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:29:41.058+0000 7f2b283a1640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-25T10:29:41.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:29:41.059+0000 7f2b283a1640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-25T10:29:41.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:29:41.059+0000 7f2b283a1640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T10:29:41.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:29:41.059+0000 7f2b283a1640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T10:29:41.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:29:41.059+0000 7f2b283a1640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T10:29:41.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:29:41.059+0000 7f2b283a1640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-25T10:29:43.922 DEBUG:teuthology.orchestra.run.smithi106:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-25T10:29:43.949 DEBUG:teuthology.orchestra.run.smithi106:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:29:44.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:29:44.042+0000 7f36cca16640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-25T10:29:44.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:29:44.042+0000 7f36cca16640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-25T10:29:44.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:29:44.042+0000 7f36cca16640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T10:29:44.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:29:44.042+0000 7f36cca16640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T10:29:44.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:29:44.042+0000 7f36cca16640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T10:29:44.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:29:44.043+0000 7f36cca16640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-25T10:29:46.913 DEBUG:teuthology.orchestra.run.smithi106:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-25T10:29:46.939 DEBUG:teuthology.orchestra.run.smithi106:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:29:47.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:29:47.034+0000 7fefb8a43640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-25T10:29:47.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:29:47.034+0000 7fefb8a43640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-25T10:29:47.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:29:47.034+0000 7fefb8a43640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T10:29:47.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:29:47.034+0000 7fefb8a43640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T10:29:47.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:29:47.034+0000 7fefb8a43640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T10:29:47.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:29:47.035+0000 7fefb8a43640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-25T10:29:49.887 DEBUG:teuthology.orchestra.run.smithi106:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-25T10:29:49.915 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-25T10:29:49.943 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T10:29:49.944 INFO:tasks.ceph:role: osd.4 2024-04-25T10:29:49.944 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi134.front.sepia.ceph.com 2024-04-25T10:29:49.944 DEBUG:teuthology.orchestra.run.smithi134:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T10:29:50.017 INFO:teuthology.orchestra.run.smithi134.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:29:50.017 INFO:teuthology.orchestra.run.smithi134.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:29:50.017 INFO:teuthology.orchestra.run.smithi134.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:29:50.017 INFO:teuthology.orchestra.run.smithi134.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T10:29:50.017 INFO:teuthology.orchestra.run.smithi134.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:29:50.017 INFO:teuthology.orchestra.run.smithi134.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:29:50.017 INFO:teuthology.orchestra.run.smithi134.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:29:50.018 INFO:teuthology.orchestra.run.smithi134.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T10:29:50.018 INFO:teuthology.orchestra.run.smithi134.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:29:50.018 INFO:teuthology.orchestra.run.smithi134.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:29:50.520 INFO:teuthology.orchestra.run.smithi134.stdout:Discarding blocks...Done. 2024-04-25T10:29:50.522 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi134.front.sepia.ceph.com -o noatime 2024-04-25T10:29:50.523 DEBUG:teuthology.orchestra.run.smithi134:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-25T10:29:50.626 DEBUG:teuthology.orchestra.run.smithi134:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-25T10:29:50.661 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-25T10:29:50.727 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T10:29:50.727 INFO:tasks.ceph:role: osd.5 2024-04-25T10:29:50.727 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi134.front.sepia.ceph.com 2024-04-25T10:29:50.727 DEBUG:teuthology.orchestra.run.smithi134:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T10:29:50.799 INFO:teuthology.orchestra.run.smithi134.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:29:50.799 INFO:teuthology.orchestra.run.smithi134.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:29:50.799 INFO:teuthology.orchestra.run.smithi134.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:29:50.799 INFO:teuthology.orchestra.run.smithi134.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T10:29:50.799 INFO:teuthology.orchestra.run.smithi134.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:29:50.800 INFO:teuthology.orchestra.run.smithi134.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:29:50.800 INFO:teuthology.orchestra.run.smithi134.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:29:50.800 INFO:teuthology.orchestra.run.smithi134.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T10:29:50.800 INFO:teuthology.orchestra.run.smithi134.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:29:50.800 INFO:teuthology.orchestra.run.smithi134.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:29:51.251 INFO:teuthology.orchestra.run.smithi134.stdout:Discarding blocks...Done. 2024-04-25T10:29:51.253 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi134.front.sepia.ceph.com -o noatime 2024-04-25T10:29:51.253 DEBUG:teuthology.orchestra.run.smithi134:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-25T10:29:51.335 DEBUG:teuthology.orchestra.run.smithi134:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-25T10:29:51.402 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-25T10:29:51.466 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T10:29:51.466 INFO:tasks.ceph:role: osd.6 2024-04-25T10:29:51.466 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi134.front.sepia.ceph.com 2024-04-25T10:29:51.466 DEBUG:teuthology.orchestra.run.smithi134:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T10:29:51.537 INFO:teuthology.orchestra.run.smithi134.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:29:51.537 INFO:teuthology.orchestra.run.smithi134.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:29:51.537 INFO:teuthology.orchestra.run.smithi134.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:29:51.537 INFO:teuthology.orchestra.run.smithi134.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T10:29:51.537 INFO:teuthology.orchestra.run.smithi134.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:29:51.537 INFO:teuthology.orchestra.run.smithi134.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:29:51.537 INFO:teuthology.orchestra.run.smithi134.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:29:51.537 INFO:teuthology.orchestra.run.smithi134.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T10:29:51.537 INFO:teuthology.orchestra.run.smithi134.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:29:51.537 INFO:teuthology.orchestra.run.smithi134.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:29:52.025 INFO:teuthology.orchestra.run.smithi134.stdout:Discarding blocks...Done. 2024-04-25T10:29:52.026 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi134.front.sepia.ceph.com -o noatime 2024-04-25T10:29:52.027 DEBUG:teuthology.orchestra.run.smithi134:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-25T10:29:52.161 DEBUG:teuthology.orchestra.run.smithi134:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-25T10:29:52.191 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-25T10:29:52.256 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T10:29:52.256 INFO:tasks.ceph:role: osd.7 2024-04-25T10:29:52.256 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi134.front.sepia.ceph.com 2024-04-25T10:29:52.256 DEBUG:teuthology.orchestra.run.smithi134:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T10:29:52.327 INFO:teuthology.orchestra.run.smithi134.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:29:52.327 INFO:teuthology.orchestra.run.smithi134.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:29:52.327 INFO:teuthology.orchestra.run.smithi134.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:29:52.327 INFO:teuthology.orchestra.run.smithi134.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T10:29:52.327 INFO:teuthology.orchestra.run.smithi134.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:29:52.327 INFO:teuthology.orchestra.run.smithi134.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:29:52.327 INFO:teuthology.orchestra.run.smithi134.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:29:52.327 INFO:teuthology.orchestra.run.smithi134.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T10:29:52.327 INFO:teuthology.orchestra.run.smithi134.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:29:52.328 INFO:teuthology.orchestra.run.smithi134.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:29:52.846 INFO:teuthology.orchestra.run.smithi134.stdout:Discarding blocks...Done. 2024-04-25T10:29:52.848 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi134.front.sepia.ceph.com -o noatime 2024-04-25T10:29:52.849 DEBUG:teuthology.orchestra.run.smithi134:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-25T10:29:52.981 DEBUG:teuthology.orchestra.run.smithi134:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-25T10:29:53.010 DEBUG:teuthology.orchestra.run.smithi134:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:29:53.119 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:29:53.117+0000 7fac59e5f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-25T10:29:53.119 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:29:53.117+0000 7fac59e5f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-25T10:29:53.119 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:29:53.117+0000 7fac59e5f640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T10:29:53.119 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:29:53.117+0000 7fac59e5f640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T10:29:53.119 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:29:53.117+0000 7fac59e5f640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T10:29:53.119 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:29:53.117+0000 7fac59e5f640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-25T10:29:55.967 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-25T10:29:55.995 DEBUG:teuthology.orchestra.run.smithi134:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:29:56.097 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:29:56.095+0000 7f017f632640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-25T10:29:56.097 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:29:56.095+0000 7f017f632640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-25T10:29:56.097 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:29:56.095+0000 7f017f632640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T10:29:56.097 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:29:56.095+0000 7f017f632640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T10:29:56.097 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:29:56.095+0000 7f017f632640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T10:29:56.097 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:29:56.096+0000 7f017f632640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-25T10:29:58.947 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-25T10:29:58.975 DEBUG:teuthology.orchestra.run.smithi134:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:29:59.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:29:59.074+0000 7f9d3e6b5640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-25T10:29:59.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:29:59.074+0000 7f9d3e6b5640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-25T10:29:59.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:29:59.074+0000 7f9d3e6b5640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T10:29:59.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:29:59.074+0000 7f9d3e6b5640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T10:29:59.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:29:59.074+0000 7f9d3e6b5640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T10:29:59.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:29:59.074+0000 7f9d3e6b5640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-25T10:30:01.595 DEBUG:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:30:01.598 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:30:01.924 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-25T10:30:01.949 DEBUG:teuthology.orchestra.run.smithi134:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:30:02.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:02.044+0000 7f6bf0a2d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-25T10:30:02.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:02.044+0000 7f6bf0a2d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-25T10:30:02.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:02.044+0000 7f6bf0a2d640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T10:30:02.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:02.044+0000 7f6bf0a2d640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T10:30:02.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:02.044+0000 7f6bf0a2d640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T10:30:02.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:02.045+0000 7f6bf0a2d640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-25T10:30:04.904 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-25T10:30:04.932 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-25T10:30:04.932 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T10:30:04.932 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-25T10:30:04.961 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T10:30:04.961 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-25T10:30:05.028 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T10:30:05.029 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-25T10:30:05.096 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T10:30:05.096 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/var/lib/ceph/mds/ceph-e/keyring of=/dev/stdout 2024-04-25T10:30:05.162 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T10:30:05.162 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-25T10:30:05.228 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T10:30:05.228 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-25T10:30:05.293 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T10:30:05.293 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-25T10:30:05.358 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T10:30:05.358 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-25T10:30:05.422 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-25T10:30:05.422 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-25T10:30:05.451 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-25T10:30:05.451 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-25T10:30:05.517 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-25T10:30:05.517 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-25T10:30:05.584 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-25T10:30:05.584 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/mds/ceph-f/keyring of=/dev/stdout 2024-04-25T10:30:05.649 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-25T10:30:05.649 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-25T10:30:05.715 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-25T10:30:05.715 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-25T10:30:05.781 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-25T10:30:05.782 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-25T10:30:05.847 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-25T10:30:05.848 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-25T10:30:05.912 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T10:30:05.912 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-25T10:30:05.930 INFO:tasks.ceph:Adding keys to all mons... 2024-04-25T10:30:05.930 DEBUG:teuthology.orchestra.run.smithi106:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T10:30:05.973 DEBUG:teuthology.orchestra.run.smithi134:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T10:30:05.995 INFO:teuthology.orchestra.run.smithi106.stdout:[mgr.y] 2024-04-25T10:30:05.995 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQCOMCpmhi32ABAAweMHCNXIWkWqkg9fmiT92g== 2024-04-25T10:30:05.995 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.a] 2024-04-25T10:30:05.995 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQCOMCpmwUwHChAAUg0EpkAJCrjAuS4wJMr58Q== 2024-04-25T10:30:05.995 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.c] 2024-04-25T10:30:05.995 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQCOMCpmKeNgERAAW7Rbw2nkBzVVwaT9rSNkpg== 2024-04-25T10:30:05.996 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.e] 2024-04-25T10:30:05.996 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQCOMCpmIFHJGBAAO2cIxzly3biv4KJsKe9Ttg== 2024-04-25T10:30:05.996 INFO:teuthology.orchestra.run.smithi106.stdout:[osd.0] 2024-04-25T10:30:05.996 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQCSMCpmQa1bBBAA/KKm97toQCLJT2ogo2ZliA== 2024-04-25T10:30:05.996 INFO:teuthology.orchestra.run.smithi106.stdout:[osd.1] 2024-04-25T10:30:05.996 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQCVMCpmQEmWAxAAlNBgdgu/Ih/eFLVEA1E78w== 2024-04-25T10:30:05.996 INFO:teuthology.orchestra.run.smithi106.stdout:[osd.2] 2024-04-25T10:30:05.996 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQCYMCpmUTGeAhAAX1bcS//iaaTrB200vdN/bw== 2024-04-25T10:30:05.996 INFO:teuthology.orchestra.run.smithi106.stdout:[osd.3] 2024-04-25T10:30:05.996 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQCbMCpmFREfAhAAbiWZqEqEGKfPlhcH30sWRw== 2024-04-25T10:30:05.996 INFO:teuthology.orchestra.run.smithi106.stdout:[mgr.x] 2024-04-25T10:30:05.996 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQCOMCpm8JFQBhAA/1Z+OBxquOVgQtMM0x14uQ== 2024-04-25T10:30:05.996 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.b] 2024-04-25T10:30:05.997 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQCOMCpmBGviHRAA+pPrYCjotv9ZuoCHs6GVew== 2024-04-25T10:30:05.997 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.d] 2024-04-25T10:30:05.997 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQCOMCpmFU1AJRAAO9cDvsSVcotvFW6RfvbOCg== 2024-04-25T10:30:05.997 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.f] 2024-04-25T10:30:05.997 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQCOMCpmBcedLBAAI1wAXuF6+8FkVMwwicONhw== 2024-04-25T10:30:05.997 INFO:teuthology.orchestra.run.smithi106.stdout:[osd.4] 2024-04-25T10:30:05.997 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQChMCpmjTkGBxAAg4sTg4Kfv/KCwR6Ck8etOA== 2024-04-25T10:30:05.997 INFO:teuthology.orchestra.run.smithi106.stdout:[osd.5] 2024-04-25T10:30:05.997 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQCkMCpmdze2BRAAhHM/mvM7HA5pdGrKjG5FRQ== 2024-04-25T10:30:05.997 INFO:teuthology.orchestra.run.smithi106.stdout:[osd.6] 2024-04-25T10:30:05.997 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQCnMCpmHaN1BBAAsezgDKhMWInFig+7DiTEuA== 2024-04-25T10:30:05.997 INFO:teuthology.orchestra.run.smithi106.stdout:[osd.7] 2024-04-25T10:30:05.997 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQCqMCpmyOeyAhAAh1SA8jNTrsaHPBfpILCdIw== 2024-04-25T10:30:05.998 INFO:teuthology.orchestra.run.smithi106.stdout:[client.0] 2024-04-25T10:30:05.998 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQCOMCpm6DQuMRAAu7zVTCnQisHdY+dfrxdcjA== 2024-04-25T10:30:05.999 INFO:teuthology.orchestra.run.smithi134.stdout:[mgr.y] 2024-04-25T10:30:05.999 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQCOMCpmhi32ABAAweMHCNXIWkWqkg9fmiT92g== 2024-04-25T10:30:05.999 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.a] 2024-04-25T10:30:05.999 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQCOMCpmwUwHChAAUg0EpkAJCrjAuS4wJMr58Q== 2024-04-25T10:30:06.000 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.c] 2024-04-25T10:30:06.000 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQCOMCpmKeNgERAAW7Rbw2nkBzVVwaT9rSNkpg== 2024-04-25T10:30:06.000 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.e] 2024-04-25T10:30:06.000 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQCOMCpmIFHJGBAAO2cIxzly3biv4KJsKe9Ttg== 2024-04-25T10:30:06.000 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.0] 2024-04-25T10:30:06.000 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQCSMCpmQa1bBBAA/KKm97toQCLJT2ogo2ZliA== 2024-04-25T10:30:06.000 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.1] 2024-04-25T10:30:06.000 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQCVMCpmQEmWAxAAlNBgdgu/Ih/eFLVEA1E78w== 2024-04-25T10:30:06.000 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.2] 2024-04-25T10:30:06.000 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQCYMCpmUTGeAhAAX1bcS//iaaTrB200vdN/bw== 2024-04-25T10:30:06.000 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.3] 2024-04-25T10:30:06.000 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQCbMCpmFREfAhAAbiWZqEqEGKfPlhcH30sWRw== 2024-04-25T10:30:06.000 INFO:teuthology.orchestra.run.smithi134.stdout:[mgr.x] 2024-04-25T10:30:06.000 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQCOMCpm8JFQBhAA/1Z+OBxquOVgQtMM0x14uQ== 2024-04-25T10:30:06.001 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.b] 2024-04-25T10:30:06.001 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQCOMCpmBGviHRAA+pPrYCjotv9ZuoCHs6GVew== 2024-04-25T10:30:06.001 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.d] 2024-04-25T10:30:06.001 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQCOMCpmFU1AJRAAO9cDvsSVcotvFW6RfvbOCg== 2024-04-25T10:30:06.001 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.f] 2024-04-25T10:30:06.001 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQCOMCpmBcedLBAAI1wAXuF6+8FkVMwwicONhw== 2024-04-25T10:30:06.001 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.4] 2024-04-25T10:30:06.001 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQChMCpmjTkGBxAAg4sTg4Kfv/KCwR6Ck8etOA== 2024-04-25T10:30:06.001 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.5] 2024-04-25T10:30:06.001 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQCkMCpmdze2BRAAhHM/mvM7HA5pdGrKjG5FRQ== 2024-04-25T10:30:06.001 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.6] 2024-04-25T10:30:06.001 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQCnMCpmHaN1BBAAsezgDKhMWInFig+7DiTEuA== 2024-04-25T10:30:06.001 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.7] 2024-04-25T10:30:06.001 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQCqMCpmyOeyAhAAh1SA8jNTrsaHPBfpILCdIw== 2024-04-25T10:30:06.002 INFO:teuthology.orchestra.run.smithi134.stdout:[client.0] 2024-04-25T10:30:06.002 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQCOMCpm6DQuMRAAu7zVTCnQisHdY+dfrxdcjA== 2024-04-25T10:30:06.002 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T10:30:06.042 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T10:30:06.104 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:30:06.106 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:30:06.171 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:30:06.174 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:30:06.236 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:30:06.238 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:30:06.300 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:30:06.302 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:30:06.361 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:30:06.363 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:30:06.422 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:30:06.425 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:30:06.485 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:30:06.487 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:30:06.546 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T10:30:06.548 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T10:30:06.606 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:30:06.609 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:30:06.670 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:30:06.673 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:30:06.734 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:30:06.736 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:30:06.798 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:30:06.800 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:30:06.859 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:30:06.861 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:30:06.921 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:30:06.923 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:30:06.987 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:30:06.989 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:30:07.048 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:30:07.050 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:30:07.113 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-25T10:30:07.114 DEBUG:teuthology.orchestra.run.smithi106:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-25T10:30:07.137 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T10:30:07.241 DEBUG:teuthology.orchestra.run.smithi106:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-25T10:30:07.264 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-25T10:30:07.289 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T10:30:07.399 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-25T10:30:07.424 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-25T10:30:07.489 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T10:30:07.594 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-25T10:30:07.617 DEBUG:teuthology.orchestra.run.smithi106:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:30:07.619 DEBUG:teuthology.orchestra.run.smithi134:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:30:07.674 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-25T10:30:07.675 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-25T10:30:07.675 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-25T10:30:07.677 INFO:tasks.ceph.mon.a:Started 2024-04-25T10:30:07.677 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-25T10:30:07.678 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-25T10:30:07.718 INFO:tasks.ceph.mon.b:Started 2024-04-25T10:30:07.718 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-25T10:30:07.718 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-25T10:30:07.720 INFO:tasks.ceph.mon.c:Started 2024-04-25T10:30:07.720 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-25T10:30:07.723 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-25T10:30:07.723 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-25T10:30:07.725 INFO:tasks.ceph.mgr.y:Started 2024-04-25T10:30:07.725 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-25T10:30:07.725 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-25T10:30:07.727 INFO:tasks.ceph.mgr.x:Started 2024-04-25T10:30:07.728 INFO:tasks.ceph:Setting crush tunables to default 2024-04-25T10:30:07.728 DEBUG:teuthology.orchestra.run.smithi106:> sudo ceph --cluster ceph osd crush tunables default 2024-04-25T10:30:07.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:07.836+0000 7fd2d599f640 1 Processor -- start 2024-04-25T10:30:07.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:07.836+0000 7fd2d599f640 1 -- start start 2024-04-25T10:30:07.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:07.837+0000 7fd2d599f640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fd2d01530c0 0x7fd2d0070120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:07.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:07.837+0000 7fd2d599f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd2d01534d0 0x7fd2d0070660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:07.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:07.837+0000 7fd2d599f640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2d015c690 con 0x7fd2d00c51a0 2024-04-25T10:30:07.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:07.837+0000 7fd2d599f640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fd2d015c970 con 0x7fd2d01530c0 2024-04-25T10:30:07.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:07.837+0000 7fd2d599f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fd2d015cc50 con 0x7fd2d01534d0 2024-04-25T10:30:07.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:07.837+0000 7fd2cffff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fd2d01530c0 0x7fd2d0070120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:07.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:07.837+0000 7fd2d499d640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fd2d00c51a0 0x7fd2d00c9d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:51260/0 (socket says 172.21.15.106:51260) 2024-04-25T10:30:07.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:07.837+0000 7fd2cffff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fd2d01530c0 0x7fd2d0070120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:52678/0 (socket says 172.21.15.106:52678) 2024-04-25T10:30:07.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:07.837+0000 7fd2d499d640 1 -- 172.21.15.106:0/4052909328 learned_addr learned my addr 172.21.15.106:0/4052909328 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:30:07.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:07.838+0000 7fd2cffff640 1 -- 172.21.15.106:0/4052909328 >> v2:172.21.15.106:3300/0 conn(0x7fd2d01530c0 msgr2=0x7fd2d0070120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T10:30:07.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:07.838+0000 7fd2cffff640 1 -- 172.21.15.106:0/4052909328 >> v2:172.21.15.106:3300/0 conn(0x7fd2d01530c0 msgr2=0x7fd2d0070120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:30:07.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:07.838+0000 7fd2cffff640 1 --2- 172.21.15.106:0/4052909328 >> v2:172.21.15.106:3300/0 conn(0x7fd2d01530c0 0x7fd2d0070120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:30:07.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:07.838+0000 7fd2d519e640 1 --2- 172.21.15.106:0/4052909328 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd2d01534d0 0x7fd2d0070660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:07.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:07.838+0000 7fd2cffff640 1 --2- 172.21.15.106:0/4052909328 >> v2:172.21.15.106:3300/0 conn(0x7fd2d01530c0 0x7fd2d0070120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:30:07.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:07.838+0000 7fd2d519e640 1 -- 172.21.15.106:0/4052909328 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd2d01534d0 msgr2=0x7fd2d0070660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T10:30:07.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:07.838+0000 7fd2d519e640 1 -- 172.21.15.106:0/4052909328 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd2d01534d0 msgr2=0x7fd2d0070660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:30:07.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:07.838+0000 7fd2d519e640 1 --2- 172.21.15.106:0/4052909328 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd2d01534d0 0x7fd2d0070660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:30:07.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:07.838+0000 7fd2d519e640 1 --2- 172.21.15.106:0/4052909328 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd2d01534d0 0x7fd2d0070660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:30:08.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:07.999+0000 7fd2cf7fe640 1 -- 172.21.15.106:0/4052909328 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 353798543 0 0) 0x7fd2d015c690 con 0x7fd2d00c51a0 2024-04-25T10:30:08.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:07.999+0000 7fd2cf7fe640 1 -- 172.21.15.106:0/4052909328 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd2c0003520 con 0x7fd2d00c51a0 2024-04-25T10:30:08.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.000+0000 7fd2cf7fe640 1 -- 172.21.15.106:0/4052909328 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 26851103 0 0) 0x7fd2c0003520 con 0x7fd2d00c51a0 2024-04-25T10:30:08.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.000+0000 7fd2cf7fe640 1 -- 172.21.15.106:0/4052909328 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd2d01534d0 msgr2=0x7fd2d0070660 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:30:08.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.000+0000 7fd2cf7fe640 1 --2- 172.21.15.106:0/4052909328 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd2d01534d0 0x7fd2d0070660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.000+0000 7fd2cf7fe640 1 -- 172.21.15.106:0/4052909328 >> v2:172.21.15.106:3300/0 conn(0x7fd2d01530c0 msgr2=0x7fd2d0070120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:30:08.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.000+0000 7fd2cf7fe640 1 --2- 172.21.15.106:0/4052909328 >> v2:172.21.15.106:3300/0 conn(0x7fd2d01530c0 0x7fd2d0070120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.000+0000 7fd2cf7fe640 1 -- 172.21.15.106:0/4052909328 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2d015cf30 con 0x7fd2d00c51a0 2024-04-25T10:30:08.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.000+0000 7fd2cf7fe640 1 -- 172.21.15.106:0/4052909328 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3776459350 0 0) 0x7fd2c4001670 con 0x7fd2d00c51a0 2024-04-25T10:30:08.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.000+0000 7fd2cf7fe640 1 -- 172.21.15.106:0/4052909328 >> v1:172.21.15.106:6789/0 conn(0x7fd2d00c51a0 legacy=0x7fd2d00c9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:08.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.000+0000 7fd2cf7fe640 1 --2- 172.21.15.106:0/4052909328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2c00039b0 0x7fd2c0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:08.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.001+0000 7fd2cf7fe640 1 --2- 172.21.15.106:0/4052909328 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd2d01534d0 0x7fd2c0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:08.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.001+0000 7fd2cf7fe640 1 --2- 172.21.15.106:0/4052909328 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd2c0004d40 0x7fd2c0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:08.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.001+0000 7fd2cf7fe640 1 -- 172.21.15.106:0/4052909328 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d015c970 con 0x7fd2c0004d40 2024-04-25T10:30:08.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.001+0000 7fd2cf7fe640 1 -- 172.21.15.106:0/4052909328 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d015cc50 con 0x7fd2c00039b0 2024-04-25T10:30:08.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.001+0000 7fd2cf7fe640 1 -- 172.21.15.106:0/4052909328 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fd2c000def0 con 0x7fd2d01534d0 2024-04-25T10:30:08.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.001+0000 7fd2d519e640 1 --2- 172.21.15.106:0/4052909328 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd2c0004d40 0x7fd2c0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:08.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.001+0000 7fd2d499d640 1 --2- 172.21.15.106:0/4052909328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2c00039b0 0x7fd2c0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:08.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.001+0000 7fd2cffff640 1 --2- 172.21.15.106:0/4052909328 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd2d01534d0 0x7fd2c0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:08.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.001+0000 7fd2d519e640 1 -- 172.21.15.106:0/4052909328 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd2d01534d0 msgr2=0x7fd2c0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:08.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.001+0000 7fd2d519e640 1 --2- 172.21.15.106:0/4052909328 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd2d01534d0 0x7fd2c0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.001+0000 7fd2d519e640 1 -- 172.21.15.106:0/4052909328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2c00039b0 msgr2=0x7fd2c0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:08.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.001+0000 7fd2d519e640 1 --2- 172.21.15.106:0/4052909328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2c00039b0 0x7fd2c0003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.001+0000 7fd2d519e640 1 -- 172.21.15.106:0/4052909328 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2c0003520 con 0x7fd2c0004d40 2024-04-25T10:30:08.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.002+0000 7fd2d519e640 1 --2- 172.21.15.106:0/4052909328 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd2c0004d40 0x7fd2c0005210 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd2d0068f40 tx=0x7fd2c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3d88e1f6d564b56 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:08.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.003+0000 7fd2cf7fe640 1 -- 172.21.15.106:0/4052909328 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c8033070 con 0x7fd2c0004d40 2024-04-25T10:30:08.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.003+0000 7fd2cf7fe640 1 -- 172.21.15.106:0/4052909328 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2c802f400 con 0x7fd2c0004d40 2024-04-25T10:30:08.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.003+0000 7fd2cf7fe640 1 -- 172.21.15.106:0/4052909328 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c8009bc0 con 0x7fd2c0004d40 2024-04-25T10:30:08.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.004+0000 7fd2d599f640 1 -- 172.21.15.106:0/4052909328 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd2c0004d40 msgr2=0x7fd2c0005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:08.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.004+0000 7fd2d599f640 1 --2- 172.21.15.106:0/4052909328 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd2c0004d40 0x7fd2c0005210 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd2d0068f40 tx=0x7fd2c0005830 comp rx=0 tx=0).stop 2024-04-25T10:30:08.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.004+0000 7fd2d519e640 1 -- 172.21.15.106:0/4052909328 reap_dead start 2024-04-25T10:30:08.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.004+0000 7fd2d599f640 1 -- 172.21.15.106:0/4052909328 shutdown_connections 2024-04-25T10:30:08.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.004+0000 7fd2d599f640 1 -- 172.21.15.106:0/4052909328 >> 172.21.15.106:0/4052909328 conn(0x7fd2d006c9e0 msgr2=0x7fd2d006cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:08.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.004+0000 7fd2d599f640 1 -- 172.21.15.106:0/4052909328 shutdown_connections 2024-04-25T10:30:08.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.004+0000 7fd2d599f640 1 -- 172.21.15.106:0/4052909328 wait complete. 2024-04-25T10:30:08.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.006+0000 7fd2d599f640 1 Processor -- start 2024-04-25T10:30:08.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.006+0000 7fd2d599f640 1 -- start start 2024-04-25T10:30:08.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.008+0000 7fd2d599f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd2d01526c0 0x7fd2d015bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:08.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.008+0000 7fd2d599f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2d0156bd0 0x7fd2d0151cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:08.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.008+0000 7fd2d599f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd2d01557c0 0x7fd2d0155bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:08.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.008+0000 7fd2d599f640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d00707e0 con 0x7fd2d01557c0 2024-04-25T10:30:08.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.008+0000 7fd2d599f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d0070660 con 0x7fd2d0156bd0 2024-04-25T10:30:08.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.008+0000 7fd2d599f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fd2d0070960 con 0x7fd2d01526c0 2024-04-25T10:30:08.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.008+0000 7fd2d519e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd2d01557c0 0x7fd2d0155bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:08.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.008+0000 7fd2d499d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd2d01526c0 0x7fd2d015bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:08.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.008+0000 7fd2cffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2d0156bd0 0x7fd2d0151cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:08.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.008+0000 7fd2d519e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd2d01557c0 0x7fd2d0155bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:52718/0 (socket says 172.21.15.106:52718) 2024-04-25T10:30:08.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.008+0000 7fd2d519e640 1 -- 172.21.15.106:0/2837810200 learned_addr learned my addr 172.21.15.106:0/2837810200 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:08.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.008+0000 7fd2d499d640 1 -- 172.21.15.106:0/2837810200 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2d0156bd0 msgr2=0x7fd2d0151cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:08.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.008+0000 7fd2d499d640 1 --2- 172.21.15.106:0/2837810200 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2d0156bd0 0x7fd2d0151cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.008+0000 7fd2d499d640 1 -- 172.21.15.106:0/2837810200 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd2d01557c0 msgr2=0x7fd2d0155bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:08.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.008+0000 7fd2d499d640 1 --2- 172.21.15.106:0/2837810200 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd2d01557c0 0x7fd2d0155bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.008+0000 7fd2d499d640 1 -- 172.21.15.106:0/2837810200 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2d015c3b0 con 0x7fd2d01526c0 2024-04-25T10:30:08.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.008+0000 7fd2d519e640 1 --2- 172.21.15.106:0/2837810200 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd2d01557c0 0x7fd2d0155bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:08.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.008+0000 7fd2d499d640 1 --2- 172.21.15.106:0/2837810200 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd2d01526c0 0x7fd2d015bd20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd2c40017e0 tx=0x7fd2c40074f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:08.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.008+0000 7fd2cd7fa640 1 -- 172.21.15.106:0/2837810200 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c400f7b0 con 0x7fd2d01526c0 2024-04-25T10:30:08.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.008+0000 7fd2cd7fa640 1 -- 172.21.15.106:0/2837810200 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2c4018070 con 0x7fd2d01526c0 2024-04-25T10:30:08.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.008+0000 7fd2d599f640 1 -- 172.21.15.106:0/2837810200 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2d01edf40 con 0x7fd2d01526c0 2024-04-25T10:30:08.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.009+0000 7fd2cd7fa640 1 -- 172.21.15.106:0/2837810200 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c400fb90 con 0x7fd2d01526c0 2024-04-25T10:30:08.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.009+0000 7fd2d599f640 1 -- 172.21.15.106:0/2837810200 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2d01ee500 con 0x7fd2d01526c0 2024-04-25T10:30:08.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.009+0000 7fd2cd7fa640 1 -- 172.21.15.106:0/2837810200 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd2c400fd30 con 0x7fd2d01526c0 2024-04-25T10:30:08.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.009+0000 7fd2cd7fa640 1 -- 172.21.15.106:0/2837810200 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd2c4013700 con 0x7fd2d01526c0 2024-04-25T10:30:08.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.010+0000 7fd2d599f640 1 -- 172.21.15.106:0/2837810200 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2d0156110 con 0x7fd2d01526c0 2024-04-25T10:30:08.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.012+0000 7fd2cd7fa640 1 -- 172.21.15.106:0/2837810200 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fd2c4013950 con 0x7fd2d01526c0 2024-04-25T10:30:08.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.077+0000 7fd2d599f640 1 -- 172.21.15.106:0/2837810200 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fd2d00695a0 con 0x7fd2d01526c0 2024-04-25T10:30:08.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.081+0000 7fd2cd7fa640 1 -- 172.21.15.106:0/2837810200 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7fd2c4013b30 con 0x7fd2d01526c0 2024-04-25T10:30:08.083 INFO:teuthology.orchestra.run.smithi106.stderr:adjusted tunables profile to default 2024-04-25T10:30:08.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.082+0000 7fd2d599f640 1 -- 172.21.15.106:0/2837810200 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd2d01526c0 msgr2=0x7fd2d015bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:08.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.082+0000 7fd2d599f640 1 --2- 172.21.15.106:0/2837810200 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd2d01526c0 0x7fd2d015bd20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd2c40017e0 tx=0x7fd2c40074f0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.082+0000 7fd2d599f640 1 -- 172.21.15.106:0/2837810200 shutdown_connections 2024-04-25T10:30:08.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.082+0000 7fd2d599f640 1 --2- 172.21.15.106:0/2837810200 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd2d01557c0 0x7fd2d0155bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.082+0000 7fd2d599f640 1 --2- 172.21.15.106:0/2837810200 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2d0156bd0 0x7fd2d0151cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.082+0000 7fd2d599f640 1 --2- 172.21.15.106:0/2837810200 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd2d01526c0 0x7fd2d015bd20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.082+0000 7fd2d599f640 1 -- 172.21.15.106:0/2837810200 >> 172.21.15.106:0/2837810200 conn(0x7fd2d006c9e0 msgr2=0x7fd2d00c92d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:08.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.082+0000 7fd2d599f640 1 -- 172.21.15.106:0/2837810200 shutdown_connections 2024-04-25T10:30:08.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:08.082+0000 7fd2d599f640 1 -- 172.21.15.106:0/2837810200 wait complete. 2024-04-25T10:30:08.095 INFO:tasks.ceph:check_enable_crimson: False 2024-04-25T10:30:08.095 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-25T10:30:08.096 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T10:30:08.096 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-25T10:30:08.112 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-25T10:30:08.109+0000 7f94ad81d040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T10:30:08.119 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T10:30:08.119 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-25T10:30:08.130 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-25T10:30:08.128+0000 7fa5d89e3040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T10:30:08.152 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T10:30:08.152 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-25T10:30:08.216 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T10:30:08.216 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-25T10:30:08.268 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-25T10:30:08.266+0000 7f94ad81d040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:30:08.279 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-25T10:30:08.279 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-25T10:30:08.288 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-25T10:30:08.286+0000 7fa5d89e3040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:30:08.303 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-25T10:30:08.303 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-25T10:30:08.324 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-25T10:30:08.321+0000 7f94ad81d040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T10:30:08.342 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-25T10:30:08.340+0000 7fa5d89e3040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T10:30:08.348 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-25T10:30:08.348 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-25T10:30:08.413 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-25T10:30:08.413 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-25T10:30:08.477 DEBUG:teuthology.orchestra.run.smithi134:> sudo ceph --cluster ceph osd new a5962cb1-dbbd-47b1-8d15-9bae114bfdf1 0 2024-04-25T10:30:08.566 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-25T10:30:08.563+0000 7f94ad81d040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T10:30:08.599 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-25T10:30:08.597+0000 7fa5d89e3040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T10:30:08.615 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-25T10:30:08.612+0000 7f94ad81d040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T10:30:08.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.623+0000 7f7823f38640 1 Processor -- start 2024-04-25T10:30:08.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.623+0000 7f7823f38640 1 -- start start 2024-04-25T10:30:08.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.624+0000 7f7823f38640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f781c0c7110 0x7f781c156ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:08.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.624+0000 7f7823f38640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f781c15da90 con 0x7f781c160880 2024-04-25T10:30:08.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.624+0000 7f7823f38640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f781c071360 con 0x7f781c071e20 2024-04-25T10:30:08.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.624+0000 7f7823f38640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f781c071640 con 0x7f781c0c7110 2024-04-25T10:30:08.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.624+0000 7f78214d3640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f781c0c7110 0x7f781c156ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:08.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.625+0000 7f78214d3640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f781c0c7110 0x7f781c156ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:36572/0 (socket says 172.21.15.134:36572) 2024-04-25T10:30:08.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.625+0000 7f78214d3640 1 -- 172.21.15.134:0/295651172 learned_addr learned my addr 172.21.15.134:0/295651172 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-25T10:30:08.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.625+0000 7f78214d3640 1 -- 172.21.15.134:0/295651172 >> v1:172.21.15.134:6789/0 conn(0x7f781c071e20 legacy=0x7f781c0cbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:08.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.625+0000 7f78214d3640 1 -- 172.21.15.134:0/295651172 >> v1:172.21.15.106:6789/0 conn(0x7f781c160880 legacy=0x7f781c160c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:08.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.625+0000 7f78214d3640 1 -- 172.21.15.134:0/295651172 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f781c071920 con 0x7f781c0c7110 2024-04-25T10:30:08.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.625+0000 7f78214d3640 1 --2- 172.21.15.134:0/295651172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f781c0c7110 0x7f781c156ce0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7818009870 tx=0x7f781802a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c7948184af3bf71 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:08.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.626+0000 7f7820cd2640 1 -- 172.21.15.134:0/295651172 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7818037070 con 0x7f781c0c7110 2024-04-25T10:30:08.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.626+0000 7f7820cd2640 1 -- 172.21.15.134:0/295651172 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7818032440 con 0x7f781c0c7110 2024-04-25T10:30:08.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.626+0000 7f7820cd2640 1 -- 172.21.15.134:0/295651172 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7818032810 con 0x7f781c0c7110 2024-04-25T10:30:08.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.626+0000 7f7823f38640 1 -- 172.21.15.134:0/295651172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f781c0c7110 msgr2=0x7f781c156ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:08.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.626+0000 7f7823f38640 1 --2- 172.21.15.134:0/295651172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f781c0c7110 0x7f781c156ce0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7818009870 tx=0x7f781802a270 comp rx=0 tx=0).stop 2024-04-25T10:30:08.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.626+0000 7f7823f38640 1 -- 172.21.15.134:0/295651172 shutdown_connections 2024-04-25T10:30:08.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.626+0000 7f7823f38640 1 --2- 172.21.15.134:0/295651172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f781c0c7110 0x7f781c156ce0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.626+0000 7f7823f38640 1 -- 172.21.15.134:0/295651172 >> 172.21.15.134:0/295651172 conn(0x7f781c06daf0 msgr2=0x7f781c06df00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:08.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.627+0000 7f7823f38640 1 -- 172.21.15.134:0/295651172 shutdown_connections 2024-04-25T10:30:08.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.627+0000 7f7823f38640 1 -- 172.21.15.134:0/295651172 wait complete. 2024-04-25T10:30:08.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.628+0000 7f7823f38640 1 Processor -- start 2024-04-25T10:30:08.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.628+0000 7f7823f38640 1 -- start start 2024-04-25T10:30:08.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.629+0000 7f7823f38640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f781c071e20 0x7f781c0c9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:08.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.629+0000 7f7823f38640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f781c0c7110 0x7f781c1e6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:08.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.629+0000 7f7823f38640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f781c160880 0x7f781c1e59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:08.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.629+0000 7f7823f38640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f781c15d750 con 0x7f781c160880 2024-04-25T10:30:08.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.629+0000 7f7823f38640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f781c0c7520 con 0x7f781c0c7110 2024-04-25T10:30:08.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.629+0000 7f7823f38640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f781c0c9890 con 0x7f781c071e20 2024-04-25T10:30:08.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.630+0000 7f78214d3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f781c0c7110 0x7f781c1e6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:08.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.630+0000 7f78214d3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f781c0c7110 0x7f781c1e6ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:36584/0 (socket says 172.21.15.134:36584) 2024-04-25T10:30:08.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.630+0000 7f7821cd4640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f781c071e20 0x7f781c0c9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:08.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.630+0000 7f78214d3640 1 -- 172.21.15.134:0/1213733162 learned_addr learned my addr 172.21.15.134:0/1213733162 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-25T10:30:08.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.630+0000 7f78224d5640 1 --2- 172.21.15.134:0/1213733162 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f781c160880 0x7f781c1e59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:08.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.630+0000 7f78214d3640 1 -- 172.21.15.134:0/1213733162 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f781c071e20 msgr2=0x7f781c0c9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:08.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.630+0000 7f78214d3640 1 --2- 172.21.15.134:0/1213733162 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f781c071e20 0x7f781c0c9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.630+0000 7f78214d3640 1 -- 172.21.15.134:0/1213733162 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f781c160880 msgr2=0x7f781c1e59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:08.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.630+0000 7f78214d3640 1 --2- 172.21.15.134:0/1213733162 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f781c160880 0x7f781c1e59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.630+0000 7f78214d3640 1 -- 172.21.15.134:0/1213733162 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f781c150d10 con 0x7f781c0c7110 2024-04-25T10:30:08.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.630+0000 7f78214d3640 1 --2- 172.21.15.134:0/1213733162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f781c0c7110 0x7f781c1e6ee0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7818000c00 tx=0x7f78180046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:08.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.630+0000 7f78127fc640 1 -- 172.21.15.134:0/1213733162 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7818037040 con 0x7f781c0c7110 2024-04-25T10:30:08.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.630+0000 7f78127fc640 1 -- 172.21.15.134:0/1213733162 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7818042070 con 0x7f781c0c7110 2024-04-25T10:30:08.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.630+0000 7f7823f38640 1 -- 172.21.15.134:0/1213733162 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f781c1e7810 con 0x7f781c0c7110 2024-04-25T10:30:08.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.630+0000 7f78127fc640 1 -- 172.21.15.134:0/1213733162 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7818030bd0 con 0x7f781c0c7110 2024-04-25T10:30:08.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.630+0000 7f7823f38640 1 -- 172.21.15.134:0/1213733162 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f781c1f30e0 con 0x7f781c0c7110 2024-04-25T10:30:08.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.631+0000 7f78127fc640 1 -- 172.21.15.134:0/1213733162 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7818047460 con 0x7f781c0c7110 2024-04-25T10:30:08.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.631+0000 7f78127fc640 1 -- 172.21.15.134:0/1213733162 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f781803d440 con 0x7f781c0c7110 2024-04-25T10:30:08.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.631+0000 7f7823f38640 1 -- 172.21.15.134:0/1213733162 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f781c150f20 con 0x7f781c0c7110 2024-04-25T10:30:08.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.634+0000 7f78127fc640 1 -- 172.21.15.134:0/1213733162 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f781803d690 con 0x7f781c0c7110 2024-04-25T10:30:08.644 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-25T10:30:08.642+0000 7fa5d89e3040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T10:30:08.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.705+0000 7f7823f38640 1 -- 172.21.15.134:0/1213733162 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a5962cb1-dbbd-47b1-8d15-9bae114bfdf1", "id": 0} v 0) -- 0x7f781c06a3d0 con 0x7f781c0c7110 2024-04-25T10:30:08.712 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.709+0000 7f78127fc640 1 -- 172.21.15.134:0/1213733162 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a5962cb1-dbbd-47b1-8d15-9bae114bfdf1", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f78180478e0 con 0x7f781c0c7110 2024-04-25T10:30:08.712 INFO:teuthology.orchestra.run.smithi134.stdout:0 2024-04-25T10:30:08.713 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.710+0000 7f7823f38640 1 -- 172.21.15.134:0/1213733162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f781c0c7110 msgr2=0x7f781c1e6ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:08.713 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.710+0000 7f7823f38640 1 --2- 172.21.15.134:0/1213733162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f781c0c7110 0x7f781c1e6ee0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7818000c00 tx=0x7f78180046c0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.713 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.710+0000 7f7823f38640 1 -- 172.21.15.134:0/1213733162 shutdown_connections 2024-04-25T10:30:08.713 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.710+0000 7f7823f38640 1 --2- 172.21.15.134:0/1213733162 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f781c160880 0x7f781c1e59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.713 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.710+0000 7f7823f38640 1 --2- 172.21.15.134:0/1213733162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f781c0c7110 0x7f781c1e6ee0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.713 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.710+0000 7f7823f38640 1 --2- 172.21.15.134:0/1213733162 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f781c071e20 0x7f781c0c9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.713 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.710+0000 7f7823f38640 1 -- 172.21.15.134:0/1213733162 >> 172.21.15.134:0/1213733162 conn(0x7f781c06daf0 msgr2=0x7f781c1496c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:08.714 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.710+0000 7f7823f38640 1 -- 172.21.15.134:0/1213733162 shutdown_connections 2024-04-25T10:30:08.714 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.710+0000 7f7823f38640 1 -- 172.21.15.134:0/1213733162 wait complete. 2024-04-25T10:30:08.724 DEBUG:teuthology.orchestra.run.smithi134:> sudo ceph --cluster ceph osd new 94436e13-4769-4752-8397-b5e396a6f7ad 1 2024-04-25T10:30:08.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.828+0000 7ff43ca05640 1 Processor -- start 2024-04-25T10:30:08.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.828+0000 7ff43ca05640 1 -- start start 2024-04-25T10:30:08.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.829+0000 7ff43ca05640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff4380ca790 0x7ff4380c6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:08.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.829+0000 7ff43ca05640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff438153b70 0x7ff4380cace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:08.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.829+0000 7ff43ca05640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff4380c9cc0 0x7ff4380ca0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:08.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.829+0000 7ff43ca05640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7ff4380c7210 con 0x7ff438153b70 2024-04-25T10:30:08.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.829+0000 7ff43ca05640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7ff4380722d0 con 0x7ff4380ca790 2024-04-25T10:30:08.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.829+0000 7ff43ca05640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff4380c7430 con 0x7ff4380c9cc0 2024-04-25T10:30:08.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.829+0000 7ff43659c640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff4380ca790 0x7ff4380c6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:08.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.829+0000 7ff436d9d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff4380c9cc0 0x7ff4380ca0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:08.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.829+0000 7ff436d9d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff4380c9cc0 0x7ff4380ca0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:33172/0 (socket says 172.21.15.134:33172) 2024-04-25T10:30:08.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.829+0000 7ff43659c640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff4380ca790 0x7ff4380c6cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:36598/0 (socket says 172.21.15.134:36598) 2024-04-25T10:30:08.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.829+0000 7ff436d9d640 1 -- 172.21.15.134:0/2760947591 learned_addr learned my addr 172.21.15.134:0/2760947591 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-25T10:30:08.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.829+0000 7ff435d9b640 1 --2- 172.21.15.134:0/2760947591 >> v2:172.21.15.106:3300/0 conn(0x7ff438153b70 0x7ff4380cace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:08.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.830+0000 7ff436d9d640 1 -- 172.21.15.134:0/2760947591 >> v2:172.21.15.134:3300/0 conn(0x7ff4380ca790 msgr2=0x7ff4380c6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:08.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.830+0000 7ff436d9d640 1 --2- 172.21.15.134:0/2760947591 >> v2:172.21.15.134:3300/0 conn(0x7ff4380ca790 0x7ff4380c6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.830+0000 7ff436d9d640 1 -- 172.21.15.134:0/2760947591 >> v2:172.21.15.106:3300/0 conn(0x7ff438153b70 msgr2=0x7ff4380cace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:08.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.830+0000 7ff436d9d640 1 --2- 172.21.15.134:0/2760947591 >> v2:172.21.15.106:3300/0 conn(0x7ff438153b70 0x7ff4380cace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.830+0000 7ff436d9d640 1 -- 172.21.15.134:0/2760947591 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4380c6430 con 0x7ff4380c9cc0 2024-04-25T10:30:08.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.830+0000 7ff43659c640 1 --2- 172.21.15.134:0/2760947591 >> v2:172.21.15.134:3300/0 conn(0x7ff4380ca790 0x7ff4380c6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:08.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.830+0000 7ff436d9d640 1 --2- 172.21.15.134:0/2760947591 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff4380c9cc0 0x7ff4380ca0d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff438071370 tx=0x7ff43002a730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbd9dca8e8e7a571 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:08.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.830+0000 7ff43559a640 1 -- 172.21.15.134:0/2760947591 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff430036070 con 0x7ff4380c9cc0 2024-04-25T10:30:08.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.830+0000 7ff43559a640 1 -- 172.21.15.134:0/2760947591 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff430031600 con 0x7ff4380c9cc0 2024-04-25T10:30:08.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.831+0000 7ff43559a640 1 -- 172.21.15.134:0/2760947591 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4300319d0 con 0x7ff4380c9cc0 2024-04-25T10:30:08.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.831+0000 7ff43ca05640 1 -- 172.21.15.134:0/2760947591 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff4380c9cc0 msgr2=0x7ff4380ca0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:08.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.831+0000 7ff43ca05640 1 --2- 172.21.15.134:0/2760947591 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff4380c9cc0 0x7ff4380ca0d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff438071370 tx=0x7ff43002a730 comp rx=0 tx=0).stop 2024-04-25T10:30:08.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.831+0000 7ff43ca05640 1 -- 172.21.15.134:0/2760947591 shutdown_connections 2024-04-25T10:30:08.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.831+0000 7ff43ca05640 1 --2- 172.21.15.134:0/2760947591 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff4380c9cc0 0x7ff4380ca0d0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.831+0000 7ff43ca05640 1 --2- 172.21.15.134:0/2760947591 >> v2:172.21.15.106:3300/0 conn(0x7ff438153b70 0x7ff4380cace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.831+0000 7ff43ca05640 1 --2- 172.21.15.134:0/2760947591 >> v2:172.21.15.134:3300/0 conn(0x7ff4380ca790 0x7ff4380c6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.831+0000 7ff43ca05640 1 -- 172.21.15.134:0/2760947591 >> 172.21.15.134:0/2760947591 conn(0x7ff43806daf0 msgr2=0x7ff43806df00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:08.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.831+0000 7ff43ca05640 1 -- 172.21.15.134:0/2760947591 shutdown_connections 2024-04-25T10:30:08.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.831+0000 7ff43ca05640 1 -- 172.21.15.134:0/2760947591 wait complete. 2024-04-25T10:30:08.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.833+0000 7ff43ca05640 1 Processor -- start 2024-04-25T10:30:08.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.833+0000 7ff43ca05640 1 -- start start 2024-04-25T10:30:08.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.834+0000 7ff43ca05640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4380c9cc0 0x7ff43814d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:08.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.834+0000 7ff43ca05640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff4380ca790 0x7ff4380ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:08.840 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.834+0000 7ff43ca05640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff438153b70 0x7ff438156980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:08.840 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.834+0000 7ff43ca05640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff4380cbf00 con 0x7ff438153b70 2024-04-25T10:30:08.840 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.834+0000 7ff43ca05640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff4380c69f0 con 0x7ff4380c9cc0 2024-04-25T10:30:08.840 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.834+0000 7ff43ca05640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff4380cb220 con 0x7ff4380ca790 2024-04-25T10:30:08.840 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.835+0000 7ff43659c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4380c9cc0 0x7ff43814d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:08.840 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.835+0000 7ff435d9b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff4380ca790 0x7ff4380ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:08.840 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.835+0000 7ff43659c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4380c9cc0 0x7ff43814d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:36600/0 (socket says 172.21.15.134:36600) 2024-04-25T10:30:08.840 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.835+0000 7ff43659c640 1 -- 172.21.15.134:0/1355747679 learned_addr learned my addr 172.21.15.134:0/1355747679 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-25T10:30:08.840 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.835+0000 7ff43659c640 1 -- 172.21.15.134:0/1355747679 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff4380ca790 msgr2=0x7ff4380ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:08.840 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.835+0000 7ff43659c640 1 --2- 172.21.15.134:0/1355747679 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff4380ca790 0x7ff4380ce0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.840 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.835+0000 7ff43659c640 1 -- 172.21.15.134:0/1355747679 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff438153b70 msgr2=0x7ff438156980 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:30:08.840 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.835+0000 7ff43659c640 1 --2- 172.21.15.134:0/1355747679 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff438153b70 0x7ff438156980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.840 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.835+0000 7ff43659c640 1 -- 172.21.15.134:0/1355747679 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4381e17a0 con 0x7ff4380c9cc0 2024-04-25T10:30:08.840 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.835+0000 7ff435d9b640 1 --2- 172.21.15.134:0/1355747679 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff4380ca790 0x7ff4380ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:08.841 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.835+0000 7ff43659c640 1 --2- 172.21.15.134:0/1355747679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4380c9cc0 0x7ff43814d3e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff428010df0 tx=0x7ff42800e730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:08.841 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.835+0000 7ff426ffd640 1 -- 172.21.15.134:0/1355747679 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42801e070 con 0x7ff4380c9cc0 2024-04-25T10:30:08.841 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.835+0000 7ff426ffd640 1 -- 172.21.15.134:0/1355747679 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff428011650 con 0x7ff4380c9cc0 2024-04-25T10:30:08.841 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.835+0000 7ff43ca05640 1 -- 172.21.15.134:0/1355747679 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4381e1a40 con 0x7ff4380c9cc0 2024-04-25T10:30:08.841 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.835+0000 7ff426ffd640 1 -- 172.21.15.134:0/1355747679 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff428011a20 con 0x7ff4380c9cc0 2024-04-25T10:30:08.841 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.835+0000 7ff43ca05640 1 -- 172.21.15.134:0/1355747679 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4381e1ce0 con 0x7ff4380c9cc0 2024-04-25T10:30:08.841 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.835+0000 7ff426ffd640 1 -- 172.21.15.134:0/1355747679 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff428021070 con 0x7ff4380c9cc0 2024-04-25T10:30:08.841 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.836+0000 7ff426ffd640 1 -- 172.21.15.134:0/1355747679 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7ff428006590 con 0x7ff4380c9cc0 2024-04-25T10:30:08.841 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.836+0000 7ff43ca05640 1 -- 172.21.15.134:0/1355747679 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff43806a3d0 con 0x7ff4380c9cc0 2024-04-25T10:30:08.841 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.838+0000 7ff426ffd640 1 -- 172.21.15.134:0/1355747679 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7ff428028070 con 0x7ff4380c9cc0 2024-04-25T10:30:08.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.908+0000 7ff43ca05640 1 -- 172.21.15.134:0/1355747679 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd new", "uuid": "94436e13-4769-4752-8397-b5e396a6f7ad", "id": 1} v 0) -- 0x7ff43814dc80 con 0x7ff4380c9cc0 2024-04-25T10:30:08.914 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.912+0000 7ff426ffd640 1 -- 172.21.15.134:0/1355747679 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "94436e13-4769-4752-8397-b5e396a6f7ad", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7ff428006790 con 0x7ff4380c9cc0 2024-04-25T10:30:08.914 INFO:teuthology.orchestra.run.smithi134.stdout:1 2024-04-25T10:30:08.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.913+0000 7ff43ca05640 1 -- 172.21.15.134:0/1355747679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4380c9cc0 msgr2=0x7ff43814d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:08.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.913+0000 7ff43ca05640 1 --2- 172.21.15.134:0/1355747679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4380c9cc0 0x7ff43814d3e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff428010df0 tx=0x7ff42800e730 comp rx=0 tx=0).stop 2024-04-25T10:30:08.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.913+0000 7ff43ca05640 1 -- 172.21.15.134:0/1355747679 shutdown_connections 2024-04-25T10:30:08.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.913+0000 7ff43ca05640 1 --2- 172.21.15.134:0/1355747679 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff438153b70 0x7ff438156980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.913+0000 7ff43ca05640 1 --2- 172.21.15.134:0/1355747679 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff4380ca790 0x7ff4380ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.913+0000 7ff43ca05640 1 --2- 172.21.15.134:0/1355747679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4380c9cc0 0x7ff43814d3e0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.913+0000 7ff43ca05640 1 -- 172.21.15.134:0/1355747679 >> 172.21.15.134:0/1355747679 conn(0x7ff43806daf0 msgr2=0x7ff438152760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:08.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.913+0000 7ff43ca05640 1 -- 172.21.15.134:0/1355747679 shutdown_connections 2024-04-25T10:30:08.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:08.913+0000 7ff43ca05640 1 -- 172.21.15.134:0/1355747679 wait complete. 2024-04-25T10:30:08.926 DEBUG:teuthology.orchestra.run.smithi134:> sudo ceph --cluster ceph osd new 429aae0b-70c6-4e7b-9043-6a11334c7efc 2 2024-04-25T10:30:09.035 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.032+0000 7fc26e1c5640 1 Processor -- start 2024-04-25T10:30:09.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.033+0000 7fc26e1c5640 1 -- start start 2024-04-25T10:30:09.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.034+0000 7fc26e1c5640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fc26806fbf0 0x7fc268156fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:09.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.034+0000 7fc26e1c5640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2680c4ae0 0x7fc2680c4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:09.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.034+0000 7fc26e1c5640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2680cab40 con 0x7fc2680b45a0 2024-04-25T10:30:09.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.034+0000 7fc26e1c5640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fc26806ee20 con 0x7fc26806fbf0 2024-04-25T10:30:09.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.034+0000 7fc26e1c5640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc2681602a0 con 0x7fc2680c4ae0 2024-04-25T10:30:09.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.034+0000 7fc267fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2680c4ae0 0x7fc2680c4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:09.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.034+0000 7fc2677fe640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fc26806fbf0 0x7fc268156fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:09.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.034+0000 7fc2677fe640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fc26806fbf0 0x7fc268156fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:36616/0 (socket says 172.21.15.134:36616) 2024-04-25T10:30:09.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.034+0000 7fc267fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2680c4ae0 0x7fc2680c4ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:33182/0 (socket says 172.21.15.134:33182) 2024-04-25T10:30:09.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.034+0000 7fc2677fe640 1 -- 172.21.15.134:0/769997927 learned_addr learned my addr 172.21.15.134:0/769997927 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-25T10:30:09.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.034+0000 7fc266ffd640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fc2680b45a0 0x7fc268160730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:41206/0 (socket says 172.21.15.134:41206) 2024-04-25T10:30:09.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.034+0000 7fc2677fe640 1 -- 172.21.15.134:0/769997927 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2680c4ae0 msgr2=0x7fc2680c4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:09.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.034+0000 7fc2677fe640 1 --2- 172.21.15.134:0/769997927 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2680c4ae0 0x7fc2680c4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.034+0000 7fc2677fe640 1 -- 172.21.15.134:0/769997927 >> v1:172.21.15.134:6789/0 conn(0x7fc2680b45a0 legacy=0x7fc268160730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:09.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.034+0000 7fc2677fe640 1 -- 172.21.15.134:0/769997927 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2680cae20 con 0x7fc26806fbf0 2024-04-25T10:30:09.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.034+0000 7fc267fff640 1 --2- 172.21.15.134:0/769997927 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2680c4ae0 0x7fc2680c4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:09.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.035+0000 7fc2677fe640 1 --2- 172.21.15.134:0/769997927 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc26806fbf0 0x7fc268156fb0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc258009870 tx=0x7fc25802a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb323cdab1ad2a0c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:09.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.035+0000 7fc2667fc640 1 -- 172.21.15.134:0/769997927 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc258037070 con 0x7fc26806fbf0 2024-04-25T10:30:09.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.035+0000 7fc2667fc640 1 -- 172.21.15.134:0/769997927 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc258032440 con 0x7fc26806fbf0 2024-04-25T10:30:09.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.035+0000 7fc2667fc640 1 -- 172.21.15.134:0/769997927 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc258032810 con 0x7fc26806fbf0 2024-04-25T10:30:09.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.036+0000 7fc26e1c5640 1 -- 172.21.15.134:0/769997927 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc26806fbf0 msgr2=0x7fc268156fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:09.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.036+0000 7fc26e1c5640 1 --2- 172.21.15.134:0/769997927 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc26806fbf0 0x7fc268156fb0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc258009870 tx=0x7fc25802a270 comp rx=0 tx=0).stop 2024-04-25T10:30:09.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.036+0000 7fc26e1c5640 1 -- 172.21.15.134:0/769997927 shutdown_connections 2024-04-25T10:30:09.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.036+0000 7fc26e1c5640 1 --2- 172.21.15.134:0/769997927 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2680c4ae0 0x7fc2680c4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.036+0000 7fc26e1c5640 1 --2- 172.21.15.134:0/769997927 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc26806fbf0 0x7fc268156fb0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.036+0000 7fc26e1c5640 1 -- 172.21.15.134:0/769997927 >> 172.21.15.134:0/769997927 conn(0x7fc26806c5c0 msgr2=0x7fc26806c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:09.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.036+0000 7fc26e1c5640 1 -- 172.21.15.134:0/769997927 shutdown_connections 2024-04-25T10:30:09.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.036+0000 7fc26e1c5640 1 -- 172.21.15.134:0/769997927 wait complete. 2024-04-25T10:30:09.041 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.038+0000 7fc26e1c5640 1 Processor -- start 2024-04-25T10:30:09.041 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.038+0000 7fc26e1c5640 1 -- start start 2024-04-25T10:30:09.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.039+0000 7fc26e1c5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc26806fbf0 0x7fc2680c7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:09.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.039+0000 7fc26e1c5640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2680b45a0 0x7fc2681f3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:09.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.039+0000 7fc26e1c5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc2680c4ae0 0x7fc268154070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:09.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.039+0000 7fc26e1c5640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc26806f780 con 0x7fc2680c4ae0 2024-04-25T10:30:09.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.039+0000 7fc26e1c5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc26806f350 con 0x7fc26806fbf0 2024-04-25T10:30:09.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.039+0000 7fc26e1c5640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc268160e40 con 0x7fc2680b45a0 2024-04-25T10:30:09.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.039+0000 7fc2677fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc26806fbf0 0x7fc2680c7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:09.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.039+0000 7fc266ffd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2680b45a0 0x7fc2681f3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:09.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.039+0000 7fc2677fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc26806fbf0 0x7fc2680c7910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:36620/0 (socket says 172.21.15.134:36620) 2024-04-25T10:30:09.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.039+0000 7fc266ffd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2680b45a0 0x7fc2681f3580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:33190/0 (socket says 172.21.15.134:33190) 2024-04-25T10:30:09.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.039+0000 7fc266ffd640 1 -- 172.21.15.134:0/2452143219 learned_addr learned my addr 172.21.15.134:0/2452143219 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-25T10:30:09.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.039+0000 7fc267fff640 1 --2- 172.21.15.134:0/2452143219 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc2680c4ae0 0x7fc268154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:09.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.039+0000 7fc266ffd640 1 -- 172.21.15.134:0/2452143219 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc26806fbf0 msgr2=0x7fc2680c7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:09.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.039+0000 7fc266ffd640 1 --2- 172.21.15.134:0/2452143219 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc26806fbf0 0x7fc2680c7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.039+0000 7fc266ffd640 1 -- 172.21.15.134:0/2452143219 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc2680c4ae0 msgr2=0x7fc268154070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:09.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.039+0000 7fc266ffd640 1 --2- 172.21.15.134:0/2452143219 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc2680c4ae0 0x7fc268154070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.039+0000 7fc266ffd640 1 -- 172.21.15.134:0/2452143219 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2681f3ac0 con 0x7fc2680b45a0 2024-04-25T10:30:09.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.039+0000 7fc2677fe640 1 --2- 172.21.15.134:0/2452143219 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc26806fbf0 0x7fc2680c7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:09.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.040+0000 7fc266ffd640 1 --2- 172.21.15.134:0/2452143219 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2680b45a0 0x7fc2681f3580 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc25c00c440 tx=0x7fc25c00c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:09.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.040+0000 7fc24ffff640 1 -- 172.21.15.134:0/2452143219 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc25c0074e0 con 0x7fc2680b45a0 2024-04-25T10:30:09.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.040+0000 7fc24ffff640 1 -- 172.21.15.134:0/2452143219 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc25c007680 con 0x7fc2680b45a0 2024-04-25T10:30:09.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.040+0000 7fc26e1c5640 1 -- 172.21.15.134:0/2452143219 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc268157db0 con 0x7fc2680b45a0 2024-04-25T10:30:09.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.040+0000 7fc24ffff640 1 -- 172.21.15.134:0/2452143219 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc25c004a20 con 0x7fc2680b45a0 2024-04-25T10:30:09.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.040+0000 7fc26e1c5640 1 -- 172.21.15.134:0/2452143219 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc268158370 con 0x7fc2680b45a0 2024-04-25T10:30:09.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.040+0000 7fc24ffff640 1 -- 172.21.15.134:0/2452143219 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc25c01a020 con 0x7fc2680b45a0 2024-04-25T10:30:09.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.040+0000 7fc24ffff640 1 -- 172.21.15.134:0/2452143219 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7fc25c012810 con 0x7fc2680b45a0 2024-04-25T10:30:09.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.041+0000 7fc26e1c5640 1 -- 172.21.15.134:0/2452143219 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc26806cf00 con 0x7fc2680b45a0 2024-04-25T10:30:09.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.043+0000 7fc24ffff640 1 -- 172.21.15.134:0/2452143219 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fc26806cf00 con 0x7fc2680b45a0 2024-04-25T10:30:09.118 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.114+0000 7fc26e1c5640 1 -- 172.21.15.134:0/2452143219 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "osd new", "uuid": "429aae0b-70c6-4e7b-9043-6a11334c7efc", "id": 2} v 0) -- 0x7fc26806a3d0 con 0x7fc2680b45a0 2024-04-25T10:30:09.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.121+0000 7fc24ffff640 1 -- 172.21.15.134:0/2452143219 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "429aae0b-70c6-4e7b-9043-6a11334c7efc", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7fc25c021070 con 0x7fc2680b45a0 2024-04-25T10:30:09.123 INFO:teuthology.orchestra.run.smithi134.stdout:2 2024-04-25T10:30:09.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.122+0000 7fc26e1c5640 1 -- 172.21.15.134:0/2452143219 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2680b45a0 msgr2=0x7fc2681f3580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:09.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.122+0000 7fc26e1c5640 1 --2- 172.21.15.134:0/2452143219 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2680b45a0 0x7fc2681f3580 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc25c00c440 tx=0x7fc25c00c900 comp rx=0 tx=0).stop 2024-04-25T10:30:09.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.122+0000 7fc26e1c5640 1 -- 172.21.15.134:0/2452143219 shutdown_connections 2024-04-25T10:30:09.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.122+0000 7fc26e1c5640 1 --2- 172.21.15.134:0/2452143219 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc2680c4ae0 0x7fc268154070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.122+0000 7fc26e1c5640 1 --2- 172.21.15.134:0/2452143219 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2680b45a0 0x7fc2681f3580 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.122+0000 7fc26e1c5640 1 --2- 172.21.15.134:0/2452143219 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc26806fbf0 0x7fc2680c7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.122+0000 7fc26e1c5640 1 -- 172.21.15.134:0/2452143219 >> 172.21.15.134:0/2452143219 conn(0x7fc26806c5c0 msgr2=0x7fc268073aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:09.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.122+0000 7fc26e1c5640 1 -- 172.21.15.134:0/2452143219 shutdown_connections 2024-04-25T10:30:09.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.122+0000 7fc26e1c5640 1 -- 172.21.15.134:0/2452143219 wait complete. 2024-04-25T10:30:09.136 DEBUG:teuthology.orchestra.run.smithi134:> sudo ceph --cluster ceph osd new 9593a4d1-772f-4667-ab0d-2982f841e342 3 2024-04-25T10:30:09.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.242+0000 7fad04d84640 1 Processor -- start 2024-04-25T10:30:09.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.243+0000 7fad04d84640 1 -- start start 2024-04-25T10:30:09.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.244+0000 7fad04d84640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad000c6ba0 0x7fad000c6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:09.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.244+0000 7fad04d84640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fad001599f0 0x7fad00159e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:09.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.244+0000 7fad04d84640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad000c6320 con 0x7fad000c9820 2024-04-25T10:30:09.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.244+0000 7fad04d84640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fad000cfb40 con 0x7fad001599f0 2024-04-25T10:30:09.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.244+0000 7fad04d84640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fad000c9320 con 0x7fad000c6ba0 2024-04-25T10:30:09.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.244+0000 7facfed9d640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fad001599f0 0x7fad00159e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:09.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.244+0000 7facf7fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad000c6ba0 0x7fad000c6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:09.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.244+0000 7facfed9d640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fad001599f0 0x7fad00159e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:36628/0 (socket says 172.21.15.134:36628) 2024-04-25T10:30:09.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.244+0000 7facfed9d640 1 -- 172.21.15.134:0/3409890399 learned_addr learned my addr 172.21.15.134:0/3409890399 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-25T10:30:09.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.244+0000 7facf7fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad000c6ba0 0x7fad000c6fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:33204/0 (socket says 172.21.15.134:33204) 2024-04-25T10:30:09.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.244+0000 7facfed9d640 1 -- 172.21.15.134:0/3409890399 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad000c6ba0 msgr2=0x7fad000c6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:09.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.244+0000 7facfed9d640 1 --2- 172.21.15.134:0/3409890399 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad000c6ba0 0x7fad000c6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.244+0000 7facfed9d640 1 -- 172.21.15.134:0/3409890399 >> v1:172.21.15.134:6789/0 conn(0x7fad000c9820 legacy=0x7fad000c8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:09.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.244+0000 7facfed9d640 1 -- 172.21.15.134:0/3409890399 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad000c65e0 con 0x7fad001599f0 2024-04-25T10:30:09.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.244+0000 7facf7fff640 1 --2- 172.21.15.134:0/3409890399 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad000c6ba0 0x7fad000c6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:09.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.245+0000 7facfed9d640 1 --2- 172.21.15.134:0/3409890399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad001599f0 0x7fad00159e00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7facf800d150 tx=0x7facf802c9b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9b3e9459e62b9da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:09.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.245+0000 7facfdd9b640 1 -- 172.21.15.134:0/3409890399 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf8035020 con 0x7fad001599f0 2024-04-25T10:30:09.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.245+0000 7facfdd9b640 1 -- 172.21.15.134:0/3409890399 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facf800b2e0 con 0x7fad001599f0 2024-04-25T10:30:09.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.245+0000 7facfdd9b640 1 -- 172.21.15.134:0/3409890399 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf8009820 con 0x7fad001599f0 2024-04-25T10:30:09.250 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.246+0000 7fad04d84640 1 -- 172.21.15.134:0/3409890399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad001599f0 msgr2=0x7fad00159e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:09.250 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.246+0000 7fad04d84640 1 --2- 172.21.15.134:0/3409890399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad001599f0 0x7fad00159e00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7facf800d150 tx=0x7facf802c9b0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.250 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.246+0000 7fad04d84640 1 -- 172.21.15.134:0/3409890399 shutdown_connections 2024-04-25T10:30:09.250 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.246+0000 7fad04d84640 1 --2- 172.21.15.134:0/3409890399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad001599f0 0x7fad00159e00 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.251 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.246+0000 7fad04d84640 1 --2- 172.21.15.134:0/3409890399 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad000c6ba0 0x7fad000c6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.251 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.246+0000 7fad04d84640 1 -- 172.21.15.134:0/3409890399 >> 172.21.15.134:0/3409890399 conn(0x7fad0006d8b0 msgr2=0x7fad0006dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:09.251 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.246+0000 7fad04d84640 1 -- 172.21.15.134:0/3409890399 shutdown_connections 2024-04-25T10:30:09.251 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.246+0000 7fad04d84640 1 -- 172.21.15.134:0/3409890399 wait complete. 2024-04-25T10:30:09.251 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.248+0000 7fad04d84640 1 Processor -- start 2024-04-25T10:30:09.251 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.248+0000 7fad04d84640 1 -- start start 2024-04-25T10:30:09.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.249+0000 7fad04d84640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad000c6ba0 0x7fad000cde50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:09.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.249+0000 7fad04d84640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad001599f0 0x7fad000c4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:09.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.249+0000 7fad04d84640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad0015b9c0 0x7fad000c51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:09.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.249+0000 7fad04d84640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fad000cf620 con 0x7fad000c6ba0 2024-04-25T10:30:09.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.249+0000 7fad04d84640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fad000cf340 con 0x7fad0015b9c0 2024-04-25T10:30:09.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.249+0000 7fad04d84640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fad000746a0 con 0x7fad001599f0 2024-04-25T10:30:09.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.249+0000 7facf7fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad001599f0 0x7fad000c4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:09.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.249+0000 7facfed9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad0015b9c0 0x7fad000c51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:09.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.249+0000 7facfed9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad0015b9c0 0x7fad000c51d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:36632/0 (socket says 172.21.15.134:36632) 2024-04-25T10:30:09.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.249+0000 7facf7fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad001599f0 0x7fad000c4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:33220/0 (socket says 172.21.15.134:33220) 2024-04-25T10:30:09.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.249+0000 7facfed9d640 1 -- 172.21.15.134:0/589456059 learned_addr learned my addr 172.21.15.134:0/589456059 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-25T10:30:09.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.249+0000 7facfe59c640 1 --2- 172.21.15.134:0/589456059 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad000c6ba0 0x7fad000cde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:09.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.249+0000 7facfed9d640 1 -- 172.21.15.134:0/589456059 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad001599f0 msgr2=0x7fad000c4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:09.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.249+0000 7facfed9d640 1 --2- 172.21.15.134:0/589456059 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad001599f0 0x7fad000c4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.250+0000 7facfed9d640 1 -- 172.21.15.134:0/589456059 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad000c6ba0 msgr2=0x7fad000cde50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:09.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.250+0000 7facfed9d640 1 --2- 172.21.15.134:0/589456059 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad000c6ba0 0x7fad000cde50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.250+0000 7facfed9d640 1 -- 172.21.15.134:0/589456059 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad0015c820 con 0x7fad0015b9c0 2024-04-25T10:30:09.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.250+0000 7facfe59c640 1 --2- 172.21.15.134:0/589456059 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad000c6ba0 0x7fad000cde50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:09.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.250+0000 7facf7fff640 1 --2- 172.21.15.134:0/589456059 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad001599f0 0x7fad000c4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:09.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.250+0000 7facfed9d640 1 --2- 172.21.15.134:0/589456059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad0015b9c0 0x7fad000c51d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7facf800d240 tx=0x7facf8004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:09.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.250+0000 7facf6ffd640 1 -- 172.21.15.134:0/589456059 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf8035040 con 0x7fad0015b9c0 2024-04-25T10:30:09.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.250+0000 7facf6ffd640 1 -- 172.21.15.134:0/589456059 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facf8009e30 con 0x7fad0015b9c0 2024-04-25T10:30:09.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.250+0000 7fad04d84640 1 -- 172.21.15.134:0/589456059 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad000c5710 con 0x7fad0015b9c0 2024-04-25T10:30:09.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.250+0000 7facf6ffd640 1 -- 172.21.15.134:0/589456059 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf800b720 con 0x7fad0015b9c0 2024-04-25T10:30:09.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.250+0000 7fad04d84640 1 -- 172.21.15.134:0/589456059 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad0015cac0 con 0x7fad0015b9c0 2024-04-25T10:30:09.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.250+0000 7facf6ffd640 1 -- 172.21.15.134:0/589456059 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7facf8046020 con 0x7fad0015b9c0 2024-04-25T10:30:09.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.250+0000 7facf6ffd640 1 -- 172.21.15.134:0/589456059 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7facf803acd0 con 0x7fad0015b9c0 2024-04-25T10:30:09.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.251+0000 7fad04d84640 1 -- 172.21.15.134:0/589456059 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad0006a3d0 con 0x7fad0015b9c0 2024-04-25T10:30:09.256 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.253+0000 7facf6ffd640 1 -- 172.21.15.134:0/589456059 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7facf804d020 con 0x7fad0015b9c0 2024-04-25T10:30:09.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.326+0000 7fad04d84640 1 -- 172.21.15.134:0/589456059 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd new", "uuid": "9593a4d1-772f-4667-ab0d-2982f841e342", "id": 3} v 0) -- 0x7fad0015deb0 con 0x7fad0015b9c0 2024-04-25T10:30:09.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.331+0000 7facf6ffd640 1 -- 172.21.15.134:0/589456059 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9593a4d1-772f-4667-ab0d-2982f841e342", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7facf800b2e0 con 0x7fad0015b9c0 2024-04-25T10:30:09.334 INFO:teuthology.orchestra.run.smithi134.stdout:3 2024-04-25T10:30:09.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.332+0000 7fad04d84640 1 -- 172.21.15.134:0/589456059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad0015b9c0 msgr2=0x7fad000c51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:09.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.332+0000 7fad04d84640 1 --2- 172.21.15.134:0/589456059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad0015b9c0 0x7fad000c51d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7facf800d240 tx=0x7facf8004a40 comp rx=0 tx=0).stop 2024-04-25T10:30:09.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.332+0000 7fad04d84640 1 -- 172.21.15.134:0/589456059 shutdown_connections 2024-04-25T10:30:09.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.332+0000 7fad04d84640 1 --2- 172.21.15.134:0/589456059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad0015b9c0 0x7fad000c51d0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.332+0000 7fad04d84640 1 --2- 172.21.15.134:0/589456059 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad001599f0 0x7fad000c4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.332+0000 7fad04d84640 1 --2- 172.21.15.134:0/589456059 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad000c6ba0 0x7fad000cde50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.332+0000 7fad04d84640 1 -- 172.21.15.134:0/589456059 >> 172.21.15.134:0/589456059 conn(0x7fad0006d8b0 msgr2=0x7fad00159e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:09.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.332+0000 7fad04d84640 1 -- 172.21.15.134:0/589456059 shutdown_connections 2024-04-25T10:30:09.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.333+0000 7fad04d84640 1 -- 172.21.15.134:0/589456059 wait complete. 2024-04-25T10:30:09.338 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-25T10:30:09.336+0000 7fa5d89e3040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T10:30:09.346 DEBUG:teuthology.orchestra.run.smithi134:> sudo ceph --cluster ceph osd new 1b61f16d-5c60-480a-b449-c745aa3f6b73 4 2024-04-25T10:30:09.365 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-25T10:30:09.362+0000 7f94ad81d040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T10:30:09.424 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-25T10:30:09.422+0000 7fa5d89e3040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T10:30:09.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.450+0000 7f2b29e96640 1 Processor -- start 2024-04-25T10:30:09.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.450+0000 7f2b29e96640 1 -- start start 2024-04-25T10:30:09.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.452+0000 7f2b29e96640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2b2406f390 0x7f2b2406dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:09.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.452+0000 7f2b29e96640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f2b240c6250 0x7f2b240c6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:09.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.452+0000 7f2b29e96640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b2415d750 con 0x7f2b241570d0 2024-04-25T10:30:09.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.452+0000 7f2b29e96640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f2b2415dcf0 con 0x7f2b240c6250 2024-04-25T10:30:09.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.452+0000 7f2b29e96640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f2b2406ef00 con 0x7f2b2406f390 2024-04-25T10:30:09.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.452+0000 7f2b23fff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f2b240c6250 0x7f2b240c6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:09.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.452+0000 7f2b23fff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f2b240c6250 0x7f2b240c6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:36646/0 (socket says 172.21.15.134:36646) 2024-04-25T10:30:09.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.452+0000 7f2b23fff640 1 -- 172.21.15.134:0/3831899548 learned_addr learned my addr 172.21.15.134:0/3831899548 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-25T10:30:09.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.452+0000 7f2b237fe640 1 --2- 172.21.15.134:0/3831899548 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2b2406f390 0x7f2b2406dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:09.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.452+0000 7f2b23fff640 1 -- 172.21.15.134:0/3831899548 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2b2406f390 msgr2=0x7f2b2406dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:09.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.452+0000 7f2b23fff640 1 --2- 172.21.15.134:0/3831899548 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2b2406f390 0x7f2b2406dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.452+0000 7f2b23fff640 1 -- 172.21.15.134:0/3831899548 >> v1:172.21.15.134:6789/0 conn(0x7f2b241570d0 legacy=0x7f2b240ca960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:09.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.452+0000 7f2b23fff640 1 -- 172.21.15.134:0/3831899548 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b2415da10 con 0x7f2b240c6250 2024-04-25T10:30:09.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.452+0000 7f2b237fe640 1 --2- 172.21.15.134:0/3831899548 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2b2406f390 0x7f2b2406dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:09.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.453+0000 7f2b23fff640 1 --2- 172.21.15.134:0/3831899548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b240c6250 0x7f2b240c6660 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c009e50 tx=0x7f2b1c02a7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35fc8f8358728130 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:09.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.453+0000 7f2b22ffd640 1 -- 172.21.15.134:0/3831899548 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c036070 con 0x7f2b240c6250 2024-04-25T10:30:09.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.453+0000 7f2b22ffd640 1 -- 172.21.15.134:0/3831899548 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b1c031600 con 0x7f2b240c6250 2024-04-25T10:30:09.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.453+0000 7f2b22ffd640 1 -- 172.21.15.134:0/3831899548 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c0319d0 con 0x7f2b240c6250 2024-04-25T10:30:09.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.454+0000 7f2b29e96640 1 -- 172.21.15.134:0/3831899548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b240c6250 msgr2=0x7f2b240c6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:09.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.454+0000 7f2b29e96640 1 --2- 172.21.15.134:0/3831899548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b240c6250 0x7f2b240c6660 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c009e50 tx=0x7f2b1c02a7a0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.454+0000 7f2b29e96640 1 -- 172.21.15.134:0/3831899548 shutdown_connections 2024-04-25T10:30:09.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.454+0000 7f2b29e96640 1 --2- 172.21.15.134:0/3831899548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b240c6250 0x7f2b240c6660 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.454+0000 7f2b29e96640 1 --2- 172.21.15.134:0/3831899548 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2b2406f390 0x7f2b2406dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.454+0000 7f2b29e96640 1 -- 172.21.15.134:0/3831899548 >> 172.21.15.134:0/3831899548 conn(0x7f2b240cff60 msgr2=0x7f2b240d0370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:09.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.454+0000 7f2b29e96640 1 -- 172.21.15.134:0/3831899548 shutdown_connections 2024-04-25T10:30:09.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.454+0000 7f2b29e96640 1 -- 172.21.15.134:0/3831899548 wait complete. 2024-04-25T10:30:09.459 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-25T10:30:09.454+0000 7f94ad81d040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T10:30:09.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.456+0000 7f2b29e96640 1 Processor -- start 2024-04-25T10:30:09.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.456+0000 7f2b29e96640 1 -- start start 2024-04-25T10:30:09.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.457+0000 7f2b29e96640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b2406f390 0x7f2b24158cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:09.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.457+0000 7f2b29e96640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2b240c6250 0x7f2b2415e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:09.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.457+0000 7f2b29e96640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b241570d0 0x7f2b240cca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:09.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.457+0000 7f2b29e96640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2b2406eb90 con 0x7f2b2406f390 2024-04-25T10:30:09.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.457+0000 7f2b29e96640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2b2406e8b0 con 0x7f2b241570d0 2024-04-25T10:30:09.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.457+0000 7f2b29e96640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f2b2406d840 con 0x7f2b240c6250 2024-04-25T10:30:09.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.457+0000 7f2b1bfff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2b240c6250 0x7f2b2415e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:09.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.457+0000 7f2b23fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b241570d0 0x7f2b240cca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:09.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.457+0000 7f2b1bfff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2b240c6250 0x7f2b2415e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:33234/0 (socket says 172.21.15.134:33234) 2024-04-25T10:30:09.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.457+0000 7f2b1bfff640 1 -- 172.21.15.134:0/2735644315 learned_addr learned my addr 172.21.15.134:0/2735644315 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-25T10:30:09.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.457+0000 7f2b237fe640 1 --2- 172.21.15.134:0/2735644315 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b2406f390 0x7f2b24158cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:09.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.457+0000 7f2b23fff640 1 -- 172.21.15.134:0/2735644315 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2b240c6250 msgr2=0x7f2b2415e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:09.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.457+0000 7f2b23fff640 1 --2- 172.21.15.134:0/2735644315 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2b240c6250 0x7f2b2415e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.457+0000 7f2b23fff640 1 -- 172.21.15.134:0/2735644315 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b2406f390 msgr2=0x7f2b24158cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:09.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.457+0000 7f2b23fff640 1 --2- 172.21.15.134:0/2735644315 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b2406f390 0x7f2b24158cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.457+0000 7f2b23fff640 1 -- 172.21.15.134:0/2735644315 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b2415e6b0 con 0x7f2b241570d0 2024-04-25T10:30:09.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.457+0000 7f2b1bfff640 1 --2- 172.21.15.134:0/2735644315 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2b240c6250 0x7f2b2415e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:09.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.458+0000 7f2b23fff640 1 --2- 172.21.15.134:0/2735644315 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b241570d0 0x7f2b240cca10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2b24074440 tx=0x7f2b1c004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:09.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.458+0000 7f2b20ff9640 1 -- 172.21.15.134:0/2735644315 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c03f020 con 0x7f2b241570d0 2024-04-25T10:30:09.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.458+0000 7f2b20ff9640 1 -- 172.21.15.134:0/2735644315 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b1c037040 con 0x7f2b241570d0 2024-04-25T10:30:09.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.458+0000 7f2b20ff9640 1 -- 172.21.15.134:0/2735644315 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c036070 con 0x7f2b241570d0 2024-04-25T10:30:09.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.458+0000 7f2b29e96640 1 -- 172.21.15.134:0/2735644315 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b24159410 con 0x7f2b241570d0 2024-04-25T10:30:09.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.458+0000 7f2b29e96640 1 -- 172.21.15.134:0/2735644315 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b24160630 con 0x7f2b241570d0 2024-04-25T10:30:09.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.458+0000 7f2b20ff9640 1 -- 172.21.15.134:0/2735644315 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2b1c042020 con 0x7f2b241570d0 2024-04-25T10:30:09.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.458+0000 7f2b20ff9640 1 -- 172.21.15.134:0/2735644315 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f2b1c044aa0 con 0x7f2b241570d0 2024-04-25T10:30:09.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.459+0000 7f2b29e96640 1 -- 172.21.15.134:0/2735644315 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b2406a3d0 con 0x7f2b241570d0 2024-04-25T10:30:09.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.463+0000 7f2b20ff9640 1 -- 172.21.15.134:0/2735644315 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f2b1c049030 con 0x7f2b241570d0 2024-04-25T10:30:09.467 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-25T10:30:09.465+0000 7fa5d89e3040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T10:30:09.503 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-25T10:30:09.500+0000 7f94ad81d040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T10:30:09.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.535+0000 7f2b29e96640 1 -- 172.21.15.134:0/2735644315 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd new", "uuid": "1b61f16d-5c60-480a-b449-c745aa3f6b73", "id": 4} v 0) -- 0x7f2b2415a6f0 con 0x7f2b241570d0 2024-04-25T10:30:09.544 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.542+0000 7f2b20ff9640 1 -- 172.21.15.134:0/2735644315 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1b61f16d-5c60-480a-b449-c745aa3f6b73", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f2b1c009510 con 0x7f2b241570d0 2024-04-25T10:30:09.545 INFO:teuthology.orchestra.run.smithi134.stdout:4 2024-04-25T10:30:09.546 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.543+0000 7f2b29e96640 1 -- 172.21.15.134:0/2735644315 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b241570d0 msgr2=0x7f2b240cca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:09.546 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.543+0000 7f2b29e96640 1 --2- 172.21.15.134:0/2735644315 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b241570d0 0x7f2b240cca10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2b24074440 tx=0x7f2b1c004a40 comp rx=0 tx=0).stop 2024-04-25T10:30:09.546 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.543+0000 7f2b29e96640 1 -- 172.21.15.134:0/2735644315 shutdown_connections 2024-04-25T10:30:09.546 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.543+0000 7f2b29e96640 1 --2- 172.21.15.134:0/2735644315 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b241570d0 0x7f2b240cca10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.546 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.543+0000 7f2b29e96640 1 --2- 172.21.15.134:0/2735644315 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2b240c6250 0x7f2b2415e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.546 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.543+0000 7f2b29e96640 1 --2- 172.21.15.134:0/2735644315 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b2406f390 0x7f2b24158cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.546 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.543+0000 7f2b29e96640 1 -- 172.21.15.134:0/2735644315 >> 172.21.15.134:0/2735644315 conn(0x7f2b240cff60 msgr2=0x7f2b240cb1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:09.546 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.543+0000 7f2b29e96640 1 -- 172.21.15.134:0/2735644315 shutdown_connections 2024-04-25T10:30:09.546 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.543+0000 7f2b29e96640 1 -- 172.21.15.134:0/2735644315 wait complete. 2024-04-25T10:30:09.557 DEBUG:teuthology.orchestra.run.smithi134:> sudo ceph --cluster ceph osd new 944cdd7f-47c0-4b71-b7e0-d093542f4528 5 2024-04-25T10:30:09.631 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-25T10:30:09.629+0000 7fa5d89e3040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T10:30:09.678 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-25T10:30:09.675+0000 7fa5d89e3040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T10:30:09.678 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-25T10:30:09.675+0000 7f94ad81d040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T10:30:09.705 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.702+0000 7f0f5ae9a640 1 Processor -- start 2024-04-25T10:30:09.705 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.702+0000 7f0f5ae9a640 1 -- start start 2024-04-25T10:30:09.707 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.703+0000 7f0f5ae9a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f54156f70 0x7f0f54160c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:09.707 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.703+0000 7f0f5ae9a640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f54161190 con 0x7f0f5415d790 2024-04-25T10:30:09.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.703+0000 7f0f5ae9a640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f540cb590 con 0x7f0f5415d380 2024-04-25T10:30:09.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.703+0000 7f0f5ae9a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f0f5406e040 con 0x7f0f54156f70 2024-04-25T10:30:09.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.703+0000 7f0f58c36640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f54156f70 0x7f0f54160c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:09.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.703+0000 7f0f53fff640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f0f5415d380 0x7f0f540d00a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:41220/0 (socket says 172.21.15.134:41220) 2024-04-25T10:30:09.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.703+0000 7f0f53fff640 1 -- 172.21.15.134:0/681135048 learned_addr learned my addr 172.21.15.134:0/681135048 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-04-25T10:30:09.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.704+0000 7f0f58c36640 1 -- 172.21.15.134:0/681135048 >> v1:172.21.15.134:6789/0 conn(0x7f0f5415d380 legacy=0x7f0f540d00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:09.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.704+0000 7f0f537fe640 1 -- 172.21.15.134:0/681135048 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2388167230 0 0) 0x7f0f540cb590 con 0x7f0f5415d380 2024-04-25T10:30:09.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.704+0000 7f0f58c36640 1 -- 172.21.15.134:0/681135048 >> v1:172.21.15.106:6789/0 conn(0x7f0f5415d790 legacy=0x7f0f541675d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:09.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.704+0000 7f0f58c36640 1 -- 172.21.15.134:0/681135048 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f5406da80 con 0x7f0f54156f70 2024-04-25T10:30:09.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.704+0000 7f0f58c36640 1 --2- 172.21.15.134:0/681135048 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f54156f70 0x7f0f54160c50 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0f480097d0 tx=0x7f0f4802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47dfa778b95a62a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:09.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.705+0000 7f0f537fe640 1 -- 172.21.15.134:0/681135048 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f48037070 con 0x7f0f54156f70 2024-04-25T10:30:09.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.705+0000 7f0f537fe640 1 -- 172.21.15.134:0/681135048 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f48032440 con 0x7f0f54156f70 2024-04-25T10:30:09.709 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.705+0000 7f0f537fe640 1 -- 172.21.15.134:0/681135048 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f48032810 con 0x7f0f54156f70 2024-04-25T10:30:09.710 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.705+0000 7f0f5ae9a640 1 -- 172.21.15.134:0/681135048 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f54156f70 msgr2=0x7f0f54160c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:09.710 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.705+0000 7f0f5ae9a640 1 --2- 172.21.15.134:0/681135048 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f54156f70 0x7f0f54160c50 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0f480097d0 tx=0x7f0f4802a180 comp rx=0 tx=0).stop 2024-04-25T10:30:09.710 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.705+0000 7f0f5ae9a640 1 -- 172.21.15.134:0/681135048 shutdown_connections 2024-04-25T10:30:09.710 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.705+0000 7f0f5ae9a640 1 --2- 172.21.15.134:0/681135048 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f54156f70 0x7f0f54160c50 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.710 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.705+0000 7f0f5ae9a640 1 -- 172.21.15.134:0/681135048 >> 172.21.15.134:0/681135048 conn(0x7f0f54072730 msgr2=0x7f0f54072b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:09.710 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.705+0000 7f0f5ae9a640 1 -- 172.21.15.134:0/681135048 shutdown_connections 2024-04-25T10:30:09.710 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.705+0000 7f0f5ae9a640 1 -- 172.21.15.134:0/681135048 wait complete. 2024-04-25T10:30:09.710 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.707+0000 7f0f5ae9a640 1 Processor -- start 2024-04-25T10:30:09.711 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.707+0000 7f0f5ae9a640 1 -- start start 2024-04-25T10:30:09.714 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.709+0000 7f0f5ae9a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f54156f70 0x7f0f54157f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:09.714 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.709+0000 7f0f5ae9a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f5415d380 0x7f0f54152600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:09.714 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.709+0000 7f0f5ae9a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f5415d790 0x7f0f541605e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:09.714 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.709+0000 7f0f5ae9a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0f5415ce60 con 0x7f0f5415d380 2024-04-25T10:30:09.714 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.709+0000 7f0f5ae9a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0f54157380 con 0x7f0f5415d790 2024-04-25T10:30:09.714 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.709+0000 7f0f5ae9a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f0f5415dba0 con 0x7f0f54156f70 2024-04-25T10:30:09.714 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.709+0000 7f0f58c36640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f54156f70 0x7f0f54157f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:09.714 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.709+0000 7f0f59437640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f5415d790 0x7f0f541605e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:09.714 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.709+0000 7f0f58c36640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f54156f70 0x7f0f54157f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:33258/0 (socket says 172.21.15.134:33258) 2024-04-25T10:30:09.714 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.709+0000 7f0f58c36640 1 -- 172.21.15.134:0/1817450203 learned_addr learned my addr 172.21.15.134:0/1817450203 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-25T10:30:09.715 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.709+0000 7f0f53fff640 1 --2- 172.21.15.134:0/1817450203 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f5415d380 0x7f0f54152600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:09.715 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.709+0000 7f0f59437640 1 -- 172.21.15.134:0/1817450203 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f54156f70 msgr2=0x7f0f54157f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:09.715 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.709+0000 7f0f59437640 1 --2- 172.21.15.134:0/1817450203 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f54156f70 0x7f0f54157f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.715 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.709+0000 7f0f59437640 1 -- 172.21.15.134:0/1817450203 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f5415d380 msgr2=0x7f0f54152600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:09.715 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.709+0000 7f0f59437640 1 --2- 172.21.15.134:0/1817450203 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f5415d380 0x7f0f54152600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.715 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.709+0000 7f0f59437640 1 -- 172.21.15.134:0/1817450203 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f541f31b0 con 0x7f0f5415d790 2024-04-25T10:30:09.715 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.709+0000 7f0f58c36640 1 --2- 172.21.15.134:0/1817450203 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f54156f70 0x7f0f54157f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:09.715 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.709+0000 7f0f59437640 1 --2- 172.21.15.134:0/1817450203 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f5415d790 0x7f0f541605e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c00c760 tx=0x7f0f4c00cc20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:09.715 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.709+0000 7f0f517fa640 1 -- 172.21.15.134:0/1817450203 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c003b70 con 0x7f0f5415d790 2024-04-25T10:30:09.715 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.709+0000 7f0f517fa640 1 -- 172.21.15.134:0/1817450203 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f4c003d10 con 0x7f0f5415d790 2024-04-25T10:30:09.715 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.709+0000 7f0f5ae9a640 1 -- 172.21.15.134:0/1817450203 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f540b1880 con 0x7f0f5415d790 2024-04-25T10:30:09.715 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.709+0000 7f0f517fa640 1 -- 172.21.15.134:0/1817450203 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c00aa90 con 0x7f0f5415d790 2024-04-25T10:30:09.716 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.709+0000 7f0f5ae9a640 1 -- 172.21.15.134:0/1817450203 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f540b2230 con 0x7f0f5415d790 2024-04-25T10:30:09.716 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.710+0000 7f0f517fa640 1 -- 172.21.15.134:0/1817450203 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0f4c00ac30 con 0x7f0f5415d790 2024-04-25T10:30:09.716 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.710+0000 7f0f517fa640 1 -- 172.21.15.134:0/1817450203 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f0f4c0129a0 con 0x7f0f5415d790 2024-04-25T10:30:09.716 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.711+0000 7f0f5ae9a640 1 -- 172.21.15.134:0/1817450203 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f54154d00 con 0x7f0f5415d790 2024-04-25T10:30:09.716 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.713+0000 7f0f517fa640 1 -- 172.21.15.134:0/1817450203 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f0f54154d00 con 0x7f0f5415d790 2024-04-25T10:30:09.730 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-25T10:30:09.728+0000 7fa5d89e3040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T10:30:09.735 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-25T10:30:09.733+0000 7f94ad81d040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T10:30:09.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.785+0000 7f0f5ae9a640 1 -- 172.21.15.134:0/1817450203 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd new", "uuid": "944cdd7f-47c0-4b71-b7e0-d093542f4528", "id": 5} v 0) -- 0x7f0f5406a3d0 con 0x7f0f5415d790 2024-04-25T10:30:09.789 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-25T10:30:09.787+0000 7f94ad81d040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T10:30:09.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.792+0000 7f0f517fa640 1 -- 172.21.15.134:0/1817450203 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "944cdd7f-47c0-4b71-b7e0-d093542f4528", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f0f4c012c40 con 0x7f0f5415d790 2024-04-25T10:30:09.794 INFO:teuthology.orchestra.run.smithi134.stdout:5 2024-04-25T10:30:09.796 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.793+0000 7f0f5ae9a640 1 -- 172.21.15.134:0/1817450203 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f5415d790 msgr2=0x7f0f541605e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:09.796 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.793+0000 7f0f5ae9a640 1 --2- 172.21.15.134:0/1817450203 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f5415d790 0x7f0f541605e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c00c760 tx=0x7f0f4c00cc20 comp rx=0 tx=0).stop 2024-04-25T10:30:09.796 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.793+0000 7f0f5ae9a640 1 -- 172.21.15.134:0/1817450203 shutdown_connections 2024-04-25T10:30:09.796 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.793+0000 7f0f5ae9a640 1 --2- 172.21.15.134:0/1817450203 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f5415d790 0x7f0f541605e0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.796 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.793+0000 7f0f5ae9a640 1 --2- 172.21.15.134:0/1817450203 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f5415d380 0x7f0f54152600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.796 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.793+0000 7f0f5ae9a640 1 --2- 172.21.15.134:0/1817450203 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f54156f70 0x7f0f54157f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.796 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.793+0000 7f0f5ae9a640 1 -- 172.21.15.134:0/1817450203 >> 172.21.15.134:0/1817450203 conn(0x7f0f54072730 msgr2=0x7f0f541545f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:09.796 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.793+0000 7f0f5ae9a640 1 -- 172.21.15.134:0/1817450203 shutdown_connections 2024-04-25T10:30:09.796 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.793+0000 7f0f5ae9a640 1 -- 172.21.15.134:0/1817450203 wait complete. 2024-04-25T10:30:09.806 DEBUG:teuthology.orchestra.run.smithi134:> sudo ceph --cluster ceph osd new cc2446e2-1d21-4135-ac04-6af7c990058e 6 2024-04-25T10:30:09.842 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-25T10:30:09.840+0000 7fa5d89e3040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:30:09.910 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-25T10:30:09.908+0000 7f94ad81d040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:30:09.914 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.912+0000 7fbee9772640 1 Processor -- start 2024-04-25T10:30:09.915 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.912+0000 7fbee9772640 1 -- start start 2024-04-25T10:30:09.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.913+0000 7fbee9772640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fbee40c7e30 0x7fbee40c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:09.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.913+0000 7fbee9772640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbee40b4ce0 0x7fbee415ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:09.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.913+0000 7fbee9772640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbee40ce630 con 0x7fbee4156ef0 2024-04-25T10:30:09.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.913+0000 7fbee9772640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fbee40b50f0 con 0x7fbee40c7e30 2024-04-25T10:30:09.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.913+0000 7fbee9772640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fbee40c7c60 con 0x7fbee40b4ce0 2024-04-25T10:30:09.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.913+0000 7fbee27fc640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fbee4156ef0 0x7fbee406d000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:41228/0 (socket says 172.21.15.134:41228) 2024-04-25T10:30:09.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.913+0000 7fbee27fc640 1 -- 172.21.15.134:0/600751379 learned_addr learned my addr 172.21.15.134:0/600751379 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-04-25T10:30:09.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.913+0000 7fbee37fe640 1 --2- 172.21.15.134:0/600751379 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbee40b4ce0 0x7fbee415ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:09.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.913+0000 7fbee2ffd640 1 --2- 172.21.15.134:0/600751379 >> v2:172.21.15.106:3300/0 conn(0x7fbee40c7e30 0x7fbee40c8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:09.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.914+0000 7fbee1ffb640 1 -- 172.21.15.134:0/600751379 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2867629930 0 0) 0x7fbee40ce630 con 0x7fbee4156ef0 2024-04-25T10:30:09.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.914+0000 7fbee1ffb640 1 -- 172.21.15.134:0/600751379 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbecc003520 con 0x7fbee4156ef0 2024-04-25T10:30:09.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.914+0000 7fbee37fe640 1 -- 172.21.15.134:0/600751379 >> v2:172.21.15.106:3300/0 conn(0x7fbee40c7e30 msgr2=0x7fbee40c8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:09.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.914+0000 7fbee37fe640 1 --2- 172.21.15.134:0/600751379 >> v2:172.21.15.106:3300/0 conn(0x7fbee40c7e30 0x7fbee40c8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.914+0000 7fbee37fe640 1 -- 172.21.15.134:0/600751379 >> v1:172.21.15.134:6789/0 conn(0x7fbee4156ef0 legacy=0x7fbee406d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:09.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.914+0000 7fbee37fe640 1 -- 172.21.15.134:0/600751379 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbee40c7980 con 0x7fbee40b4ce0 2024-04-25T10:30:09.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.914+0000 7fbee37fe640 1 --2- 172.21.15.134:0/600751379 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbee40b4ce0 0x7fbee415ce30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbee40c7de0 tx=0x7fbedc02a5b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47daf81ed244a49e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:09.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.914+0000 7fbee1ffb640 1 -- 172.21.15.134:0/600751379 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbedc007740 con 0x7fbee40b4ce0 2024-04-25T10:30:09.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.915+0000 7fbee1ffb640 1 -- 172.21.15.134:0/600751379 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbedc0078e0 con 0x7fbee40b4ce0 2024-04-25T10:30:09.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.915+0000 7fbee1ffb640 1 -- 172.21.15.134:0/600751379 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbedc007cb0 con 0x7fbee40b4ce0 2024-04-25T10:30:09.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.915+0000 7fbee9772640 1 -- 172.21.15.134:0/600751379 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbee40b4ce0 msgr2=0x7fbee415ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:09.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.915+0000 7fbee9772640 1 --2- 172.21.15.134:0/600751379 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbee40b4ce0 0x7fbee415ce30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbee40c7de0 tx=0x7fbedc02a5b0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.915+0000 7fbee9772640 1 -- 172.21.15.134:0/600751379 shutdown_connections 2024-04-25T10:30:09.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.915+0000 7fbee9772640 1 --2- 172.21.15.134:0/600751379 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbee40b4ce0 0x7fbee415ce30 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.915+0000 7fbee9772640 1 --2- 172.21.15.134:0/600751379 >> v2:172.21.15.106:3300/0 conn(0x7fbee40c7e30 0x7fbee40c8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.915+0000 7fbee9772640 1 -- 172.21.15.134:0/600751379 >> 172.21.15.134:0/600751379 conn(0x7fbee4068b10 msgr2=0x7fbee40d1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:09.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.915+0000 7fbee9772640 1 -- 172.21.15.134:0/600751379 shutdown_connections 2024-04-25T10:30:09.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.915+0000 7fbee9772640 1 -- 172.21.15.134:0/600751379 wait complete. 2024-04-25T10:30:09.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.917+0000 7fbee9772640 1 Processor -- start 2024-04-25T10:30:09.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.917+0000 7fbee9772640 1 -- start start 2024-04-25T10:30:09.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.918+0000 7fbee9772640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbee40b4ce0 0x7fbee4151c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:09.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.918+0000 7fbee9772640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbee40c7e30 0x7fbee41f64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:09.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.918+0000 7fbee9772640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbee4156ef0 0x7fbee41f6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:09.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.918+0000 7fbee9772640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbee406d710 con 0x7fbee40b4ce0 2024-04-25T10:30:09.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.918+0000 7fbee9772640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbee40c72f0 con 0x7fbee4156ef0 2024-04-25T10:30:09.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.918+0000 7fbee9772640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fbee406d890 con 0x7fbee40c7e30 2024-04-25T10:30:09.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.919+0000 7fbee37fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbee4156ef0 0x7fbee41f6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:09.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.919+0000 7fbee27fc640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbee40c7e30 0x7fbee41f64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:09.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.919+0000 7fbee27fc640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbee40c7e30 0x7fbee41f64e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:33286/0 (socket says 172.21.15.134:33286) 2024-04-25T10:30:09.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.919+0000 7fbee27fc640 1 -- 172.21.15.134:0/3816496933 learned_addr learned my addr 172.21.15.134:0/3816496933 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-25T10:30:09.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.919+0000 7fbee37fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbee4156ef0 0x7fbee41f6a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:36678/0 (socket says 172.21.15.134:36678) 2024-04-25T10:30:09.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.919+0000 7fbee2ffd640 1 --2- 172.21.15.134:0/3816496933 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbee40b4ce0 0x7fbee4151c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:09.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.919+0000 7fbee27fc640 1 -- 172.21.15.134:0/3816496933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbee4156ef0 msgr2=0x7fbee41f6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:09.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.919+0000 7fbee27fc640 1 --2- 172.21.15.134:0/3816496933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbee4156ef0 0x7fbee41f6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.919+0000 7fbee27fc640 1 -- 172.21.15.134:0/3816496933 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbee40b4ce0 msgr2=0x7fbee4151c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:09.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.919+0000 7fbee27fc640 1 --2- 172.21.15.134:0/3816496933 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbee40b4ce0 0x7fbee4151c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.919+0000 7fbee27fc640 1 -- 172.21.15.134:0/3816496933 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbee415af30 con 0x7fbee40c7e30 2024-04-25T10:30:09.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.919+0000 7fbee27fc640 1 --2- 172.21.15.134:0/3816496933 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbee40c7e30 0x7fbee41f64e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbed800c430 tx=0x7fbed800c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:09.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.919+0000 7fbecb7fe640 1 -- 172.21.15.134:0/3816496933 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed8017070 con 0x7fbee40c7e30 2024-04-25T10:30:09.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.919+0000 7fbecb7fe640 1 -- 172.21.15.134:0/3816496933 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbed8007500 con 0x7fbee40c7e30 2024-04-25T10:30:09.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.919+0000 7fbee9772640 1 -- 172.21.15.134:0/3816496933 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbee41518a0 con 0x7fbee40c7e30 2024-04-25T10:30:09.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.919+0000 7fbecb7fe640 1 -- 172.21.15.134:0/3816496933 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed80059e0 con 0x7fbee40c7e30 2024-04-25T10:30:09.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.919+0000 7fbee9772640 1 -- 172.21.15.134:0/3816496933 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbee40d0ab0 con 0x7fbee40c7e30 2024-04-25T10:30:09.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.919+0000 7fbecb7fe640 1 -- 172.21.15.134:0/3816496933 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbed801a070 con 0x7fbee40c7e30 2024-04-25T10:30:09.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.920+0000 7fbecb7fe640 1 -- 172.21.15.134:0/3816496933 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7fbed80129f0 con 0x7fbee40c7e30 2024-04-25T10:30:09.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.921+0000 7fbee9772640 1 -- 172.21.15.134:0/3816496933 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbee406a3d0 con 0x7fbee40c7e30 2024-04-25T10:30:09.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.923+0000 7fbecb7fe640 1 -- 172.21.15.134:0/3816496933 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fbed8021020 con 0x7fbee40c7e30 2024-04-25T10:30:09.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:09.996+0000 7fbee9772640 1 -- 172.21.15.134:0/3816496933 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "osd new", "uuid": "cc2446e2-1d21-4135-ac04-6af7c990058e", "id": 6} v 0) -- 0x7fbee415a5a0 con 0x7fbee40c7e30 2024-04-25T10:30:10.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.008+0000 7fbecb7fe640 1 -- 172.21.15.134:0/3816496933 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "cc2446e2-1d21-4135-ac04-6af7c990058e", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7fbed8005cb0 con 0x7fbee40c7e30 2024-04-25T10:30:10.011 INFO:teuthology.orchestra.run.smithi134.stdout:6 2024-04-25T10:30:10.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.009+0000 7fbee9772640 1 -- 172.21.15.134:0/3816496933 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbee40c7e30 msgr2=0x7fbee41f64e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:10.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.009+0000 7fbee9772640 1 --2- 172.21.15.134:0/3816496933 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbee40c7e30 0x7fbee41f64e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbed800c430 tx=0x7fbed800c8f0 comp rx=0 tx=0).stop 2024-04-25T10:30:10.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.009+0000 7fbee9772640 1 -- 172.21.15.134:0/3816496933 shutdown_connections 2024-04-25T10:30:10.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.009+0000 7fbee9772640 1 --2- 172.21.15.134:0/3816496933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbee4156ef0 0x7fbee41f6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:10.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.009+0000 7fbee9772640 1 --2- 172.21.15.134:0/3816496933 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbee40c7e30 0x7fbee41f64e0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:10.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.009+0000 7fbee9772640 1 --2- 172.21.15.134:0/3816496933 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbee40b4ce0 0x7fbee4151c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:10.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.009+0000 7fbee9772640 1 -- 172.21.15.134:0/3816496933 >> 172.21.15.134:0/3816496933 conn(0x7fbee4068b10 msgr2=0x7fbee415cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:10.013 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.009+0000 7fbee9772640 1 -- 172.21.15.134:0/3816496933 shutdown_connections 2024-04-25T10:30:10.013 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.009+0000 7fbee9772640 1 -- 172.21.15.134:0/3816496933 wait complete. 2024-04-25T10:30:10.023 DEBUG:teuthology.orchestra.run.smithi134:> sudo ceph --cluster ceph osd new 3df1fc7a-82fc-4257-8629-98027b6e2aed 7 2024-04-25T10:30:10.131 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.128+0000 7f5729e92640 1 Processor -- start 2024-04-25T10:30:10.131 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.128+0000 7f5729e92640 1 -- start start 2024-04-25T10:30:10.133 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.129+0000 7f5729e92640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f5724156f70 0x7f5724160c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:10.134 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.130+0000 7f5729e92640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f572415d790 0x7f57241675d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:10.134 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.130+0000 7f5729e92640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5724161190 con 0x7f572415d380 2024-04-25T10:30:10.134 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.130+0000 7f5729e92640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f572406e040 con 0x7f5724156f70 2024-04-25T10:30:10.134 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.130+0000 7f5729e92640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f5724157380 con 0x7f572415d790 2024-04-25T10:30:10.134 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.130+0000 7f5723fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f572415d790 0x7f57241675d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:10.134 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.130+0000 7f5723fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f572415d790 0x7f57241675d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:33302/0 (socket says 172.21.15.134:33302) 2024-04-25T10:30:10.134 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.130+0000 7f5723fff640 1 -- 172.21.15.134:0/2014048931 learned_addr learned my addr 172.21.15.134:0/2014048931 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-25T10:30:10.134 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.130+0000 7f57237fe640 1 --2- 172.21.15.134:0/2014048931 >> v2:172.21.15.106:3300/0 conn(0x7f5724156f70 0x7f5724160c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:10.134 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.130+0000 7f5723fff640 1 -- 172.21.15.134:0/2014048931 >> v2:172.21.15.106:3300/0 conn(0x7f5724156f70 msgr2=0x7f5724160c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:10.134 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.130+0000 7f5723fff640 1 --2- 172.21.15.134:0/2014048931 >> v2:172.21.15.106:3300/0 conn(0x7f5724156f70 0x7f5724160c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:10.134 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.130+0000 7f5723fff640 1 -- 172.21.15.134:0/2014048931 >> v1:172.21.15.106:6789/0 conn(0x7f572415d380 legacy=0x7f57240d00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:10.134 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.130+0000 7f5723fff640 1 -- 172.21.15.134:0/2014048931 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57240cb590 con 0x7f572415d790 2024-04-25T10:30:10.135 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.131+0000 7f5723fff640 1 --2- 172.21.15.134:0/2014048931 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f572415d790 0x7f57241675d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f571c009e50 tx=0x7f571c02a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a191a969bdcea7ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:10.136 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.131+0000 7f5722ffd640 1 -- 172.21.15.134:0/2014048931 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f571c036070 con 0x7f572415d790 2024-04-25T10:30:10.136 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.131+0000 7f5722ffd640 1 -- 172.21.15.134:0/2014048931 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f571c031600 con 0x7f572415d790 2024-04-25T10:30:10.136 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.131+0000 7f5722ffd640 1 -- 172.21.15.134:0/2014048931 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f571c0319d0 con 0x7f572415d790 2024-04-25T10:30:10.136 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.132+0000 7f5729e92640 1 -- 172.21.15.134:0/2014048931 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f572415d790 msgr2=0x7f57241675d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:10.136 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.132+0000 7f5729e92640 1 --2- 172.21.15.134:0/2014048931 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f572415d790 0x7f57241675d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f571c009e50 tx=0x7f571c02a680 comp rx=0 tx=0).stop 2024-04-25T10:30:10.136 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.132+0000 7f5729e92640 1 -- 172.21.15.134:0/2014048931 shutdown_connections 2024-04-25T10:30:10.136 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.132+0000 7f5729e92640 1 --2- 172.21.15.134:0/2014048931 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f572415d790 0x7f57241675d0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:10.136 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.132+0000 7f5729e92640 1 --2- 172.21.15.134:0/2014048931 >> v2:172.21.15.106:3300/0 conn(0x7f5724156f70 0x7f5724160c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:10.136 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.132+0000 7f5729e92640 1 -- 172.21.15.134:0/2014048931 >> 172.21.15.134:0/2014048931 conn(0x7f5724072730 msgr2=0x7f5724072b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:10.136 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.132+0000 7f5729e92640 1 -- 172.21.15.134:0/2014048931 shutdown_connections 2024-04-25T10:30:10.136 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.132+0000 7f5729e92640 1 -- 172.21.15.134:0/2014048931 wait complete. 2024-04-25T10:30:10.137 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.134+0000 7f5729e92640 1 Processor -- start 2024-04-25T10:30:10.137 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.134+0000 7f5729e92640 1 -- start start 2024-04-25T10:30:10.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.135+0000 7f5729e92640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5724156f70 0x7f57241f6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:10.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.135+0000 7f5729e92640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f572415d380 0x7f5724075e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:10.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.135+0000 7f5729e92640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f572415d790 0x7f5724149a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:10.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.135+0000 7f5729e92640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f572415ce60 con 0x7f572415d380 2024-04-25T10:30:10.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.135+0000 7f5729e92640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f572406dd60 con 0x7f5724156f70 2024-04-25T10:30:10.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.135+0000 7f5729e92640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f572415dba0 con 0x7f572415d790 2024-04-25T10:30:10.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.135+0000 7f5723fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f572415d790 0x7f5724149a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:10.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.135+0000 7f57237fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5724156f70 0x7f57241f6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:10.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.135+0000 7f5723fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f572415d790 0x7f5724149a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:33308/0 (socket says 172.21.15.134:33308) 2024-04-25T10:30:10.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.135+0000 7f57237fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5724156f70 0x7f57241f6840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:36690/0 (socket says 172.21.15.134:36690) 2024-04-25T10:30:10.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.135+0000 7f5723fff640 1 -- 172.21.15.134:0/2390490118 learned_addr learned my addr 172.21.15.134:0/2390490118 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-25T10:30:10.141 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.135+0000 7f571bfff640 1 --2- 172.21.15.134:0/2390490118 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f572415d380 0x7f5724075e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:10.141 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.136+0000 7f5723fff640 1 -- 172.21.15.134:0/2390490118 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5724156f70 msgr2=0x7f57241f6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:10.141 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.136+0000 7f5723fff640 1 --2- 172.21.15.134:0/2390490118 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5724156f70 0x7f57241f6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:10.141 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.136+0000 7f5723fff640 1 -- 172.21.15.134:0/2390490118 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f572415d380 msgr2=0x7f5724075e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:10.141 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.136+0000 7f5723fff640 1 --2- 172.21.15.134:0/2390490118 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f572415d380 0x7f5724075e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:10.141 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.136+0000 7f5723fff640 1 -- 172.21.15.134:0/2390490118 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57241f6d80 con 0x7f572415d790 2024-04-25T10:30:10.141 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.136+0000 7f5723fff640 1 --2- 172.21.15.134:0/2390490118 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f572415d790 0x7f5724149a20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f571c02ac70 tx=0x7f571c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:10.141 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.136+0000 7f5720ff9640 1 -- 172.21.15.134:0/2390490118 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f571c03f020 con 0x7f572415d790 2024-04-25T10:30:10.141 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.136+0000 7f5720ff9640 1 -- 172.21.15.134:0/2390490118 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f571c037040 con 0x7f572415d790 2024-04-25T10:30:10.141 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.136+0000 7f5729e92640 1 -- 172.21.15.134:0/2390490118 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f572415e060 con 0x7f572415d790 2024-04-25T10:30:10.141 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.136+0000 7f5720ff9640 1 -- 172.21.15.134:0/2390490118 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f571c036070 con 0x7f572415d790 2024-04-25T10:30:10.141 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.136+0000 7f5729e92640 1 -- 172.21.15.134:0/2390490118 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f57240c88f0 con 0x7f572415d790 2024-04-25T10:30:10.141 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.136+0000 7f5720ff9640 1 -- 172.21.15.134:0/2390490118 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f571c042020 con 0x7f572415d790 2024-04-25T10:30:10.141 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.136+0000 7f5720ff9640 1 -- 172.21.15.134:0/2390490118 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f571c003cb0 con 0x7f572415d790 2024-04-25T10:30:10.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.137+0000 7f5729e92640 1 -- 172.21.15.134:0/2390490118 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f572406a3d0 con 0x7f572415d790 2024-04-25T10:30:10.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.141+0000 7f5720ff9640 1 -- 172.21.15.134:0/2390490118 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f571c049070 con 0x7f572415d790 2024-04-25T10:30:10.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.214+0000 7f5729e92640 1 -- 172.21.15.134:0/2390490118 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "osd new", "uuid": "3df1fc7a-82fc-4257-8629-98027b6e2aed", "id": 7} v 0) -- 0x7f572415a670 con 0x7f572415d790 2024-04-25T10:30:10.221 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-25T10:30:10.219+0000 7fa5d89e3040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T10:30:10.224 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.221+0000 7f5720ff9640 1 -- 172.21.15.134:0/2390490118 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3df1fc7a-82fc-4257-8629-98027b6e2aed", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f571c009510 con 0x7f572415d790 2024-04-25T10:30:10.224 INFO:teuthology.orchestra.run.smithi134.stdout:7 2024-04-25T10:30:10.225 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.222+0000 7f5729e92640 1 -- 172.21.15.134:0/2390490118 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f572415d790 msgr2=0x7f5724149a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:10.225 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.222+0000 7f5729e92640 1 --2- 172.21.15.134:0/2390490118 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f572415d790 0x7f5724149a20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f571c02ac70 tx=0x7f571c004a40 comp rx=0 tx=0).stop 2024-04-25T10:30:10.225 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.222+0000 7f5729e92640 1 -- 172.21.15.134:0/2390490118 shutdown_connections 2024-04-25T10:30:10.225 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.222+0000 7f5729e92640 1 --2- 172.21.15.134:0/2390490118 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f572415d790 0x7f5724149a20 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:10.225 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.222+0000 7f5729e92640 1 --2- 172.21.15.134:0/2390490118 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f572415d380 0x7f5724075e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:10.226 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.222+0000 7f5729e92640 1 --2- 172.21.15.134:0/2390490118 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5724156f70 0x7f57241f6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:10.226 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.222+0000 7f5729e92640 1 -- 172.21.15.134:0/2390490118 >> 172.21.15.134:0/2390490118 conn(0x7f5724072730 msgr2=0x7f57240cff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:10.226 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.222+0000 7f5729e92640 1 -- 172.21.15.134:0/2390490118 shutdown_connections 2024-04-25T10:30:10.226 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-25T10:30:10.222+0000 7f5729e92640 1 -- 172.21.15.134:0/2390490118 wait complete. 2024-04-25T10:30:10.236 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-25T10:30:10.236 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-25T10:30:10.238 INFO:tasks.ceph.osd.0:Started 2024-04-25T10:30:10.239 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-25T10:30:10.239 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-25T10:30:10.240 INFO:tasks.ceph.osd.1:Started 2024-04-25T10:30:10.241 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-25T10:30:10.241 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-25T10:30:10.243 INFO:tasks.ceph.osd.2:Started 2024-04-25T10:30:10.243 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-25T10:30:10.243 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-25T10:30:10.245 INFO:tasks.ceph.osd.3:Started 2024-04-25T10:30:10.245 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-25T10:30:10.245 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-25T10:30:10.247 INFO:tasks.ceph.osd.4:Started 2024-04-25T10:30:10.247 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-25T10:30:10.248 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-25T10:30:10.250 INFO:tasks.ceph.osd.5:Started 2024-04-25T10:30:10.250 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-25T10:30:10.250 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-25T10:30:10.252 INFO:tasks.ceph.osd.6:Started 2024-04-25T10:30:10.252 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-25T10:30:10.252 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-25T10:30:10.254 INFO:tasks.ceph.osd.7:Started 2024-04-25T10:30:10.257 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:30:10.327 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-25T10:30:10.324+0000 7f94ad81d040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T10:30:10.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:10.394+0000 7f26a0906640 1 Processor -- start 2024-04-25T10:30:10.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:10.394+0000 7f26a0906640 1 -- start start 2024-04-25T10:30:10.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:10.396+0000 7f26a0906640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f269c154c60 0x7f269c070ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:10.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:10.396+0000 7f26a0906640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f269c071220 0x7f269c0c5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:10.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:10.396+0000 7f26a0906640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f269c0c5940 0x7f269c15aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:10.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:10.396+0000 7f26a0906640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f269c0cb880 con 0x7f269c071220 2024-04-25T10:30:10.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:10.396+0000 7f26a0906640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f269c0caa00 con 0x7f269c0c5940 2024-04-25T10:30:10.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:10.396+0000 7f26a0906640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f269c0cac70 con 0x7f269c154c60 2024-04-25T10:30:10.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:10.396+0000 7f2693fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f269c071220 0x7f269c0c5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:10.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:10.396+0000 7f2693fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f269c071220 0x7f269c0c5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:52746/0 (socket says 172.21.15.106:52746) 2024-04-25T10:30:10.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:10.396+0000 7f2693fff640 1 -- 172.21.15.106:0/1989526109 learned_addr learned my addr 172.21.15.106:0/1989526109 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:10.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:10.396+0000 7f269ad9d640 1 --2- 172.21.15.106:0/1989526109 >> v2:172.21.15.134:3300/0 conn(0x7f269c0c5940 0x7f269c15aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:10.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:10.396+0000 7f269a59c640 1 --2- 172.21.15.106:0/1989526109 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f269c154c60 0x7f269c070ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:10.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:10.396+0000 7f2693fff640 1 -- 172.21.15.106:0/1989526109 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f269c154c60 msgr2=0x7f269c070ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:10.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:10.396+0000 7f2693fff640 1 --2- 172.21.15.106:0/1989526109 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f269c154c60 0x7f269c070ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:10.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:10.396+0000 7f2693fff640 1 -- 172.21.15.106:0/1989526109 >> v2:172.21.15.134:3300/0 conn(0x7f269c0c5940 msgr2=0x7f269c15aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:10.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:10.396+0000 7f2693fff640 1 --2- 172.21.15.106:0/1989526109 >> v2:172.21.15.134:3300/0 conn(0x7f269c0c5940 0x7f269c15aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:10.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:10.396+0000 7f2693fff640 1 -- 172.21.15.106:0/1989526109 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f269c0caf50 con 0x7f269c071220 2024-04-25T10:30:10.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:10.397+0000 7f2693fff640 1 --2- 172.21.15.106:0/1989526109 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f269c071220 0x7f269c0c5400 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2684009870 tx=0x7f268402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9377935b50facdc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:10.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:10.397+0000 7f2699d9b640 1 -- 172.21.15.106:0/1989526109 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2684037070 con 0x7f269c071220 2024-04-25T10:30:10.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:10.397+0000 7f2699d9b640 1 -- 172.21.15.106:0/1989526109 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2684032440 con 0x7f269c071220 2024-04-25T10:30:10.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:10.397+0000 7f2699d9b640 1 -- 172.21.15.106:0/1989526109 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2684032810 con 0x7f269c071220 2024-04-25T10:30:10.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:10.398+0000 7f26a0906640 1 -- 172.21.15.106:0/1989526109 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f269c071220 msgr2=0x7f269c0c5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:10.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:10.398+0000 7f26a0906640 1 --2- 172.21.15.106:0/1989526109 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f269c071220 0x7f269c0c5400 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2684009870 tx=0x7f268402a2a0 comp rx=0 tx=0).stop 2024-04-25T10:30:10.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:10.398+0000 7f26a0906640 1 -- 172.21.15.106:0/1989526109 shutdown_connections 2024-04-25T10:30:10.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:10.398+0000 7f26a0906640 1 --2- 172.21.15.106:0/1989526109 >> v2:172.21.15.134:3300/0 conn(0x7f269c0c5940 0x7f269c15aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:10.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:10.398+0000 7f26a0906640 1 --2- 172.21.15.106:0/1989526109 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f269c071220 0x7f269c0c5400 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:10.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:10.398+0000 7f26a0906640 1 --2- 172.21.15.106:0/1989526109 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f269c154c60 0x7f269c070ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:10.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:10.398+0000 7f26a0906640 1 -- 172.21.15.106:0/1989526109 >> 172.21.15.106:0/1989526109 conn(0x7f269c0cff60 msgr2=0x7f269c0d0370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:10.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:10.398+0000 7f26a0906640 1 -- 172.21.15.106:0/1989526109 shutdown_connections 2024-04-25T10:30:10.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:10.398+0000 7f26a0906640 1 -- 172.21.15.106:0/1989526109 wait complete. 2024-04-25T10:30:10.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:10.400+0000 7f26a0906640 1 Processor -- start 2024-04-25T10:30:10.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:10.400+0000 7f26a0906640 1 -- start start 2024-04-25T10:30:10.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:10.401+0000 7f26a0906640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f269c071220 0x7f269c1f7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:10.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:10.401+0000 7f26a0906640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f269c0c5940 0x7f269c0cf150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:10.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:10.401+0000 7f26a0906640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f269c154c60 0x7f269c13c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:10.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:10.401+0000 7f26a0906640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f269c155070 con 0x7f269c071220 2024-04-25T10:30:10.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:10.401+0000 7f26a0906640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f269c0cb510 con 0x7f269c154c60 2024-04-25T10:30:10.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:10.401+0000 7f26a0906640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f269c1551f0 con 0x7f269c0c5940 2024-04-25T10:30:10.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:10.401+0000 7f269a59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f269c071220 0x7f269c1f7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:10.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:10.401+0000 7f269ad9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f269c154c60 0x7f269c13c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:10.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:10.401+0000 7f269ad9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f269c154c60 0x7f269c13c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.106:43492/0 (socket says 172.21.15.106:43492) 2024-04-25T10:30:10.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:10.401+0000 7f269ad9d640 1 -- 172.21.15.106:0/3926349991 learned_addr learned my addr 172.21.15.106:0/3926349991 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:10.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:10.401+0000 7f2693fff640 1 --2- 172.21.15.106:0/3926349991 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f269c0c5940 0x7f269c0cf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:10.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:10.401+0000 7f269a59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f269c071220 0x7f269c1f7270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:52760/0 (socket says 172.21.15.106:52760) 2024-04-25T10:30:10.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:10.402+0000 7f269ad9d640 1 -- 172.21.15.106:0/3926349991 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f269c0c5940 msgr2=0x7f269c0cf150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:10.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:10.402+0000 7f269ad9d640 1 --2- 172.21.15.106:0/3926349991 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f269c0c5940 0x7f269c0cf150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:10.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:10.402+0000 7f269ad9d640 1 -- 172.21.15.106:0/3926349991 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f269c071220 msgr2=0x7f269c1f7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:10.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:10.402+0000 7f269ad9d640 1 --2- 172.21.15.106:0/3926349991 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f269c071220 0x7f269c1f7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:10.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:10.402+0000 7f269ad9d640 1 -- 172.21.15.106:0/3926349991 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f269c0ca090 con 0x7f269c154c60 2024-04-25T10:30:10.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:10.402+0000 7f269ad9d640 1 --2- 172.21.15.106:0/3926349991 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f269c154c60 0x7f269c13c5d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2694009cb0 tx=0x7f269400c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:10.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:10.402+0000 7f2692ffd640 1 -- 172.21.15.106:0/3926349991 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2694018020 con 0x7f269c154c60 2024-04-25T10:30:10.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:10.402+0000 7f2692ffd640 1 -- 172.21.15.106:0/3926349991 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2694014480 con 0x7f269c154c60 2024-04-25T10:30:10.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:10.402+0000 7f2692ffd640 1 -- 172.21.15.106:0/3926349991 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2694014870 con 0x7f269c154c60 2024-04-25T10:30:10.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:10.403+0000 7f26a0906640 1 -- 172.21.15.106:0/3926349991 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f269c0c9db0 con 0x7f269c154c60 2024-04-25T10:30:10.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:10.403+0000 7f26a0906640 1 -- 172.21.15.106:0/3926349991 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f269c1f6a70 con 0x7f269c154c60 2024-04-25T10:30:10.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:10.403+0000 7f2692ffd640 1 -- 172.21.15.106:0/3926349991 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2694005d10 con 0x7f269c154c60 2024-04-25T10:30:10.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:10.403+0000 7f2692ffd640 1 -- 172.21.15.106:0/3926349991 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f269401dd10 con 0x7f269c154c60 2024-04-25T10:30:10.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:10.404+0000 7f26a0906640 1 -- 172.21.15.106:0/3926349991 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f269c0c9b00 con 0x7f269c154c60 2024-04-25T10:30:10.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:10.408+0000 7f2692ffd640 1 -- 172.21.15.106:0/3926349991 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f2694022030 con 0x7f269c154c60 2024-04-25T10:30:10.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:10.469+0000 7f26a0906640 1 -- 172.21.15.106:0/3926349991 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f269c06a3d0 con 0x7f269c154c60 2024-04-25T10:30:10.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:10.470+0000 7f2692ffd640 1 -- 172.21.15.106:0/3926349991 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f2694010040 con 0x7f269c154c60 2024-04-25T10:30:10.472 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:30:10.472 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":10,"fsid":"3ee77985-e8e4-41cf-81e9-7c774cf7891f","created":"2024-04-25T10:30:07.981054+0000","modified":"2024-04-25T10:30:10.217553+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T10:30:10.217553+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"a5962cb1-dbbd-47b1-8d15-9bae114bfdf1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"94436e13-4769-4752-8397-b5e396a6f7ad","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"429aae0b-70c6-4e7b-9043-6a11334c7efc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"9593a4d1-772f-4667-ab0d-2982f841e342","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"1b61f16d-5c60-480a-b449-c745aa3f6b73","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"944cdd7f-47c0-4b71-b7e0-d093542f4528","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"cc2446e2-1d21-4135-ac04-6af7c990058e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"3df1fc7a-82fc-4257-8629-98027b6e2aed","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:30:10.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:10.471+0000 7f26a0906640 1 -- 172.21.15.106:0/3926349991 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f269c154c60 msgr2=0x7f269c13c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:10.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:10.471+0000 7f26a0906640 1 --2- 172.21.15.106:0/3926349991 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f269c154c60 0x7f269c13c5d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2694009cb0 tx=0x7f269400c490 comp rx=0 tx=0).stop 2024-04-25T10:30:10.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:10.471+0000 7f26a0906640 1 -- 172.21.15.106:0/3926349991 shutdown_connections 2024-04-25T10:30:10.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:10.471+0000 7f26a0906640 1 --2- 172.21.15.106:0/3926349991 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f269c154c60 0x7f269c13c5d0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:10.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:10.471+0000 7f26a0906640 1 --2- 172.21.15.106:0/3926349991 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f269c0c5940 0x7f269c0cf150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:10.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:10.471+0000 7f26a0906640 1 --2- 172.21.15.106:0/3926349991 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f269c071220 0x7f269c1f7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:10.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:10.471+0000 7f26a0906640 1 -- 172.21.15.106:0/3926349991 >> 172.21.15.106:0/3926349991 conn(0x7f269c0cff60 msgr2=0x7f269c0b45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:10.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:10.471+0000 7f26a0906640 1 -- 172.21.15.106:0/3926349991 shutdown_connections 2024-04-25T10:30:10.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:10.471+0000 7f26a0906640 1 -- 172.21.15.106:0/3926349991 wait complete. 2024-04-25T10:30:10.484 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-25T10:30:10.484 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-25T10:30:10.528 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-25T10:30:10.525+0000 7fa5d89e3040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T10:30:10.579 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-25T10:30:10.577+0000 7fa5d89e3040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T10:30:10.598 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-25T10:30:10.595+0000 7f94ad81d040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T10:30:10.652 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-25T10:30:10.649+0000 7f94ad81d040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T10:30:10.696 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-25T10:30:10.694+0000 7fa5d89e3040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T10:30:10.767 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-25T10:30:10.765+0000 7f94ad81d040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T10:30:10.861 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-04-25T10:30:10.858+0000 7f531aeb5640 -1 Falling back to public interface 2024-04-25T10:30:10.862 INFO:tasks.ceph.osd.0.smithi106.stderr:2024-04-25T10:30:10.860+0000 7f5f8d955640 -1 Falling back to public interface 2024-04-25T10:30:10.867 INFO:tasks.ceph.osd.2.smithi106.stderr:2024-04-25T10:30:10.865+0000 7f73e540d640 -1 Falling back to public interface 2024-04-25T10:30:10.874 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-04-25T10:30:10.871+0000 7f7839418640 -1 Falling back to public interface 2024-04-25T10:30:10.880 INFO:tasks.ceph.osd.3.smithi106.stderr:2024-04-25T10:30:10.878+0000 7f03323b6640 -1 Falling back to public interface 2024-04-25T10:30:10.883 INFO:tasks.ceph.osd.1.smithi106.stderr:2024-04-25T10:30:10.881+0000 7f9c6afcb640 -1 Falling back to public interface 2024-04-25T10:30:10.887 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-04-25T10:30:10.884+0000 7f6e415fc640 -1 Falling back to public interface 2024-04-25T10:30:10.895 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-04-25T10:30:10.892+0000 7f9794b31640 -1 Falling back to public interface 2024-04-25T10:30:10.976 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-25T10:30:10.974+0000 7fa5d89e3040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T10:30:11.026 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-25T10:30:11.023+0000 7fa5d89e3040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T10:30:11.049 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-25T10:30:11.047+0000 7f94ad81d040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T10:30:11.097 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-25T10:30:11.094+0000 7f94ad81d040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T10:30:11.183 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-25T10:30:11.181+0000 7fa5d89e3040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T10:30:11.254 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-25T10:30:11.251+0000 7f94ad81d040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T10:30:11.290 DEBUG:teuthology.orchestra.run.smithi106:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T10:30:11.318 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-25T10:30:11.316+0000 7fa5d89e3040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T10:30:11.372 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-25T10:30:11.369+0000 7fa5d89e3040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T10:30:11.386 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-25T10:30:11.384+0000 7f94ad81d040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T10:30:11.391 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:11.389+0000 7fcc619a8640 1 Processor -- start 2024-04-25T10:30:11.391 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:11.389+0000 7fcc619a8640 1 -- start start 2024-04-25T10:30:11.393 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:11.390+0000 7fcc619a8640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcc5c156f70 0x7fcc5c0c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:11.393 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:11.390+0000 7fcc619a8640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc5c15cb70 con 0x7fcc5c0ca530 2024-04-25T10:30:11.393 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:11.390+0000 7fcc619a8640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc5c15ce30 con 0x7fcc5c0b7970 2024-04-25T10:30:11.394 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:11.390+0000 7fcc619a8640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fcc5c15d110 con 0x7fcc5c156f70 2024-04-25T10:30:11.394 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:11.390+0000 7fcc5b7fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fcc5c0ca530 0x7fcc5c0ca940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:51278/0 (socket says 172.21.15.106:51278) 2024-04-25T10:30:11.394 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:11.390+0000 7fcc5b7fe640 1 -- 172.21.15.106:0/3007361434 learned_addr learned my addr 172.21.15.106:0/3007361434 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:30:11.394 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:11.391+0000 7fcc5affd640 1 --2- 172.21.15.106:0/3007361434 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcc5c156f70 0x7fcc5c0c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:11.394 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:11.391+0000 7fcc59ffb640 1 -- 172.21.15.106:0/3007361434 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3888201203 0 0) 0x7fcc5c15cb70 con 0x7fcc5c0ca530 2024-04-25T10:30:11.394 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:11.391+0000 7fcc59ffb640 1 -- 172.21.15.106:0/3007361434 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcc44003520 con 0x7fcc5c0ca530 2024-04-25T10:30:11.394 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:11.391+0000 7fcc5affd640 1 -- 172.21.15.106:0/3007361434 >> v1:172.21.15.134:6789/0 conn(0x7fcc5c0b7970 legacy=0x7fcc5c0cd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:11.394 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:11.391+0000 7fcc5affd640 1 -- 172.21.15.106:0/3007361434 >> v1:172.21.15.106:6789/0 conn(0x7fcc5c0ca530 legacy=0x7fcc5c0ca940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:11.394 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:11.391+0000 7fcc5affd640 1 -- 172.21.15.106:0/3007361434 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc5c15d3f0 con 0x7fcc5c156f70 2024-04-25T10:30:11.396 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:11.392+0000 7fcc5affd640 1 --2- 172.21.15.106:0/3007361434 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcc5c156f70 0x7fcc5c0c9d40 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c0097d0 tx=0x7fcc4c02a290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8c1dc11bba9eff9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:11.396 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:11.392+0000 7fcc59ffb640 1 -- 172.21.15.106:0/3007361434 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc4c037070 con 0x7fcc5c156f70 2024-04-25T10:30:11.396 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:11.392+0000 7fcc59ffb640 1 -- 172.21.15.106:0/3007361434 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc4c032440 con 0x7fcc5c156f70 2024-04-25T10:30:11.396 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:11.392+0000 7fcc59ffb640 1 -- 172.21.15.106:0/3007361434 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc4c0327f0 con 0x7fcc5c156f70 2024-04-25T10:30:11.396 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:11.393+0000 7fcc619a8640 1 -- 172.21.15.106:0/3007361434 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcc5c156f70 msgr2=0x7fcc5c0c9d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:11.396 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:11.393+0000 7fcc619a8640 1 --2- 172.21.15.106:0/3007361434 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcc5c156f70 0x7fcc5c0c9d40 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c0097d0 tx=0x7fcc4c02a290 comp rx=0 tx=0).stop 2024-04-25T10:30:11.396 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:11.393+0000 7fcc619a8640 1 -- 172.21.15.106:0/3007361434 shutdown_connections 2024-04-25T10:30:11.396 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:11.393+0000 7fcc619a8640 1 --2- 172.21.15.106:0/3007361434 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcc5c156f70 0x7fcc5c0c9d40 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:11.396 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:11.393+0000 7fcc619a8640 1 -- 172.21.15.106:0/3007361434 >> 172.21.15.106:0/3007361434 conn(0x7fcc5c0d1f80 msgr2=0x7fcc5c0d2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:11.396 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:11.393+0000 7fcc619a8640 1 -- 172.21.15.106:0/3007361434 shutdown_connections 2024-04-25T10:30:11.396 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:11.393+0000 7fcc619a8640 1 -- 172.21.15.106:0/3007361434 wait complete. 2024-04-25T10:30:11.397 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:11.394+0000 7fcc619a8640 1 Processor -- start 2024-04-25T10:30:11.397 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:11.395+0000 7fcc619a8640 1 -- start start 2024-04-25T10:30:11.399 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:11.396+0000 7fcc619a8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc5c0b7970 0x7fcc5c1f31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:11.399 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:11.396+0000 7fcc619a8640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcc5c0ca530 0x7fcc5c151ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:11.400 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:11.396+0000 7fcc619a8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc5c156f70 0x7fcc5c14c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:11.400 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:11.396+0000 7fcc619a8640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fcc5c0b7d80 con 0x7fcc5c0b7970 2024-04-25T10:30:11.400 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:11.396+0000 7fcc619a8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcc5c15d9b0 con 0x7fcc5c156f70 2024-04-25T10:30:11.400 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:11.396+0000 7fcc619a8640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fcc5c0cdcc0 con 0x7fcc5c0ca530 2024-04-25T10:30:11.400 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:11.396+0000 7fcc5affd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc5c0b7970 0x7fcc5c1f31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:11.400 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:11.396+0000 7fcc5affd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc5c0b7970 0x7fcc5c1f31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:52770/0 (socket says 172.21.15.106:52770) 2024-04-25T10:30:11.400 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:11.396+0000 7fcc5affd640 1 -- 172.21.15.106:0/4258092021 learned_addr learned my addr 172.21.15.106:0/4258092021 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:11.400 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:11.396+0000 7fcc5b7fe640 1 --2- 172.21.15.106:0/4258092021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc5c156f70 0x7fcc5c14c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:11.400 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:11.396+0000 7fcc5a7fc640 1 --2- 172.21.15.106:0/4258092021 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcc5c0ca530 0x7fcc5c151ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:11.400 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:11.396+0000 7fcc5affd640 1 -- 172.21.15.106:0/4258092021 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcc5c0ca530 msgr2=0x7fcc5c151ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:11.400 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:11.396+0000 7fcc5affd640 1 --2- 172.21.15.106:0/4258092021 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcc5c0ca530 0x7fcc5c151ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:11.400 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:11.396+0000 7fcc5affd640 1 -- 172.21.15.106:0/4258092021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc5c156f70 msgr2=0x7fcc5c14c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:11.400 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:11.396+0000 7fcc5affd640 1 --2- 172.21.15.106:0/4258092021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc5c156f70 0x7fcc5c14c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:11.401 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:11.396+0000 7fcc5affd640 1 -- 172.21.15.106:0/4258092021 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc5c1f36e0 con 0x7fcc5c0b7970 2024-04-25T10:30:11.401 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:11.396+0000 7fcc5affd640 1 --2- 172.21.15.106:0/4258092021 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc5c0b7970 0x7fcc5c1f31a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c035040 tx=0x7fcc4c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:11.401 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:11.397+0000 7fcc43fff640 1 -- 172.21.15.106:0/4258092021 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc4c037050 con 0x7fcc5c0b7970 2024-04-25T10:30:11.401 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:11.397+0000 7fcc43fff640 1 -- 172.21.15.106:0/4258092021 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc4c038040 con 0x7fcc5c0b7970 2024-04-25T10:30:11.401 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:11.397+0000 7fcc619a8640 1 -- 172.21.15.106:0/4258092021 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc5c1f2ca0 con 0x7fcc5c0b7970 2024-04-25T10:30:11.401 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:11.397+0000 7fcc43fff640 1 -- 172.21.15.106:0/4258092021 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc4c004490 con 0x7fcc5c0b7970 2024-04-25T10:30:11.401 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:11.397+0000 7fcc619a8640 1 -- 172.21.15.106:0/4258092021 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc5c157610 con 0x7fcc5c0b7970 2024-04-25T10:30:11.401 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:11.397+0000 7fcc43fff640 1 -- 172.21.15.106:0/4258092021 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fcc4c03c6d0 con 0x7fcc5c0b7970 2024-04-25T10:30:11.401 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:11.397+0000 7fcc43fff640 1 -- 172.21.15.106:0/4258092021 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fcc4c03eb20 con 0x7fcc5c0b7970 2024-04-25T10:30:11.401 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:11.398+0000 7fcc619a8640 1 -- 172.21.15.106:0/4258092021 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc5c050740 con 0x7fcc5c0b7970 2024-04-25T10:30:11.402 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:11.400+0000 7fcc43fff640 1 -- 172.21.15.106:0/4258092021 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fcc4c003610 con 0x7fcc5c0b7970 2024-04-25T10:30:11.437 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-25T10:30:11.435+0000 7f94ad81d040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T10:30:11.446 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-25T10:30:11.444+0000 7fa5d89e3040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T10:30:11.464 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:11.462+0000 7fcc619a8640 1 -- 172.21.15.106:0/4258092021 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcc5c050c90 con 0x7fcc5c0b7970 2024-04-25T10:30:11.464 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:11.462+0000 7fcc43fff640 1 -- 172.21.15.106:0/4258092021 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fcc4c03edc0 con 0x7fcc5c0b7970 2024-04-25T10:30:11.465 INFO:teuthology.misc.health.smithi106.stdout: 2024-04-25T10:30:11.465 INFO:teuthology.misc.health.smithi106.stdout:{"epoch":10,"fsid":"3ee77985-e8e4-41cf-81e9-7c774cf7891f","created":"2024-04-25T10:30:07.981054+0000","modified":"2024-04-25T10:30:10.217553+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T10:30:10.217553+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"a5962cb1-dbbd-47b1-8d15-9bae114bfdf1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"94436e13-4769-4752-8397-b5e396a6f7ad","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"429aae0b-70c6-4e7b-9043-6a11334c7efc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"9593a4d1-772f-4667-ab0d-2982f841e342","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"1b61f16d-5c60-480a-b449-c745aa3f6b73","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"944cdd7f-47c0-4b71-b7e0-d093542f4528","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"cc2446e2-1d21-4135-ac04-6af7c990058e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"3df1fc7a-82fc-4257-8629-98027b6e2aed","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:30:11.466 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:11.463+0000 7fcc619a8640 1 -- 172.21.15.106:0/4258092021 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc5c0b7970 msgr2=0x7fcc5c1f31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:11.466 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:11.463+0000 7fcc619a8640 1 --2- 172.21.15.106:0/4258092021 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc5c0b7970 0x7fcc5c1f31a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c035040 tx=0x7fcc4c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:30:11.466 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:11.463+0000 7fcc619a8640 1 -- 172.21.15.106:0/4258092021 shutdown_connections 2024-04-25T10:30:11.466 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:11.463+0000 7fcc619a8640 1 --2- 172.21.15.106:0/4258092021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc5c156f70 0x7fcc5c14c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:11.466 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:11.463+0000 7fcc619a8640 1 --2- 172.21.15.106:0/4258092021 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcc5c0ca530 0x7fcc5c151ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:11.466 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:11.463+0000 7fcc619a8640 1 --2- 172.21.15.106:0/4258092021 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc5c0b7970 0x7fcc5c1f31a0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:11.466 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:11.463+0000 7fcc619a8640 1 -- 172.21.15.106:0/4258092021 >> 172.21.15.106:0/4258092021 conn(0x7fcc5c0d1f80 msgr2=0x7fcc5c06cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:11.466 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:11.463+0000 7fcc619a8640 1 -- 172.21.15.106:0/4258092021 shutdown_connections 2024-04-25T10:30:11.466 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:11.464+0000 7fcc619a8640 1 -- 172.21.15.106:0/4258092021 wait complete. 2024-04-25T10:30:11.475 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-25T10:30:11.494 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-25T10:30:11.492+0000 7fa5d89e3040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T10:30:11.508 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-25T10:30:11.505+0000 7f94ad81d040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T10:30:11.552 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-25T10:30:11.549+0000 7f94ad81d040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T10:30:11.969 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-25T10:30:11.968+0000 7fa5d89e3040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T10:30:12.027 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-25T10:30:12.024+0000 7f94ad81d040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T10:30:12.086 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-25T10:30:12.085+0000 7fa5d89e3040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T10:30:12.145 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-04-25T10:30:12.142+0000 7f94ad81d040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T10:30:13.101 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-25T10:30:13.099+0000 7fa579469640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-25T10:30:13.101 INFO:tasks.ceph.mgr.y.smithi106.stderr:2024-04-25T10:30:13.099+0000 7fa579469640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-25T10:30:14.362 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-04-25T10:30:14.359+0000 7f7839418640 -1 osd.5 0 log_to_monitors true 2024-04-25T10:30:14.364 INFO:tasks.ceph.osd.3.smithi106.stderr:2024-04-25T10:30:14.363+0000 7f03323b6640 -1 osd.3 0 log_to_monitors true 2024-04-25T10:30:14.388 INFO:tasks.ceph.osd.2.smithi106.stderr:2024-04-25T10:30:14.387+0000 7f73e540d640 -1 osd.2 0 log_to_monitors true 2024-04-25T10:30:14.390 INFO:tasks.ceph.osd.0.smithi106.stderr:2024-04-25T10:30:14.389+0000 7f5f8d955640 -1 osd.0 0 log_to_monitors true 2024-04-25T10:30:14.391 INFO:tasks.ceph.osd.1.smithi106.stderr:2024-04-25T10:30:14.390+0000 7f9c6afcb640 -1 osd.1 0 log_to_monitors true 2024-04-25T10:30:14.401 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-04-25T10:30:14.399+0000 7f531aeb5640 -1 osd.6 0 log_to_monitors true 2024-04-25T10:30:14.420 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-04-25T10:30:14.418+0000 7f6e415fc640 -1 osd.7 0 log_to_monitors true 2024-04-25T10:30:14.425 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-04-25T10:30:14.423+0000 7f9794b31640 -1 osd.4 0 log_to_monitors true 2024-04-25T10:30:18.183 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-04-25T10:30:18.180+0000 7f6e3d592640 -1 osd.7 0 waiting for initial osdmap 2024-04-25T10:30:18.194 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-04-25T10:30:18.192+0000 7f6e3538d640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:30:18.211 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-04-25T10:30:18.209+0000 7f5316e4b640 -1 osd.6 0 waiting for initial osdmap 2024-04-25T10:30:18.220 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-04-25T10:30:18.219+0000 7f530ec46640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:30:18.223 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-04-25T10:30:18.222+0000 7f9790ac7640 -1 osd.4 0 waiting for initial osdmap 2024-04-25T10:30:18.235 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-04-25T10:30:18.234+0000 7f97888c2640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:30:18.239 INFO:tasks.ceph.osd.2.smithi106.stderr:2024-04-25T10:30:18.238+0000 7f73e0c9d640 -1 osd.2 0 waiting for initial osdmap 2024-04-25T10:30:18.250 INFO:tasks.ceph.osd.2.smithi106.stderr:2024-04-25T10:30:18.249+0000 7f73d8a98640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:30:18.256 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-04-25T10:30:18.255+0000 7f7834d80640 -1 osd.5 0 waiting for initial osdmap 2024-04-25T10:30:18.265 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-04-25T10:30:18.264+0000 7f782cb7b640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:30:18.267 INFO:tasks.ceph.osd.0.smithi106.stderr:2024-04-25T10:30:18.266+0000 7f5f898eb640 -1 osd.0 0 waiting for initial osdmap 2024-04-25T10:30:18.279 INFO:tasks.ceph.osd.0.smithi106.stderr:2024-04-25T10:30:18.278+0000 7f5f816e6640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:30:18.280 INFO:tasks.ceph.osd.1.smithi106.stderr:2024-04-25T10:30:18.279+0000 7f9c66f61640 -1 osd.1 0 waiting for initial osdmap 2024-04-25T10:30:18.282 DEBUG:teuthology.orchestra.run.smithi106:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T10:30:18.287 INFO:tasks.ceph.osd.3.smithi106.stderr:2024-04-25T10:30:18.286+0000 7f032e34c640 -1 osd.3 0 waiting for initial osdmap 2024-04-25T10:30:18.293 INFO:tasks.ceph.osd.1.smithi106.stderr:2024-04-25T10:30:18.292+0000 7f9c5ed5c640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:30:18.306 INFO:tasks.ceph.osd.3.smithi106.stderr:2024-04-25T10:30:18.305+0000 7f0326147640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:30:18.384 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.383+0000 7f7c642e7640 1 Processor -- start 2024-04-25T10:30:18.384 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.383+0000 7f7c642e7640 1 -- start start 2024-04-25T10:30:18.387 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.384+0000 7f7c642e7640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f7c5c153bf0 0x7f7c5c15cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:18.387 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.384+0000 7f7c642e7640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c5c154000 con 0x7f7c5c0735c0 2024-04-25T10:30:18.387 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.384+0000 7f7c642e7640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c5c073db0 con 0x7f7c5c074850 2024-04-25T10:30:18.387 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.384+0000 7f7c642e7640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f7c5c0c7040 con 0x7f7c5c153bf0 2024-04-25T10:30:18.387 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.384+0000 7f7c62083640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f7c5c0735c0 0x7f7c5c0c9740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:53684/0 (socket says 172.21.15.106:53684) 2024-04-25T10:30:18.387 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.384+0000 7f7c62083640 1 -- 172.21.15.106:0/2717826172 learned_addr learned my addr 172.21.15.106:0/2717826172 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:30:18.387 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.385+0000 7f7c62884640 1 --2- 172.21.15.106:0/2717826172 >> v2:172.21.15.134:3300/0 conn(0x7f7c5c153bf0 0x7f7c5c15cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:18.387 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.385+0000 7f7c61081640 1 -- 172.21.15.106:0/2717826172 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2443114983 0 0) 0x7f7c5c154000 con 0x7f7c5c0735c0 2024-04-25T10:30:18.387 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.385+0000 7f7c61081640 1 -- 172.21.15.106:0/2717826172 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7c48003520 con 0x7f7c5c0735c0 2024-04-25T10:30:18.387 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.385+0000 7f7c61081640 1 -- 172.21.15.106:0/2717826172 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 860191499 0 0) 0x7f7c48003520 con 0x7f7c5c0735c0 2024-04-25T10:30:18.388 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.385+0000 7f7c62884640 1 -- 172.21.15.106:0/2717826172 >> v1:172.21.15.134:6789/0 conn(0x7f7c5c074850 legacy=0x7f7c5c162270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:18.388 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.385+0000 7f7c62884640 1 -- 172.21.15.106:0/2717826172 >> v1:172.21.15.106:6789/0 conn(0x7f7c5c0735c0 legacy=0x7f7c5c0c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:18.388 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.385+0000 7f7c62884640 1 -- 172.21.15.106:0/2717826172 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c5c0c5b20 con 0x7f7c5c153bf0 2024-04-25T10:30:18.388 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.386+0000 7f7c62884640 1 --2- 172.21.15.106:0/2717826172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c5c153bf0 0x7f7c5c15cef0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7c54009d90 tx=0x7f7c5402a5b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24ec223381c2e68f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:18.388 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.386+0000 7f7c61081640 1 -- 172.21.15.106:0/2717826172 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c54035020 con 0x7f7c5c153bf0 2024-04-25T10:30:18.388 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.386+0000 7f7c61081640 1 -- 172.21.15.106:0/2717826172 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c54007540 con 0x7f7c5c153bf0 2024-04-25T10:30:18.388 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.386+0000 7f7c61081640 1 -- 172.21.15.106:0/2717826172 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c580017f0 con 0x7f7c5c153bf0 2024-04-25T10:30:18.389 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.386+0000 7f7c642e7640 1 -- 172.21.15.106:0/2717826172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c5c153bf0 msgr2=0x7f7c5c15cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:18.389 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.386+0000 7f7c642e7640 1 --2- 172.21.15.106:0/2717826172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c5c153bf0 0x7f7c5c15cef0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7c54009d90 tx=0x7f7c5402a5b0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.389 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.386+0000 7f7c642e7640 1 -- 172.21.15.106:0/2717826172 shutdown_connections 2024-04-25T10:30:18.389 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.386+0000 7f7c642e7640 1 --2- 172.21.15.106:0/2717826172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c5c153bf0 0x7f7c5c15cef0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.389 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.386+0000 7f7c642e7640 1 -- 172.21.15.106:0/2717826172 >> 172.21.15.106:0/2717826172 conn(0x7f7c5c06cbf0 msgr2=0x7f7c5c0ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:18.389 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.386+0000 7f7c642e7640 1 -- 172.21.15.106:0/2717826172 shutdown_connections 2024-04-25T10:30:18.390 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.386+0000 7f7c642e7640 1 -- 172.21.15.106:0/2717826172 wait complete. 2024-04-25T10:30:18.390 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.388+0000 7f7c642e7640 1 Processor -- start 2024-04-25T10:30:18.390 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.388+0000 7f7c642e7640 1 -- start start 2024-04-25T10:30:18.392 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.389+0000 7f7c642e7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7c5c0735c0 0x7f7c5c0c9950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:18.392 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.389+0000 7f7c642e7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7c5c074850 0x7f7c5c152d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:18.392 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.389+0000 7f7c642e7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c5c153bf0 0x7f7c5c154a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:18.392 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.389+0000 7f7c642e7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5c074330 con 0x7f7c5c0735c0 2024-04-25T10:30:18.392 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.389+0000 7f7c642e7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5c074050 con 0x7f7c5c153bf0 2024-04-25T10:30:18.392 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.389+0000 7f7c642e7640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f7c5c0c60e0 con 0x7f7c5c074850 2024-04-25T10:30:18.392 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.389+0000 7f7c62083640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7c5c0735c0 0x7f7c5c0c9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:18.392 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.389+0000 7f7c62083640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7c5c0735c0 0x7f7c5c0c9950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41634/0 (socket says 172.21.15.106:41634) 2024-04-25T10:30:18.392 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.389+0000 7f7c62083640 1 -- 172.21.15.106:0/160882050 learned_addr learned my addr 172.21.15.106:0/160882050 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:18.392 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.390+0000 7f7c62884640 1 --2- 172.21.15.106:0/160882050 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c5c153bf0 0x7f7c5c154a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:18.392 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.390+0000 7f7c61882640 1 --2- 172.21.15.106:0/160882050 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7c5c074850 0x7f7c5c152d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:18.393 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.390+0000 7f7c62083640 1 -- 172.21.15.106:0/160882050 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7c5c074850 msgr2=0x7f7c5c152d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:18.393 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.390+0000 7f7c62083640 1 --2- 172.21.15.106:0/160882050 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7c5c074850 0x7f7c5c152d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.393 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.390+0000 7f7c62083640 1 -- 172.21.15.106:0/160882050 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c5c153bf0 msgr2=0x7f7c5c154a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:18.393 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.390+0000 7f7c62083640 1 --2- 172.21.15.106:0/160882050 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c5c153bf0 0x7f7c5c154a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.393 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.390+0000 7f7c62083640 1 -- 172.21.15.106:0/160882050 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c5c151c00 con 0x7f7c5c0735c0 2024-04-25T10:30:18.393 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.390+0000 7f7c61882640 1 --2- 172.21.15.106:0/160882050 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7c5c074850 0x7f7c5c152d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:18.393 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.390+0000 7f7c62083640 1 --2- 172.21.15.106:0/160882050 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7c5c0735c0 0x7f7c5c0c9950 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7c5800c4d0 tx=0x7f7c5800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:18.393 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.390+0000 7f7c52ffd640 1 -- 172.21.15.106:0/160882050 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c58017070 con 0x7f7c5c0735c0 2024-04-25T10:30:18.393 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.390+0000 7f7c52ffd640 1 -- 172.21.15.106:0/160882050 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c58007480 con 0x7f7c5c0735c0 2024-04-25T10:30:18.393 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.390+0000 7f7c642e7640 1 -- 172.21.15.106:0/160882050 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c5c151640 con 0x7f7c5c0735c0 2024-04-25T10:30:18.393 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.390+0000 7f7c52ffd640 1 -- 172.21.15.106:0/160882050 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c580059e0 con 0x7f7c5c0735c0 2024-04-25T10:30:18.393 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.390+0000 7f7c642e7640 1 -- 172.21.15.106:0/160882050 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c5c1520e0 con 0x7f7c5c0735c0 2024-04-25T10:30:18.394 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.391+0000 7f7c642e7640 1 -- 172.21.15.106:0/160882050 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c5c050740 con 0x7f7c5c0735c0 2024-04-25T10:30:18.394 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.392+0000 7f7c52ffd640 1 -- 172.21.15.106:0/160882050 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 4) ==== 96726+0+0 (secure 0 0 0) 0x7f7c5801a070 con 0x7f7c5c0735c0 2024-04-25T10:30:18.394 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.392+0000 7f7c52ffd640 1 --2- 172.21.15.106:0/160882050 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f7c40074420 0x7f7c40076930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:18.394 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.392+0000 7f7c52ffd640 1 -- 172.21.15.106:0/160882050 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2231+0+0 (secure 0 0 0) 0x7f7c58093800 con 0x7f7c5c0735c0 2024-04-25T10:30:18.395 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.392+0000 7f7c61882640 1 --2- 172.21.15.106:0/160882050 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f7c40074420 0x7f7c40076930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:18.395 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.393+0000 7f7c61882640 1 --2- 172.21.15.106:0/160882050 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f7c40074420 0x7f7c40076930 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c007fd0 tx=0x7f7c4c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:18.398 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.397+0000 7f7c52ffd640 1 -- 172.21.15.106:0/160882050 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7c5805ee00 con 0x7f7c5c0735c0 2024-04-25T10:30:18.573 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.572+0000 7f7c642e7640 1 -- 172.21.15.106:0/160882050 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7c5c157320 con 0x7f7c5c0735c0 2024-04-25T10:30:18.574 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.573+0000 7f7c52ffd640 1 -- 172.21.15.106:0/160882050 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) ==== 74+0+7058 (secure 0 0 0) 0x7f7c5805e700 con 0x7f7c5c0735c0 2024-04-25T10:30:18.574 INFO:teuthology.misc.health.smithi106.stdout: 2024-04-25T10:30:18.574 INFO:teuthology.misc.health.smithi106.stdout:{"epoch":12,"fsid":"3ee77985-e8e4-41cf-81e9-7c774cf7891f","created":"2024-04-25T10:30:07.981054+0000","modified":"2024-04-25T10:30:16.104072+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T10:30:10.217553+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"a5962cb1-dbbd-47b1-8d15-9bae114bfdf1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"94436e13-4769-4752-8397-b5e396a6f7ad","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"429aae0b-70c6-4e7b-9043-6a11334c7efc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"9593a4d1-772f-4667-ab0d-2982f841e342","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"1b61f16d-5c60-480a-b449-c745aa3f6b73","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"944cdd7f-47c0-4b71-b7e0-d093542f4528","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"cc2446e2-1d21-4135-ac04-6af7c990058e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"3df1fc7a-82fc-4257-8629-98027b6e2aed","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:30:18.576 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.575+0000 7f7c642e7640 1 -- 172.21.15.106:0/160882050 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f7c40074420 msgr2=0x7f7c40076930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:18.576 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.575+0000 7f7c642e7640 1 --2- 172.21.15.106:0/160882050 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f7c40074420 0x7f7c40076930 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c007fd0 tx=0x7f7c4c009040 comp rx=0 tx=0).stop 2024-04-25T10:30:18.576 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.575+0000 7f7c642e7640 1 -- 172.21.15.106:0/160882050 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7c5c0735c0 msgr2=0x7f7c5c0c9950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:18.576 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.575+0000 7f7c642e7640 1 --2- 172.21.15.106:0/160882050 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7c5c0735c0 0x7f7c5c0c9950 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7c5800c4d0 tx=0x7f7c5800c990 comp rx=0 tx=0).stop 2024-04-25T10:30:18.577 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.575+0000 7f7c642e7640 1 -- 172.21.15.106:0/160882050 shutdown_connections 2024-04-25T10:30:18.577 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.575+0000 7f7c642e7640 1 --2- 172.21.15.106:0/160882050 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f7c40074420 0x7f7c40076930 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.577 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.575+0000 7f7c642e7640 1 --2- 172.21.15.106:0/160882050 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c5c153bf0 0x7f7c5c154a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.577 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.575+0000 7f7c642e7640 1 --2- 172.21.15.106:0/160882050 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7c5c074850 0x7f7c5c152d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.577 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.575+0000 7f7c642e7640 1 --2- 172.21.15.106:0/160882050 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7c5c0735c0 0x7f7c5c0c9950 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.577 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.575+0000 7f7c642e7640 1 -- 172.21.15.106:0/160882050 >> 172.21.15.106:0/160882050 conn(0x7f7c5c06cbf0 msgr2=0x7f7c5c14d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:18.577 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.575+0000 7f7c642e7640 1 -- 172.21.15.106:0/160882050 shutdown_connections 2024-04-25T10:30:18.577 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:18.575+0000 7f7c642e7640 1 -- 172.21.15.106:0/160882050 wait complete. 2024-04-25T10:30:18.586 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-25T10:30:25.392 DEBUG:teuthology.orchestra.run.smithi106:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T10:30:25.506 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.504+0000 7fa05f3f3640 1 Processor -- start 2024-04-25T10:30:25.506 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.504+0000 7fa05f3f3640 1 -- start start 2024-04-25T10:30:25.508 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.505+0000 7fa05f3f3640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fa0580c8810 0x7fa058153b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:25.508 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.505+0000 7fa05f3f3640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa0580c9790 0x7fa0581597a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:25.508 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.505+0000 7fa05f3f3640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0580c8d00 con 0x7fa0580caad0 2024-04-25T10:30:25.508 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.505+0000 7fa05f3f3640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fa0580c6c40 con 0x7fa0580c9790 2024-04-25T10:30:25.508 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.505+0000 7fa05f3f3640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fa0580c8fe0 con 0x7fa0580c8810 2024-04-25T10:30:25.508 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.505+0000 7fa05c98e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa0580c9790 0x7fa0581597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:25.508 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.505+0000 7fa05c98e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa0580c9790 0x7fa0581597a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57806/0 (socket says 172.21.15.106:57806) 2024-04-25T10:30:25.508 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.505+0000 7fa05c98e640 1 -- 172.21.15.106:0/2125332075 learned_addr learned my addr 172.21.15.106:0/2125332075 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:25.508 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.506+0000 7fa05d18f640 1 --2- 172.21.15.106:0/2125332075 >> v2:172.21.15.134:3300/0 conn(0x7fa0580c8810 0x7fa058153b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:25.509 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.506+0000 7fa05c98e640 1 -- 172.21.15.106:0/2125332075 >> v2:172.21.15.134:3300/0 conn(0x7fa0580c8810 msgr2=0x7fa058153b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:25.509 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.506+0000 7fa05c98e640 1 --2- 172.21.15.106:0/2125332075 >> v2:172.21.15.134:3300/0 conn(0x7fa0580c8810 0x7fa058153b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.509 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.506+0000 7fa05c98e640 1 -- 172.21.15.106:0/2125332075 >> v1:172.21.15.134:6789/0 conn(0x7fa0580caad0 legacy=0x7fa05815cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:25.509 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.506+0000 7fa05c98e640 1 -- 172.21.15.106:0/2125332075 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0580c65f0 con 0x7fa0580c9790 2024-04-25T10:30:25.510 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.506+0000 7fa05c98e640 1 --2- 172.21.15.106:0/2125332075 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa0580c9790 0x7fa0581597a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa058073430 tx=0x7fa04802c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cd8c080612b1e8a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:25.510 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.506+0000 7fa04ffff640 1 -- 172.21.15.106:0/2125332075 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa048036020 con 0x7fa0580c9790 2024-04-25T10:30:25.510 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.507+0000 7fa04ffff640 1 -- 172.21.15.106:0/2125332075 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa048032830 con 0x7fa0580c9790 2024-04-25T10:30:25.510 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.507+0000 7fa04ffff640 1 -- 172.21.15.106:0/2125332075 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa048032c00 con 0x7fa0580c9790 2024-04-25T10:30:25.510 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.507+0000 7fa05f3f3640 1 -- 172.21.15.106:0/2125332075 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa0580c9790 msgr2=0x7fa0581597a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:25.510 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.507+0000 7fa05f3f3640 1 --2- 172.21.15.106:0/2125332075 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa0580c9790 0x7fa0581597a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa058073430 tx=0x7fa04802c970 comp rx=0 tx=0).stop 2024-04-25T10:30:25.510 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.507+0000 7fa05f3f3640 1 -- 172.21.15.106:0/2125332075 shutdown_connections 2024-04-25T10:30:25.510 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.507+0000 7fa05f3f3640 1 --2- 172.21.15.106:0/2125332075 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa0580c9790 0x7fa0581597a0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.511 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.507+0000 7fa05f3f3640 1 --2- 172.21.15.106:0/2125332075 >> v2:172.21.15.134:3300/0 conn(0x7fa0580c8810 0x7fa058153b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.511 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.507+0000 7fa05f3f3640 1 -- 172.21.15.106:0/2125332075 >> 172.21.15.106:0/2125332075 conn(0x7fa05806daf0 msgr2=0x7fa05806df00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:25.511 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.507+0000 7fa05f3f3640 1 -- 172.21.15.106:0/2125332075 shutdown_connections 2024-04-25T10:30:25.511 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.507+0000 7fa05f3f3640 1 -- 172.21.15.106:0/2125332075 wait complete. 2024-04-25T10:30:25.511 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.509+0000 7fa05f3f3640 1 Processor -- start 2024-04-25T10:30:25.511 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.509+0000 7fa05f3f3640 1 -- start start 2024-04-25T10:30:25.514 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.510+0000 7fa05f3f3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa0580c8810 0x7fa0581557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:25.514 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.510+0000 7fa05f3f3640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa0580c9790 0x7fa058156940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:25.514 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.510+0000 7fa05f3f3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0580caad0 0x7fa05814d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:25.514 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.510+0000 7fa05f3f3640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa0580c70d0 con 0x7fa0580c8810 2024-04-25T10:30:25.514 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.510+0000 7fa05f3f3640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa058072190 con 0x7fa0580caad0 2024-04-25T10:30:25.514 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.510+0000 7fa05f3f3640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa0581561d0 con 0x7fa0580c9790 2024-04-25T10:30:25.514 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.510+0000 7fa05d18f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa0580c8810 0x7fa0581557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:25.514 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.510+0000 7fa05d18f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa0580c8810 0x7fa0581557c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57808/0 (socket says 172.21.15.106:57808) 2024-04-25T10:30:25.514 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.510+0000 7fa05d18f640 1 -- 172.21.15.106:0/1697806765 learned_addr learned my addr 172.21.15.106:0/1697806765 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:25.514 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.511+0000 7fa05c98e640 1 --2- 172.21.15.106:0/1697806765 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa0580c9790 0x7fa058156940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:25.514 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.511+0000 7fa05d990640 1 --2- 172.21.15.106:0/1697806765 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0580caad0 0x7fa05814d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:25.515 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.511+0000 7fa05d18f640 1 -- 172.21.15.106:0/1697806765 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa0580c9790 msgr2=0x7fa058156940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:25.515 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.511+0000 7fa05d18f640 1 --2- 172.21.15.106:0/1697806765 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa0580c9790 0x7fa058156940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.515 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.511+0000 7fa05d18f640 1 -- 172.21.15.106:0/1697806765 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0580caad0 msgr2=0x7fa05814d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:25.515 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.511+0000 7fa05d18f640 1 --2- 172.21.15.106:0/1697806765 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0580caad0 0x7fa05814d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.515 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.511+0000 7fa05d18f640 1 -- 172.21.15.106:0/1697806765 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa058155d00 con 0x7fa0580c8810 2024-04-25T10:30:25.515 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.511+0000 7fa05d18f640 1 --2- 172.21.15.106:0/1697806765 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa0580c8810 0x7fa0581557c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa05400c820 tx=0x7fa05400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:25.515 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.511+0000 7fa04dffb640 1 -- 172.21.15.106:0/1697806765 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa054015070 con 0x7fa0580c8810 2024-04-25T10:30:25.515 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.511+0000 7fa04dffb640 1 -- 172.21.15.106:0/1697806765 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa054004020 con 0x7fa0580c8810 2024-04-25T10:30:25.515 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.511+0000 7fa05f3f3640 1 -- 172.21.15.106:0/1697806765 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa058154170 con 0x7fa0580c8810 2024-04-25T10:30:25.515 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.511+0000 7fa04dffb640 1 -- 172.21.15.106:0/1697806765 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0540043f0 con 0x7fa0580c8810 2024-04-25T10:30:25.515 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.511+0000 7fa05f3f3640 1 -- 172.21.15.106:0/1697806765 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa058145680 con 0x7fa0580c8810 2024-04-25T10:30:25.516 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.512+0000 7fa05f3f3640 1 -- 172.21.15.106:0/1697806765 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0580ceac0 con 0x7fa0580c8810 2024-04-25T10:30:25.516 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.513+0000 7fa04dffb640 1 -- 172.21.15.106:0/1697806765 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa054007830 con 0x7fa0580c8810 2024-04-25T10:30:25.516 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.513+0000 7fa04dffb640 1 --2- 172.21.15.106:0/1697806765 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fa03c0744e0 0x7fa03c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:25.516 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.513+0000 7fa04dffb640 1 -- 172.21.15.106:0/1697806765 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fa054094c30 con 0x7fa0580c8810 2024-04-25T10:30:25.516 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.513+0000 7fa05c98e640 1 --2- 172.21.15.106:0/1697806765 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fa03c0744e0 0x7fa03c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:25.517 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.514+0000 7fa05c98e640 1 --2- 172.21.15.106:0/1697806765 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fa03c0744e0 0x7fa03c0769f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa048037000 tx=0x7fa048032000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:25.520 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.518+0000 7fa04dffb640 1 -- 172.21.15.106:0/1697806765 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa0580ceac0 con 0x7fa0580c8810 2024-04-25T10:30:25.694 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.691+0000 7fa05f3f3640 1 -- 172.21.15.106:0/1697806765 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa058050740 con 0x7fa0580c8810 2024-04-25T10:30:25.694 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.692+0000 7fa04dffb640 1 -- 172.21.15.106:0/1697806765 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12920 (secure 0 0 0) 0x7fa05405f7a0 con 0x7fa0580c8810 2024-04-25T10:30:25.695 INFO:teuthology.misc.health.smithi106.stdout: 2024-04-25T10:30:25.695 INFO:teuthology.misc.health.smithi106.stdout:{"epoch":16,"fsid":"3ee77985-e8e4-41cf-81e9-7c774cf7891f","created":"2024-04-25T10:30:07.981054+0000","modified":"2024-04-25T10:30:22.366852+0000","last_up_change":"2024-04-25T10:30:19.184325+0000","last_in_change":"2024-04-25T10:30:10.217553+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:30:20.111046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a5962cb1-dbbd-47b1-8d15-9bae114bfdf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1240717079},{"type":"v1","addr":"172.21.15.106:6801","nonce":1240717079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1240717079},{"type":"v1","addr":"172.21.15.106:6803","nonce":1240717079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1240717079},{"type":"v1","addr":"172.21.15.106:6807","nonce":1240717079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1240717079},{"type":"v1","addr":"172.21.15.106:6805","nonce":1240717079}]},"public_addr":"172.21.15.106:6801/1240717079","cluster_addr":"172.21.15.106:6803/1240717079","heartbeat_back_addr":"172.21.15.106:6807/1240717079","heartbeat_front_addr":"172.21.15.106:6805/1240717079","state":["exists","up"]},{"osd":1,"uuid":"94436e13-4769-4752-8397-b5e396a6f7ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4089653051},{"type":"v1","addr":"172.21.15.106:6825","nonce":4089653051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4089653051},{"type":"v1","addr":"172.21.15.106:6827","nonce":4089653051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4089653051},{"type":"v1","addr":"172.21.15.106:6831","nonce":4089653051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4089653051},{"type":"v1","addr":"172.21.15.106:6829","nonce":4089653051}]},"public_addr":"172.21.15.106:6825/4089653051","cluster_addr":"172.21.15.106:6827/4089653051","heartbeat_back_addr":"172.21.15.106:6831/4089653051","heartbeat_front_addr":"172.21.15.106:6829/4089653051","state":["exists","up"]},{"osd":2,"uuid":"429aae0b-70c6-4e7b-9043-6a11334c7efc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":1441018239},{"type":"v1","addr":"172.21.15.106:6809","nonce":1441018239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1441018239},{"type":"v1","addr":"172.21.15.106:6811","nonce":1441018239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1441018239},{"type":"v1","addr":"172.21.15.106:6815","nonce":1441018239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":1441018239},{"type":"v1","addr":"172.21.15.106:6813","nonce":1441018239}]},"public_addr":"172.21.15.106:6809/1441018239","cluster_addr":"172.21.15.106:6811/1441018239","heartbeat_back_addr":"172.21.15.106:6815/1441018239","heartbeat_front_addr":"172.21.15.106:6813/1441018239","state":["exists","up"]},{"osd":3,"uuid":"9593a4d1-772f-4667-ab0d-2982f841e342","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":1846968711},{"type":"v1","addr":"172.21.15.106:6817","nonce":1846968711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1846968711},{"type":"v1","addr":"172.21.15.106:6819","nonce":1846968711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":1846968711},{"type":"v1","addr":"172.21.15.106:6823","nonce":1846968711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":1846968711},{"type":"v1","addr":"172.21.15.106:6821","nonce":1846968711}]},"public_addr":"172.21.15.106:6817/1846968711","cluster_addr":"172.21.15.106:6819/1846968711","heartbeat_back_addr":"172.21.15.106:6823/1846968711","heartbeat_front_addr":"172.21.15.106:6821/1846968711","state":["exists","up"]},{"osd":4,"uuid":"1b61f16d-5c60-480a-b449-c745aa3f6b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":729224739},{"type":"v1","addr":"172.21.15.134:6825","nonce":729224739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":729224739},{"type":"v1","addr":"172.21.15.134:6827","nonce":729224739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":729224739},{"type":"v1","addr":"172.21.15.134:6831","nonce":729224739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":729224739},{"type":"v1","addr":"172.21.15.134:6829","nonce":729224739}]},"public_addr":"172.21.15.134:6825/729224739","cluster_addr":"172.21.15.134:6827/729224739","heartbeat_back_addr":"172.21.15.134:6831/729224739","heartbeat_front_addr":"172.21.15.134:6829/729224739","state":["exists","up"]},{"osd":5,"uuid":"944cdd7f-47c0-4b71-b7e0-d093542f4528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3132275652},{"type":"v1","addr":"172.21.15.134:6809","nonce":3132275652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3132275652},{"type":"v1","addr":"172.21.15.134:6811","nonce":3132275652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3132275652},{"type":"v1","addr":"172.21.15.134:6815","nonce":3132275652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3132275652},{"type":"v1","addr":"172.21.15.134:6813","nonce":3132275652}]},"public_addr":"172.21.15.134:6809/3132275652","cluster_addr":"172.21.15.134:6811/3132275652","heartbeat_back_addr":"172.21.15.134:6815/3132275652","heartbeat_front_addr":"172.21.15.134:6813/3132275652","state":["exists","up"]},{"osd":6,"uuid":"cc2446e2-1d21-4135-ac04-6af7c990058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":992702668},{"type":"v1","addr":"172.21.15.134:6801","nonce":992702668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":992702668},{"type":"v1","addr":"172.21.15.134:6803","nonce":992702668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":992702668},{"type":"v1","addr":"172.21.15.134:6807","nonce":992702668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":992702668},{"type":"v1","addr":"172.21.15.134:6805","nonce":992702668}]},"public_addr":"172.21.15.134:6801/992702668","cluster_addr":"172.21.15.134:6803/992702668","heartbeat_back_addr":"172.21.15.134:6807/992702668","heartbeat_front_addr":"172.21.15.134:6805/992702668","state":["exists","up"]},{"osd":7,"uuid":"3df1fc7a-82fc-4257-8629-98027b6e2aed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":3291062284},{"type":"v1","addr":"172.21.15.134:6817","nonce":3291062284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":3291062284},{"type":"v1","addr":"172.21.15.134:6819","nonce":3291062284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":3291062284},{"type":"v1","addr":"172.21.15.134:6823","nonce":3291062284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":3291062284},{"type":"v1","addr":"172.21.15.134:6821","nonce":3291062284}]},"public_addr":"172.21.15.134:6817/3291062284","cluster_addr":"172.21.15.134:6819/3291062284","heartbeat_back_addr":"172.21.15.134:6823/3291062284","heartbeat_front_addr":"172.21.15.134:6821/3291062284","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.403941+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.382563+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.411544+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.330444+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.405430+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.387737+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.408922+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.454450+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:30:25.697 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.694+0000 7fa05f3f3640 1 -- 172.21.15.106:0/1697806765 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fa03c0744e0 msgr2=0x7fa03c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:25.697 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.694+0000 7fa05f3f3640 1 --2- 172.21.15.106:0/1697806765 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fa03c0744e0 0x7fa03c0769f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa048037000 tx=0x7fa048032000 comp rx=0 tx=0).stop 2024-04-25T10:30:25.697 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.694+0000 7fa05f3f3640 1 -- 172.21.15.106:0/1697806765 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa0580c8810 msgr2=0x7fa0581557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:25.697 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.694+0000 7fa05f3f3640 1 --2- 172.21.15.106:0/1697806765 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa0580c8810 0x7fa0581557c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa05400c820 tx=0x7fa05400cce0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.697 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.694+0000 7fa05f3f3640 1 -- 172.21.15.106:0/1697806765 shutdown_connections 2024-04-25T10:30:25.697 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.694+0000 7fa05f3f3640 1 --2- 172.21.15.106:0/1697806765 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fa03c0744e0 0x7fa03c0769f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.697 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.694+0000 7fa05f3f3640 1 --2- 172.21.15.106:0/1697806765 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0580caad0 0x7fa05814d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.697 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.694+0000 7fa05f3f3640 1 --2- 172.21.15.106:0/1697806765 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa0580c9790 0x7fa058156940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.697 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.694+0000 7fa05f3f3640 1 --2- 172.21.15.106:0/1697806765 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa0580c8810 0x7fa0581557c0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.698 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.694+0000 7fa05f3f3640 1 -- 172.21.15.106:0/1697806765 >> 172.21.15.106:0/1697806765 conn(0x7fa05806daf0 msgr2=0x7fa0580caee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:25.698 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.694+0000 7fa05f3f3640 1 -- 172.21.15.106:0/1697806765 shutdown_connections 2024-04-25T10:30:25.698 INFO:teuthology.misc.health.smithi106.stderr:2024-04-25T10:30:25.694+0000 7fa05f3f3640 1 -- 172.21.15.106:0/1697806765 wait complete. 2024-04-25T10:30:25.707 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-25T10:30:25.707 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-25T10:30:25.707 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:30:25.707 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:30:25.709 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:30:25.710 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:30:25.710 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:30:25.712 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:30:25.712 INFO:tasks.ceph.mds.e:Restarting daemon 2024-04-25T10:30:25.712 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2024-04-25T10:30:25.714 INFO:tasks.ceph.mds.e:Started 2024-04-25T10:30:25.714 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:30:25.714 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:30:25.717 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:30:25.717 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:30:25.717 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:30:25.719 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:30:25.719 INFO:tasks.ceph.mds.f:Restarting daemon 2024-04-25T10:30:25.719 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i f 2024-04-25T10:30:25.721 INFO:tasks.ceph.mds.f:Started 2024-04-25T10:30:25.722 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-25T10:30:25.722 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:30:25.796 INFO:tasks.ceph.mds.a.smithi106.stdout:starting mds.a at 2024-04-25T10:30:25.797 INFO:tasks.ceph.mds.c.smithi106.stdout:starting mds.c at 2024-04-25T10:30:25.802 INFO:tasks.ceph.mds.e.smithi106.stdout:starting mds.e at 2024-04-25T10:30:25.818 INFO:tasks.ceph.mds.f.smithi134.stdout:starting mds.f at 2024-04-25T10:30:25.819 INFO:tasks.ceph.mds.b.smithi134.stdout:starting mds.b at 2024-04-25T10:30:25.819 INFO:tasks.ceph.mds.d.smithi134.stdout:starting mds.d at 2024-04-25T10:30:25.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:25.841+0000 7f2bf59d7640 1 Processor -- start 2024-04-25T10:30:25.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:25.841+0000 7f2bf59d7640 1 -- start start 2024-04-25T10:30:25.845 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:25.842+0000 7f2bf59d7640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2bf01570d0 0x7f2bf006dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:25.845 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:25.842+0000 7f2bf59d7640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2bf015d750 con 0x7f2bf006f390 2024-04-25T10:30:25.845 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:25.842+0000 7f2bf59d7640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2bf015da10 con 0x7f2bf00c6250 2024-04-25T10:30:25.845 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:25.842+0000 7f2bf59d7640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f2bf015dcf0 con 0x7f2bf01570d0 2024-04-25T10:30:25.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:25.843+0000 7f2bee7fc640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2bf01570d0 0x7f2bf006dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:25.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:25.843+0000 7f2beeffd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f2bf006f390 0x7f2bf00ca960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:51240/0 (socket says 172.21.15.106:51240) 2024-04-25T10:30:25.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:25.843+0000 7f2beeffd640 1 -- 172.21.15.106:0/1837599247 learned_addr learned my addr 172.21.15.106:0/1837599247 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:30:25.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:25.843+0000 7f2bee7fc640 1 -- 172.21.15.106:0/1837599247 >> v1:172.21.15.134:6789/0 conn(0x7f2bf00c6250 legacy=0x7f2bf00c6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:25.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:25.843+0000 7f2bee7fc640 1 -- 172.21.15.106:0/1837599247 >> v1:172.21.15.106:6789/0 conn(0x7f2bf006f390 legacy=0x7f2bf00ca960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:25.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:25.843+0000 7f2bee7fc640 1 -- 172.21.15.106:0/1837599247 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bf006e5d0 con 0x7f2bf01570d0 2024-04-25T10:30:25.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:25.844+0000 7f2bee7fc640 1 --2- 172.21.15.106:0/1837599247 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2bf01570d0 0x7f2bf006dd50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2be00097a0 tx=0x7f2be002a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=483d09ad9d99e559 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:25.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:25.844+0000 7f2bedffb640 1 -- 172.21.15.106:0/1837599247 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be0037070 con 0x7f2bf01570d0 2024-04-25T10:30:25.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:25.844+0000 7f2bedffb640 1 -- 172.21.15.106:0/1837599247 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2be002adf0 con 0x7f2bf01570d0 2024-04-25T10:30:25.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:25.844+0000 7f2bedffb640 1 -- 172.21.15.106:0/1837599247 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be0032660 con 0x7f2bf01570d0 2024-04-25T10:30:25.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:25.844+0000 7f2bf59d7640 1 -- 172.21.15.106:0/1837599247 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2bf01570d0 msgr2=0x7f2bf006dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:25.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:25.844+0000 7f2bf59d7640 1 --2- 172.21.15.106:0/1837599247 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2bf01570d0 0x7f2bf006dd50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2be00097a0 tx=0x7f2be002a160 comp rx=0 tx=0).stop 2024-04-25T10:30:25.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:25.844+0000 7f2bf59d7640 1 -- 172.21.15.106:0/1837599247 shutdown_connections 2024-04-25T10:30:25.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:25.844+0000 7f2bf59d7640 1 --2- 172.21.15.106:0/1837599247 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2bf01570d0 0x7f2bf006dd50 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:25.844+0000 7f2bf59d7640 1 -- 172.21.15.106:0/1837599247 >> 172.21.15.106:0/1837599247 conn(0x7f2bf00cff60 msgr2=0x7f2bf00d0370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:25.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:25.844+0000 7f2bf59d7640 1 -- 172.21.15.106:0/1837599247 shutdown_connections 2024-04-25T10:30:25.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:25.844+0000 7f2bf59d7640 1 -- 172.21.15.106:0/1837599247 wait complete. 2024-04-25T10:30:25.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:25.846+0000 7f2bf59d7640 1 Processor -- start 2024-04-25T10:30:25.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:25.846+0000 7f2bf59d7640 1 -- start start 2024-04-25T10:30:25.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:25.847+0000 7f2bf59d7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2bf006f390 0x7f2bf015de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:25.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:25.848+0000 7f2bf59d7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bf00c6250 0x7f2bf0150d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:25.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:25.848+0000 7f2bf59d7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2bf01570d0 0x7f2bf0159dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:25.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:25.848+0000 7f2bf59d7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf006eb90 con 0x7f2bf01570d0 2024-04-25T10:30:25.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:25.848+0000 7f2bf59d7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf006ef00 con 0x7f2bf006f390 2024-04-25T10:30:25.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:25.848+0000 7f2bf59d7640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f2bf006d840 con 0x7f2bf00c6250 2024-04-25T10:30:25.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:25.848+0000 7f2bef7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2bf01570d0 0x7f2bf0159dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:25.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:25.848+0000 7f2bef7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2bf01570d0 0x7f2bf0159dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57860/0 (socket says 172.21.15.106:57860) 2024-04-25T10:30:25.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:25.848+0000 7f2bef7fe640 1 -- 172.21.15.106:0/3080420946 learned_addr learned my addr 172.21.15.106:0/3080420946 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:25.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:25.848+0000 7f2beeffd640 1 --2- 172.21.15.106:0/3080420946 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2bf006f390 0x7f2bf015de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:25.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:25.848+0000 7f2bee7fc640 1 --2- 172.21.15.106:0/3080420946 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bf00c6250 0x7f2bf0150d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:25.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:25.848+0000 7f2bef7fe640 1 -- 172.21.15.106:0/3080420946 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bf00c6250 msgr2=0x7f2bf0150d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:25.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:25.848+0000 7f2bef7fe640 1 --2- 172.21.15.106:0/3080420946 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bf00c6250 0x7f2bf0150d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:25.848+0000 7f2bef7fe640 1 -- 172.21.15.106:0/3080420946 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2bf006f390 msgr2=0x7f2bf015de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:25.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:25.848+0000 7f2bef7fe640 1 --2- 172.21.15.106:0/3080420946 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2bf006f390 0x7f2bf015de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:25.848+0000 7f2bef7fe640 1 -- 172.21.15.106:0/3080420946 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bf015f800 con 0x7f2bf01570d0 2024-04-25T10:30:25.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:25.848+0000 7f2bef7fe640 1 --2- 172.21.15.106:0/3080420946 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2bf01570d0 0x7f2bf0159dd0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2be800b9a0 tx=0x7f2be800be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:25.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:25.848+0000 7f2bd7fff640 1 -- 172.21.15.106:0/3080420946 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be8019070 con 0x7f2bf01570d0 2024-04-25T10:30:25.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:25.848+0000 7f2bd7fff640 1 -- 172.21.15.106:0/3080420946 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2be800cba0 con 0x7f2bf01570d0 2024-04-25T10:30:25.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:25.848+0000 7f2bf59d7640 1 -- 172.21.15.106:0/3080420946 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bf015faa0 con 0x7f2bf01570d0 2024-04-25T10:30:25.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:25.848+0000 7f2bd7fff640 1 -- 172.21.15.106:0/3080420946 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be8014440 con 0x7f2bf01570d0 2024-04-25T10:30:25.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:25.848+0000 7f2bf59d7640 1 -- 172.21.15.106:0/3080420946 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bf0160000 con 0x7f2bf01570d0 2024-04-25T10:30:25.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:25.849+0000 7f2bf59d7640 1 -- 172.21.15.106:0/3080420946 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bf006a3d0 con 0x7f2bf01570d0 2024-04-25T10:30:25.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:25.850+0000 7f2bd7fff640 1 -- 172.21.15.106:0/3080420946 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2be801c070 con 0x7f2bf01570d0 2024-04-25T10:30:25.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:25.851+0000 7f2bd7fff640 1 --2- 172.21.15.106:0/3080420946 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f2bdc0744b0 0x7f2bdc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:25.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:25.851+0000 7f2beeffd640 1 --2- 172.21.15.106:0/3080420946 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f2bdc0744b0 0x7f2bdc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:25.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:25.851+0000 7f2bd7fff640 1 -- 172.21.15.106:0/3080420946 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f2be8094a90 con 0x7f2bf01570d0 2024-04-25T10:30:25.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:25.851+0000 7f2beeffd640 1 --2- 172.21.15.106:0/3080420946 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f2bdc0744b0 0x7f2bdc0769c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2be4000f60 tx=0x7f2be4007c60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:25.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:25.855+0000 7f2bd7fff640 1 -- 172.21.15.106:0/3080420946 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2be805f690 con 0x7f2bf01570d0 2024-04-25T10:30:26.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.033+0000 7f2bf59d7640 1 -- 172.21.15.106:0/3080420946 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2bf0157b40 con 0x7f2bf01570d0 2024-04-25T10:30:26.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.033+0000 7f2bd7fff640 1 -- 172.21.15.106:0/3080420946 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12920 (secure 0 0 0) 0x7f2bf0157b40 con 0x7f2bf01570d0 2024-04-25T10:30:26.036 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:30:26.036 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":16,"fsid":"3ee77985-e8e4-41cf-81e9-7c774cf7891f","created":"2024-04-25T10:30:07.981054+0000","modified":"2024-04-25T10:30:22.366852+0000","last_up_change":"2024-04-25T10:30:19.184325+0000","last_in_change":"2024-04-25T10:30:10.217553+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:30:20.111046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a5962cb1-dbbd-47b1-8d15-9bae114bfdf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1240717079},{"type":"v1","addr":"172.21.15.106:6801","nonce":1240717079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1240717079},{"type":"v1","addr":"172.21.15.106:6803","nonce":1240717079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1240717079},{"type":"v1","addr":"172.21.15.106:6807","nonce":1240717079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1240717079},{"type":"v1","addr":"172.21.15.106:6805","nonce":1240717079}]},"public_addr":"172.21.15.106:6801/1240717079","cluster_addr":"172.21.15.106:6803/1240717079","heartbeat_back_addr":"172.21.15.106:6807/1240717079","heartbeat_front_addr":"172.21.15.106:6805/1240717079","state":["exists","up"]},{"osd":1,"uuid":"94436e13-4769-4752-8397-b5e396a6f7ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4089653051},{"type":"v1","addr":"172.21.15.106:6825","nonce":4089653051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4089653051},{"type":"v1","addr":"172.21.15.106:6827","nonce":4089653051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4089653051},{"type":"v1","addr":"172.21.15.106:6831","nonce":4089653051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4089653051},{"type":"v1","addr":"172.21.15.106:6829","nonce":4089653051}]},"public_addr":"172.21.15.106:6825/4089653051","cluster_addr":"172.21.15.106:6827/4089653051","heartbeat_back_addr":"172.21.15.106:6831/4089653051","heartbeat_front_addr":"172.21.15.106:6829/4089653051","state":["exists","up"]},{"osd":2,"uuid":"429aae0b-70c6-4e7b-9043-6a11334c7efc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":1441018239},{"type":"v1","addr":"172.21.15.106:6809","nonce":1441018239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1441018239},{"type":"v1","addr":"172.21.15.106:6811","nonce":1441018239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1441018239},{"type":"v1","addr":"172.21.15.106:6815","nonce":1441018239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":1441018239},{"type":"v1","addr":"172.21.15.106:6813","nonce":1441018239}]},"public_addr":"172.21.15.106:6809/1441018239","cluster_addr":"172.21.15.106:6811/1441018239","heartbeat_back_addr":"172.21.15.106:6815/1441018239","heartbeat_front_addr":"172.21.15.106:6813/1441018239","state":["exists","up"]},{"osd":3,"uuid":"9593a4d1-772f-4667-ab0d-2982f841e342","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":1846968711},{"type":"v1","addr":"172.21.15.106:6817","nonce":1846968711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1846968711},{"type":"v1","addr":"172.21.15.106:6819","nonce":1846968711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":1846968711},{"type":"v1","addr":"172.21.15.106:6823","nonce":1846968711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":1846968711},{"type":"v1","addr":"172.21.15.106:6821","nonce":1846968711}]},"public_addr":"172.21.15.106:6817/1846968711","cluster_addr":"172.21.15.106:6819/1846968711","heartbeat_back_addr":"172.21.15.106:6823/1846968711","heartbeat_front_addr":"172.21.15.106:6821/1846968711","state":["exists","up"]},{"osd":4,"uuid":"1b61f16d-5c60-480a-b449-c745aa3f6b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":729224739},{"type":"v1","addr":"172.21.15.134:6825","nonce":729224739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":729224739},{"type":"v1","addr":"172.21.15.134:6827","nonce":729224739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":729224739},{"type":"v1","addr":"172.21.15.134:6831","nonce":729224739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":729224739},{"type":"v1","addr":"172.21.15.134:6829","nonce":729224739}]},"public_addr":"172.21.15.134:6825/729224739","cluster_addr":"172.21.15.134:6827/729224739","heartbeat_back_addr":"172.21.15.134:6831/729224739","heartbeat_front_addr":"172.21.15.134:6829/729224739","state":["exists","up"]},{"osd":5,"uuid":"944cdd7f-47c0-4b71-b7e0-d093542f4528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3132275652},{"type":"v1","addr":"172.21.15.134:6809","nonce":3132275652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3132275652},{"type":"v1","addr":"172.21.15.134:6811","nonce":3132275652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3132275652},{"type":"v1","addr":"172.21.15.134:6815","nonce":3132275652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3132275652},{"type":"v1","addr":"172.21.15.134:6813","nonce":3132275652}]},"public_addr":"172.21.15.134:6809/3132275652","cluster_addr":"172.21.15.134:6811/3132275652","heartbeat_back_addr":"172.21.15.134:6815/3132275652","heartbeat_front_addr":"172.21.15.134:6813/3132275652","state":["exists","up"]},{"osd":6,"uuid":"cc2446e2-1d21-4135-ac04-6af7c990058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":992702668},{"type":"v1","addr":"172.21.15.134:6801","nonce":992702668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":992702668},{"type":"v1","addr":"172.21.15.134:6803","nonce":992702668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":992702668},{"type":"v1","addr":"172.21.15.134:6807","nonce":992702668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":992702668},{"type":"v1","addr":"172.21.15.134:6805","nonce":992702668}]},"public_addr":"172.21.15.134:6801/992702668","cluster_addr":"172.21.15.134:6803/992702668","heartbeat_back_addr":"172.21.15.134:6807/992702668","heartbeat_front_addr":"172.21.15.134:6805/992702668","state":["exists","up"]},{"osd":7,"uuid":"3df1fc7a-82fc-4257-8629-98027b6e2aed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":3291062284},{"type":"v1","addr":"172.21.15.134:6817","nonce":3291062284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":3291062284},{"type":"v1","addr":"172.21.15.134:6819","nonce":3291062284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":3291062284},{"type":"v1","addr":"172.21.15.134:6823","nonce":3291062284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":3291062284},{"type":"v1","addr":"172.21.15.134:6821","nonce":3291062284}]},"public_addr":"172.21.15.134:6817/3291062284","cluster_addr":"172.21.15.134:6819/3291062284","heartbeat_back_addr":"172.21.15.134:6823/3291062284","heartbeat_front_addr":"172.21.15.134:6821/3291062284","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.403941+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.382563+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.411544+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.330444+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.405430+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.387737+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.408922+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.454450+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:30:26.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.035+0000 7f2bf59d7640 1 -- 172.21.15.106:0/3080420946 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f2bdc0744b0 msgr2=0x7f2bdc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:26.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.035+0000 7f2bf59d7640 1 --2- 172.21.15.106:0/3080420946 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f2bdc0744b0 0x7f2bdc0769c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2be4000f60 tx=0x7f2be4007c60 comp rx=0 tx=0).stop 2024-04-25T10:30:26.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.035+0000 7f2bf59d7640 1 -- 172.21.15.106:0/3080420946 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2bf01570d0 msgr2=0x7f2bf0159dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:26.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.035+0000 7f2bf59d7640 1 --2- 172.21.15.106:0/3080420946 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2bf01570d0 0x7f2bf0159dd0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2be800b9a0 tx=0x7f2be800be60 comp rx=0 tx=0).stop 2024-04-25T10:30:26.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.035+0000 7f2bf59d7640 1 -- 172.21.15.106:0/3080420946 shutdown_connections 2024-04-25T10:30:26.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.036+0000 7f2bf59d7640 1 --2- 172.21.15.106:0/3080420946 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f2bdc0744b0 0x7f2bdc0769c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:26.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.036+0000 7f2bf59d7640 1 --2- 172.21.15.106:0/3080420946 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2bf01570d0 0x7f2bf0159dd0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:26.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.036+0000 7f2bf59d7640 1 --2- 172.21.15.106:0/3080420946 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bf00c6250 0x7f2bf0150d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:26.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.036+0000 7f2bf59d7640 1 --2- 172.21.15.106:0/3080420946 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2bf006f390 0x7f2bf015de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:26.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.036+0000 7f2bf59d7640 1 -- 172.21.15.106:0/3080420946 >> 172.21.15.106:0/3080420946 conn(0x7f2bf00cff60 msgr2=0x7f2bf0075c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:26.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.036+0000 7f2bf59d7640 1 -- 172.21.15.106:0/3080420946 shutdown_connections 2024-04-25T10:30:26.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.036+0000 7f2bf59d7640 1 -- 172.21.15.106:0/3080420946 wait complete. 2024-04-25T10:30:26.049 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:30:20.111046+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:30:26.050 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:30:26.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.158+0000 7fcc4719f640 1 Processor -- start 2024-04-25T10:30:26.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.158+0000 7fcc4719f640 1 -- start start 2024-04-25T10:30:26.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.160+0000 7fcc4719f640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fcc400c75d0 0x7fcc400ce350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:26.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.160+0000 7fcc4719f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcc4015d530 0x7fcc400d1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:26.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.160+0000 7fcc4719f640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc400c7b50 con 0x7fcc40156e60 2024-04-25T10:30:26.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.160+0000 7fcc4719f640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fcc400cd430 con 0x7fcc400c75d0 2024-04-25T10:30:26.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.160+0000 7fcc4719f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fcc4015d940 con 0x7fcc4015d530 2024-04-25T10:30:26.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.160+0000 7fcc4619d640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fcc40156e60 0x7fcc400ccd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:51256/0 (socket says 172.21.15.106:51256) 2024-04-25T10:30:26.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.160+0000 7fcc4599c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fcc400c75d0 0x7fcc400ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:26.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.160+0000 7fcc4619d640 1 -- 172.21.15.106:0/1077516072 learned_addr learned my addr 172.21.15.106:0/1077516072 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:30:26.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.160+0000 7fcc4519b640 1 -- 172.21.15.106:0/1077516072 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1059246441 0 0) 0x7fcc400c7b50 con 0x7fcc40156e60 2024-04-25T10:30:26.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.160+0000 7fcc4519b640 1 -- 172.21.15.106:0/1077516072 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcc2c003520 con 0x7fcc40156e60 2024-04-25T10:30:26.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.160+0000 7fcc4699e640 1 --2- 172.21.15.106:0/1077516072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcc4015d530 0x7fcc400d1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:26.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.160+0000 7fcc4599c640 1 -- 172.21.15.106:0/1077516072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcc4015d530 msgr2=0x7fcc400d1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:26.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.160+0000 7fcc4599c640 1 --2- 172.21.15.106:0/1077516072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcc4015d530 0x7fcc400d1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:26.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.160+0000 7fcc4599c640 1 -- 172.21.15.106:0/1077516072 >> v1:172.21.15.106:6789/0 conn(0x7fcc40156e60 legacy=0x7fcc400ccd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:26.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.160+0000 7fcc4599c640 1 -- 172.21.15.106:0/1077516072 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc400c7df0 con 0x7fcc400c75d0 2024-04-25T10:30:26.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.160+0000 7fcc4699e640 1 --2- 172.21.15.106:0/1077516072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcc4015d530 0x7fcc400d1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:26.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.161+0000 7fcc4599c640 1 --2- 172.21.15.106:0/1077516072 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc400c75d0 0x7fcc400ce350 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fcc300097a0 tx=0x7fcc3002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=549e640382f810e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:26.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.161+0000 7fcc4519b640 1 -- 172.21.15.106:0/1077516072 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc30037070 con 0x7fcc400c75d0 2024-04-25T10:30:26.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.161+0000 7fcc4519b640 1 -- 172.21.15.106:0/1077516072 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc30032440 con 0x7fcc400c75d0 2024-04-25T10:30:26.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.161+0000 7fcc4519b640 1 -- 172.21.15.106:0/1077516072 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc30032810 con 0x7fcc400c75d0 2024-04-25T10:30:26.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.162+0000 7fcc4719f640 1 -- 172.21.15.106:0/1077516072 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc400c75d0 msgr2=0x7fcc400ce350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:26.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.162+0000 7fcc4719f640 1 --2- 172.21.15.106:0/1077516072 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc400c75d0 0x7fcc400ce350 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fcc300097a0 tx=0x7fcc3002a210 comp rx=0 tx=0).stop 2024-04-25T10:30:26.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.162+0000 7fcc4719f640 1 -- 172.21.15.106:0/1077516072 shutdown_connections 2024-04-25T10:30:26.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.162+0000 7fcc4719f640 1 --2- 172.21.15.106:0/1077516072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcc4015d530 0x7fcc400d1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:26.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.162+0000 7fcc4719f640 1 --2- 172.21.15.106:0/1077516072 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc400c75d0 0x7fcc400ce350 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:26.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.162+0000 7fcc4719f640 1 -- 172.21.15.106:0/1077516072 >> 172.21.15.106:0/1077516072 conn(0x7fcc40073cf0 msgr2=0x7fcc4006d540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:26.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.162+0000 7fcc4719f640 1 -- 172.21.15.106:0/1077516072 shutdown_connections 2024-04-25T10:30:26.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.162+0000 7fcc4719f640 1 -- 172.21.15.106:0/1077516072 wait complete. 2024-04-25T10:30:26.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.163+0000 7fcc4719f640 1 Processor -- start 2024-04-25T10:30:26.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.163+0000 7fcc4719f640 1 -- start start 2024-04-25T10:30:26.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.165+0000 7fcc4719f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc400c75d0 0x7fcc401f6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:26.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.165+0000 7fcc4719f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc40156e60 0x7fcc4014a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:26.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.165+0000 7fcc4719f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcc4015d530 0x7fcc401546f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:26.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.165+0000 7fcc4719f640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fcc4015cd30 con 0x7fcc40156e60 2024-04-25T10:30:26.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.165+0000 7fcc4719f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcc400c83b0 con 0x7fcc400c75d0 2024-04-25T10:30:26.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.165+0000 7fcc4719f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fcc4015d010 con 0x7fcc4015d530 2024-04-25T10:30:26.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.165+0000 7fcc4599c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc40156e60 0x7fcc4014a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:26.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.165+0000 7fcc4599c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc40156e60 0x7fcc4014a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57882/0 (socket says 172.21.15.106:57882) 2024-04-25T10:30:26.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.165+0000 7fcc4599c640 1 -- 172.21.15.106:0/44405534 learned_addr learned my addr 172.21.15.106:0/44405534 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:26.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.165+0000 7fcc4699e640 1 --2- 172.21.15.106:0/44405534 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcc4015d530 0x7fcc401546f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:26.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.165+0000 7fcc4619d640 1 --2- 172.21.15.106:0/44405534 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc400c75d0 0x7fcc401f6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:26.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.165+0000 7fcc4599c640 1 -- 172.21.15.106:0/44405534 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcc4015d530 msgr2=0x7fcc401546f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:26.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.165+0000 7fcc4599c640 1 --2- 172.21.15.106:0/44405534 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcc4015d530 0x7fcc401546f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:26.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.165+0000 7fcc4599c640 1 -- 172.21.15.106:0/44405534 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc400c75d0 msgr2=0x7fcc401f6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:26.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.165+0000 7fcc4599c640 1 --2- 172.21.15.106:0/44405534 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc400c75d0 0x7fcc401f6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:26.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.165+0000 7fcc4599c640 1 -- 172.21.15.106:0/44405534 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc401f6bd0 con 0x7fcc40156e60 2024-04-25T10:30:26.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.165+0000 7fcc4599c640 1 --2- 172.21.15.106:0/44405534 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc40156e60 0x7fcc4014a6a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcc30035040 tx=0x7fcc300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:26.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.165+0000 7fcc36ffd640 1 -- 172.21.15.106:0/44405534 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc30037040 con 0x7fcc40156e60 2024-04-25T10:30:26.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.165+0000 7fcc36ffd640 1 -- 172.21.15.106:0/44405534 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc30030500 con 0x7fcc40156e60 2024-04-25T10:30:26.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.165+0000 7fcc4719f640 1 -- 172.21.15.106:0/44405534 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc400d21d0 con 0x7fcc40156e60 2024-04-25T10:30:26.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.165+0000 7fcc36ffd640 1 -- 172.21.15.106:0/44405534 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc30030910 con 0x7fcc40156e60 2024-04-25T10:30:26.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.165+0000 7fcc4719f640 1 -- 172.21.15.106:0/44405534 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc400d2790 con 0x7fcc40156e60 2024-04-25T10:30:26.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.166+0000 7fcc4719f640 1 -- 172.21.15.106:0/44405534 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc40050740 con 0x7fcc40156e60 2024-04-25T10:30:26.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.167+0000 7fcc36ffd640 1 -- 172.21.15.106:0/44405534 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fcc30044020 con 0x7fcc40156e60 2024-04-25T10:30:26.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.168+0000 7fcc36ffd640 1 --2- 172.21.15.106:0/44405534 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fcc240744e0 0x7fcc240769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:26.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.168+0000 7fcc4619d640 1 --2- 172.21.15.106:0/44405534 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fcc240744e0 0x7fcc240769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:26.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.168+0000 7fcc36ffd640 1 -- 172.21.15.106:0/44405534 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fcc300b4e60 con 0x7fcc40156e60 2024-04-25T10:30:26.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.168+0000 7fcc4619d640 1 --2- 172.21.15.106:0/44405534 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fcc240744e0 0x7fcc240769f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c001580 tx=0x7fcc3c000f40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:26.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.172+0000 7fcc36ffd640 1 -- 172.21.15.106:0/44405534 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcc3007f2f0 con 0x7fcc40156e60 2024-04-25T10:30:26.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.346+0000 7fcc4719f640 1 -- 172.21.15.106:0/44405534 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcc40152640 con 0x7fcc40156e60 2024-04-25T10:30:26.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.346+0000 7fcc36ffd640 1 -- 172.21.15.106:0/44405534 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7fcc300835f0 con 0x7fcc40156e60 2024-04-25T10:30:26.348 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-25T10:30:26.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.348+0000 7fcc4719f640 1 -- 172.21.15.106:0/44405534 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fcc240744e0 msgr2=0x7fcc240769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:26.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.348+0000 7fcc4719f640 1 --2- 172.21.15.106:0/44405534 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fcc240744e0 0x7fcc240769f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c001580 tx=0x7fcc3c000f40 comp rx=0 tx=0).stop 2024-04-25T10:30:26.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.348+0000 7fcc4719f640 1 -- 172.21.15.106:0/44405534 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc40156e60 msgr2=0x7fcc4014a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:26.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.348+0000 7fcc4719f640 1 --2- 172.21.15.106:0/44405534 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc40156e60 0x7fcc4014a6a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcc30035040 tx=0x7fcc300046c0 comp rx=0 tx=0).stop 2024-04-25T10:30:26.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.348+0000 7fcc4719f640 1 -- 172.21.15.106:0/44405534 shutdown_connections 2024-04-25T10:30:26.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.348+0000 7fcc4719f640 1 --2- 172.21.15.106:0/44405534 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fcc240744e0 0x7fcc240769f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:26.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.348+0000 7fcc4719f640 1 --2- 172.21.15.106:0/44405534 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcc4015d530 0x7fcc401546f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:26.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.348+0000 7fcc4719f640 1 --2- 172.21.15.106:0/44405534 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc40156e60 0x7fcc4014a6a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:26.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.348+0000 7fcc4719f640 1 --2- 172.21.15.106:0/44405534 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc400c75d0 0x7fcc401f6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:26.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.348+0000 7fcc4719f640 1 -- 172.21.15.106:0/44405534 >> 172.21.15.106:0/44405534 conn(0x7fcc40073cf0 msgr2=0x7fcc400b5760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:26.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.348+0000 7fcc4719f640 1 -- 172.21.15.106:0/44405534 shutdown_connections 2024-04-25T10:30:26.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.348+0000 7fcc4719f640 1 -- 172.21.15.106:0/44405534 wait complete. 2024-04-25T10:30:26.361 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:30:26.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.468+0000 7f56cee53640 1 Processor -- start 2024-04-25T10:30:26.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.468+0000 7f56cee53640 1 -- start start 2024-04-25T10:30:26.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.469+0000 7f56cee53640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f56c8154c60 0x7f56c8070ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:26.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.469+0000 7f56cee53640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56c80caa00 con 0x7f56c8071220 2024-04-25T10:30:26.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.469+0000 7f56cee53640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56c80cac70 con 0x7f56c815e020 2024-04-25T10:30:26.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.469+0000 7f56cee53640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f56c80cb880 con 0x7f56c8154c60 2024-04-25T10:30:26.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.469+0000 7f56c7fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f56c8071220 0x7f56c80c5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:51264/0 (socket says 172.21.15.106:51264) 2024-04-25T10:30:26.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.469+0000 7f56c7fff640 1 -- 172.21.15.106:0/2391120768 learned_addr learned my addr 172.21.15.106:0/2391120768 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:30:26.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.469+0000 7f56ccbef640 1 --2- 172.21.15.106:0/2391120768 >> v2:172.21.15.134:3300/0 conn(0x7f56c8154c60 0x7f56c8070ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:26.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.469+0000 7f56c77fe640 1 -- 172.21.15.106:0/2391120768 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3330653059 0 0) 0x7f56c80caa00 con 0x7f56c8071220 2024-04-25T10:30:26.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.469+0000 7f56c77fe640 1 -- 172.21.15.106:0/2391120768 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f56b4003520 con 0x7f56c8071220 2024-04-25T10:30:26.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.470+0000 7f56c77fe640 1 -- 172.21.15.106:0/2391120768 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1827559052 0 0) 0x7f56b4003520 con 0x7f56c8071220 2024-04-25T10:30:26.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.470+0000 7f56c77fe640 1 -- 172.21.15.106:0/2391120768 >> v2:172.21.15.134:3300/0 conn(0x7f56c8154c60 msgr2=0x7f56c8070ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:26.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.470+0000 7f56c77fe640 1 --2- 172.21.15.106:0/2391120768 >> v2:172.21.15.134:3300/0 conn(0x7f56c8154c60 0x7f56c8070ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:26.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.470+0000 7f56c77fe640 1 -- 172.21.15.106:0/2391120768 >> v1:172.21.15.134:6789/0 conn(0x7f56c815e020 legacy=0x7f56c815e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:26.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.470+0000 7f56c77fe640 1 -- 172.21.15.106:0/2391120768 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56c80caf50 con 0x7f56c8071220 2024-04-25T10:30:26.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.470+0000 7f56c77fe640 1 -- 172.21.15.106:0/2391120768 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3776459350 0 0) 0x7f56b0001670 con 0x7f56c8071220 2024-04-25T10:30:26.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.470+0000 7f56c77fe640 1 -- 172.21.15.106:0/2391120768 >> v1:172.21.15.106:6789/0 conn(0x7f56c8071220 legacy=0x7f56c80c5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:26.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.470+0000 7f56c77fe640 1 --2- 172.21.15.106:0/2391120768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56b40039b0 0x7f56b4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:26.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.470+0000 7f56c77fe640 1 --2- 172.21.15.106:0/2391120768 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f56b4004800 0x7f56b4004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:26.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.470+0000 7f56c77fe640 1 --2- 172.21.15.106:0/2391120768 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56b40051c0 0x7f56b400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:26.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.470+0000 7f56c77fe640 1 -- 172.21.15.106:0/2391120768 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f56c80cb880 con 0x7f56b40051c0 2024-04-25T10:30:26.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.470+0000 7f56c77fe640 1 -- 172.21.15.106:0/2391120768 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f56b400c160 con 0x7f56b40039b0 2024-04-25T10:30:26.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.470+0000 7f56c77fe640 1 -- 172.21.15.106:0/2391120768 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f56b400c360 con 0x7f56b4004800 2024-04-25T10:30:26.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.470+0000 7f56cd3f0640 1 --2- 172.21.15.106:0/2391120768 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56b40051c0 0x7f56b400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:26.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.470+0000 7f56ccbef640 1 --2- 172.21.15.106:0/2391120768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56b40039b0 0x7f56b4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:26.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.470+0000 7f56c7fff640 1 --2- 172.21.15.106:0/2391120768 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f56b4004800 0x7f56b4004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:26.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.470+0000 7f56cd3f0640 1 -- 172.21.15.106:0/2391120768 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f56b4004800 msgr2=0x7f56b4004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:26.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.470+0000 7f56cd3f0640 1 --2- 172.21.15.106:0/2391120768 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f56b4004800 0x7f56b4004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:26.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.470+0000 7f56cd3f0640 1 -- 172.21.15.106:0/2391120768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56b40039b0 msgr2=0x7f56b4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:26.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.470+0000 7f56cd3f0640 1 --2- 172.21.15.106:0/2391120768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56b40039b0 0x7f56b4003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:26.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.470+0000 7f56cd3f0640 1 -- 172.21.15.106:0/2391120768 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56b4003520 con 0x7f56b40051c0 2024-04-25T10:30:26.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.471+0000 7f56cd3f0640 1 -- 172.21.15.106:0/2391120768 reap_dead start 2024-04-25T10:30:26.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.471+0000 7f56cd3f0640 1 --2- 172.21.15.106:0/2391120768 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56b40051c0 0x7f56b400b9c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f56c0001e20 tx=0x7f56b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=100171b9dd9d5fd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:26.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.471+0000 7f56c77fe640 1 -- 172.21.15.106:0/2391120768 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c0001a70 con 0x7f56b40051c0 2024-04-25T10:30:26.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.471+0000 7f56c77fe640 1 -- 172.21.15.106:0/2391120768 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56c002ad70 con 0x7f56b40051c0 2024-04-25T10:30:26.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.471+0000 7f56c77fe640 1 -- 172.21.15.106:0/2391120768 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c0003b90 con 0x7f56b40051c0 2024-04-25T10:30:26.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.472+0000 7f56cee53640 1 -- 172.21.15.106:0/2391120768 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56b40051c0 msgr2=0x7f56b400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:26.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.472+0000 7f56cee53640 1 --2- 172.21.15.106:0/2391120768 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56b40051c0 0x7f56b400b9c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f56c0001e20 tx=0x7f56b4005830 comp rx=0 tx=0).stop 2024-04-25T10:30:26.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.472+0000 7f56cee53640 1 -- 172.21.15.106:0/2391120768 shutdown_connections 2024-04-25T10:30:26.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.472+0000 7f56cee53640 1 --2- 172.21.15.106:0/2391120768 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56b40051c0 0x7f56b400b9c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:26.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.472+0000 7f56cee53640 1 -- 172.21.15.106:0/2391120768 >> 172.21.15.106:0/2391120768 conn(0x7f56c80cff60 msgr2=0x7f56c80d0370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:26.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.472+0000 7f56cee53640 1 -- 172.21.15.106:0/2391120768 shutdown_connections 2024-04-25T10:30:26.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.472+0000 7f56cee53640 1 -- 172.21.15.106:0/2391120768 wait complete. 2024-04-25T10:30:26.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.473+0000 7f56cee53640 1 Processor -- start 2024-04-25T10:30:26.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.473+0000 7f56cee53640 1 -- start start 2024-04-25T10:30:26.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.475+0000 7f56cee53640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56b40051c0 0x7f56c80cde60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:26.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.475+0000 7f56cee53640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56c81526d0 0x7f56c81515b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:26.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.475+0000 7f56cee53640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f56c8155b50 0x7f56c814b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:26.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.475+0000 7f56cee53640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f56c8155070 con 0x7f56c81526d0 2024-04-25T10:30:26.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.475+0000 7f56cee53640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f56c80cb510 con 0x7f56b40051c0 2024-04-25T10:30:26.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.475+0000 7f56cee53640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f56c81551f0 con 0x7f56c8155b50 2024-04-25T10:30:26.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.475+0000 7f56c7fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56c81526d0 0x7f56c81515b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:26.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.475+0000 7f56c7fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56c81526d0 0x7f56c81515b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57902/0 (socket says 172.21.15.106:57902) 2024-04-25T10:30:26.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.475+0000 7f56c7fff640 1 -- 172.21.15.106:0/1671264897 learned_addr learned my addr 172.21.15.106:0/1671264897 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:26.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.475+0000 7f56cd3f0640 1 --2- 172.21.15.106:0/1671264897 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f56c8155b50 0x7f56c814b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:26.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.475+0000 7f56ccbef640 1 --2- 172.21.15.106:0/1671264897 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56b40051c0 0x7f56c80cde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:26.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.475+0000 7f56c7fff640 1 -- 172.21.15.106:0/1671264897 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f56c8155b50 msgr2=0x7f56c814b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:26.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.475+0000 7f56c7fff640 1 --2- 172.21.15.106:0/1671264897 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f56c8155b50 0x7f56c814b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:26.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.475+0000 7f56c7fff640 1 -- 172.21.15.106:0/1671264897 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56b40051c0 msgr2=0x7f56c80cde60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:26.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.475+0000 7f56c7fff640 1 --2- 172.21.15.106:0/1671264897 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56b40051c0 0x7f56c80cde60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:26.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.475+0000 7f56c7fff640 1 -- 172.21.15.106:0/1671264897 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56c80ce3a0 con 0x7f56c81526d0 2024-04-25T10:30:26.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.475+0000 7f56cd3f0640 1 --2- 172.21.15.106:0/1671264897 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f56c8155b50 0x7f56c814b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:26.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.475+0000 7f56c7fff640 1 --2- 172.21.15.106:0/1671264897 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56c81526d0 0x7f56c81515b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f56b00072c0 tx=0x7f56b0007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:26.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.475+0000 7f56c57fa640 1 -- 172.21.15.106:0/1671264897 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b001a070 con 0x7f56c81526d0 2024-04-25T10:30:26.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.475+0000 7f56c57fa640 1 -- 172.21.15.106:0/1671264897 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56b000f960 con 0x7f56c81526d0 2024-04-25T10:30:26.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.475+0000 7f56cee53640 1 -- 172.21.15.106:0/1671264897 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56c8155470 con 0x7f56c81526d0 2024-04-25T10:30:26.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.475+0000 7f56c57fa640 1 -- 172.21.15.106:0/1671264897 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b000fd30 con 0x7f56c81526d0 2024-04-25T10:30:26.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.475+0000 7f56cee53640 1 -- 172.21.15.106:0/1671264897 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56c81f1b70 con 0x7f56c81526d0 2024-04-25T10:30:26.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.476+0000 7f56cee53640 1 -- 172.21.15.106:0/1671264897 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56c80cf2a0 con 0x7f56c81526d0 2024-04-25T10:30:26.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.477+0000 7f56c57fa640 1 -- 172.21.15.106:0/1671264897 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f56b001d070 con 0x7f56c81526d0 2024-04-25T10:30:26.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.477+0000 7f56c57fa640 1 --2- 172.21.15.106:0/1671264897 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f56a80744e0 0x7f56a80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:26.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.477+0000 7f56ccbef640 1 --2- 172.21.15.106:0/1671264897 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f56a80744e0 0x7f56a80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:26.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.477+0000 7f56c57fa640 1 -- 172.21.15.106:0/1671264897 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f56b0094880 con 0x7f56c81526d0 2024-04-25T10:30:26.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.478+0000 7f56ccbef640 1 --2- 172.21.15.106:0/1671264897 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f56a80744e0 0x7f56a80769f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f56b8009a80 tx=0x7f56b8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:26.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.482+0000 7f56c57fa640 1 -- 172.21.15.106:0/1671264897 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f56c80cf2a0 con 0x7f56c81526d0 2024-04-25T10:30:26.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.695+0000 7f56cee53640 1 -- 172.21.15.106:0/1671264897 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f56c806a3d0 con 0x7f56c81526d0 2024-04-25T10:30:26.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.695+0000 7f56c57fa640 1 -- 172.21.15.106:0/1671264897 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1395 (secure 0 0 0) 0x7f56b005f360 con 0x7f56c81526d0 2024-04-25T10:30:26.698 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:30:26.698 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/2834155852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":2834155852},{"type":"v1","addr":"172.21.15.106:6835","nonce":2834155852}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T10:30:26.698 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 2 2024-04-25T10:30:26.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.697+0000 7f56cee53640 1 -- 172.21.15.106:0/1671264897 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f56a80744e0 msgr2=0x7f56a80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:26.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.697+0000 7f56cee53640 1 --2- 172.21.15.106:0/1671264897 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f56a80744e0 0x7f56a80769f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f56b8009a80 tx=0x7f56b8008040 comp rx=0 tx=0).stop 2024-04-25T10:30:26.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.697+0000 7f56cee53640 1 -- 172.21.15.106:0/1671264897 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56c81526d0 msgr2=0x7f56c81515b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:26.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.697+0000 7f56cee53640 1 --2- 172.21.15.106:0/1671264897 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56c81526d0 0x7f56c81515b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f56b00072c0 tx=0x7f56b0007780 comp rx=0 tx=0).stop 2024-04-25T10:30:26.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.698+0000 7f56cee53640 1 -- 172.21.15.106:0/1671264897 shutdown_connections 2024-04-25T10:30:26.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.698+0000 7f56cee53640 1 --2- 172.21.15.106:0/1671264897 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f56a80744e0 0x7f56a80769f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:26.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.698+0000 7f56cee53640 1 --2- 172.21.15.106:0/1671264897 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f56c8155b50 0x7f56c814b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:26.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.698+0000 7f56cee53640 1 --2- 172.21.15.106:0/1671264897 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56c81526d0 0x7f56c81515b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:26.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.698+0000 7f56cee53640 1 --2- 172.21.15.106:0/1671264897 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56b40051c0 0x7f56c80cde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:26.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.698+0000 7f56cee53640 1 -- 172.21.15.106:0/1671264897 >> 172.21.15.106:0/1671264897 conn(0x7f56c80cff60 msgr2=0x7f56c8152b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:26.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.698+0000 7f56cee53640 1 -- 172.21.15.106:0/1671264897 shutdown_connections 2024-04-25T10:30:26.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:26.698+0000 7f56cee53640 1 -- 172.21.15.106:0/1671264897 wait complete. 2024-04-25T10:30:28.712 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:30:28.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:28.837+0000 7f9d7f29f640 1 Processor -- start 2024-04-25T10:30:28.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:28.837+0000 7f9d7f29f640 1 -- start start 2024-04-25T10:30:28.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:28.838+0000 7f9d7f29f640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9d78154c60 0x7f9d78070ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:28.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:28.838+0000 7f9d7f29f640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f9d7815e020 0x7f9d7815e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:28.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:28.838+0000 7f9d7f29f640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d780caa00 con 0x7f9d78071220 2024-04-25T10:30:28.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:28.838+0000 7f9d7f29f640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f9d780cb880 con 0x7f9d78154c60 2024-04-25T10:30:28.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:28.838+0000 7f9d7f29f640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f9d780cac70 con 0x7f9d7815e020 2024-04-25T10:30:28.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:28.838+0000 7f9d7d03b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9d78154c60 0x7f9d78070ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:28.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:28.838+0000 7f9d7d03b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9d78154c60 0x7f9d78070ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57914/0 (socket says 172.21.15.106:57914) 2024-04-25T10:30:28.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:28.838+0000 7f9d7d03b640 1 -- 172.21.15.106:0/3020225182 learned_addr learned my addr 172.21.15.106:0/3020225182 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:28.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:28.839+0000 7f9d7d83c640 1 --2- 172.21.15.106:0/3020225182 >> v2:172.21.15.134:3300/0 conn(0x7f9d7815e020 0x7f9d7815e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:28.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:28.839+0000 7f9d7d03b640 1 -- 172.21.15.106:0/3020225182 >> v2:172.21.15.134:3300/0 conn(0x7f9d7815e020 msgr2=0x7f9d7815e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:28.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:28.839+0000 7f9d7d03b640 1 --2- 172.21.15.106:0/3020225182 >> v2:172.21.15.134:3300/0 conn(0x7f9d7815e020 0x7f9d7815e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:28.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:28.839+0000 7f9d7d03b640 1 -- 172.21.15.106:0/3020225182 >> v1:172.21.15.134:6789/0 conn(0x7f9d78071220 legacy=0x7f9d780c5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:28.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:28.839+0000 7f9d7d03b640 1 -- 172.21.15.106:0/3020225182 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d780caf50 con 0x7f9d78154c60 2024-04-25T10:30:28.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:28.840+0000 7f9d7d03b640 1 --2- 172.21.15.106:0/3020225182 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d78154c60 0x7f9d78070ce0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9d78074a40 tx=0x7f9d7402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57d79702b6999092 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:28.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:28.840+0000 7f9d6ffff640 1 -- 172.21.15.106:0/3020225182 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d74037070 con 0x7f9d78154c60 2024-04-25T10:30:28.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:28.840+0000 7f9d6ffff640 1 -- 172.21.15.106:0/3020225182 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d74032440 con 0x7f9d78154c60 2024-04-25T10:30:28.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:28.840+0000 7f9d6ffff640 1 -- 172.21.15.106:0/3020225182 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d74032810 con 0x7f9d78154c60 2024-04-25T10:30:28.844 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:28.840+0000 7f9d7f29f640 1 -- 172.21.15.106:0/3020225182 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d78154c60 msgr2=0x7f9d78070ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:28.844 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:28.840+0000 7f9d7f29f640 1 --2- 172.21.15.106:0/3020225182 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d78154c60 0x7f9d78070ce0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9d78074a40 tx=0x7f9d7402a270 comp rx=0 tx=0).stop 2024-04-25T10:30:28.844 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:28.840+0000 7f9d7f29f640 1 -- 172.21.15.106:0/3020225182 shutdown_connections 2024-04-25T10:30:28.844 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:28.841+0000 7f9d7f29f640 1 --2- 172.21.15.106:0/3020225182 >> v2:172.21.15.134:3300/0 conn(0x7f9d7815e020 0x7f9d7815e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:28.844 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:28.841+0000 7f9d7f29f640 1 --2- 172.21.15.106:0/3020225182 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d78154c60 0x7f9d78070ce0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:28.844 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:28.841+0000 7f9d7f29f640 1 -- 172.21.15.106:0/3020225182 >> 172.21.15.106:0/3020225182 conn(0x7f9d780cff60 msgr2=0x7f9d780d0370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:28.844 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:28.841+0000 7f9d7f29f640 1 -- 172.21.15.106:0/3020225182 shutdown_connections 2024-04-25T10:30:28.844 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:28.841+0000 7f9d7f29f640 1 -- 172.21.15.106:0/3020225182 wait complete. 2024-04-25T10:30:28.845 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:28.842+0000 7f9d7f29f640 1 Processor -- start 2024-04-25T10:30:28.845 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:28.842+0000 7f9d7f29f640 1 -- start start 2024-04-25T10:30:28.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:28.844+0000 7f9d7f29f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d78071220 0x7f9d781f4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:28.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:28.844+0000 7f9d7f29f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9d78154c60 0x7f9d7814fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:28.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:28.844+0000 7f9d7f29f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d7815e020 0x7f9d7815d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:28.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:28.844+0000 7f9d7f29f640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9d780c5b10 con 0x7f9d78071220 2024-04-25T10:30:28.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:28.844+0000 7f9d7f29f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9d780cb510 con 0x7f9d7815e020 2024-04-25T10:30:28.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:28.844+0000 7f9d7f29f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f9d780c5c90 con 0x7f9d78154c60 2024-04-25T10:30:28.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:28.844+0000 7f9d7d03b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d78071220 0x7f9d781f4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:28.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:28.844+0000 7f9d7d03b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d78071220 0x7f9d781f4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57922/0 (socket says 172.21.15.106:57922) 2024-04-25T10:30:28.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:28.844+0000 7f9d7d03b640 1 -- 172.21.15.106:0/505629806 learned_addr learned my addr 172.21.15.106:0/505629806 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:28.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:28.844+0000 7f9d7d83c640 1 --2- 172.21.15.106:0/505629806 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d7815e020 0x7f9d7815d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:28.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:28.844+0000 7f9d7c83a640 1 --2- 172.21.15.106:0/505629806 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9d78154c60 0x7f9d7814fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:28.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:28.844+0000 7f9d7d03b640 1 -- 172.21.15.106:0/505629806 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9d78154c60 msgr2=0x7f9d7814fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:28.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:28.844+0000 7f9d7d03b640 1 --2- 172.21.15.106:0/505629806 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9d78154c60 0x7f9d7814fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:28.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:28.844+0000 7f9d7d03b640 1 -- 172.21.15.106:0/505629806 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d7815e020 msgr2=0x7f9d7815d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:28.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:28.844+0000 7f9d7d03b640 1 --2- 172.21.15.106:0/505629806 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d7815e020 0x7f9d7815d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:28.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:28.844+0000 7f9d7d03b640 1 -- 172.21.15.106:0/505629806 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d781f5010 con 0x7f9d78071220 2024-04-25T10:30:28.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:28.844+0000 7f9d7d83c640 1 --2- 172.21.15.106:0/505629806 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d7815e020 0x7f9d7815d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:28.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:28.844+0000 7f9d7c83a640 1 --2- 172.21.15.106:0/505629806 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9d78154c60 0x7f9d7814fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:28.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:28.844+0000 7f9d7d03b640 1 --2- 172.21.15.106:0/505629806 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d78071220 0x7f9d781f4ad0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9d7402a2b0 tx=0x7f9d740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:28.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:28.844+0000 7f9d6dffb640 1 -- 172.21.15.106:0/505629806 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d74041020 con 0x7f9d78071220 2024-04-25T10:30:28.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:28.844+0000 7f9d6dffb640 1 -- 172.21.15.106:0/505629806 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d740306c0 con 0x7f9d78071220 2024-04-25T10:30:28.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:28.844+0000 7f9d7f29f640 1 -- 172.21.15.106:0/505629806 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d780cdfb0 con 0x7f9d78071220 2024-04-25T10:30:28.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:28.844+0000 7f9d6dffb640 1 -- 172.21.15.106:0/505629806 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d74037040 con 0x7f9d78071220 2024-04-25T10:30:28.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:28.845+0000 7f9d7f29f640 1 -- 172.21.15.106:0/505629806 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d781f42d0 con 0x7f9d78071220 2024-04-25T10:30:28.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:28.846+0000 7f9d7f29f640 1 -- 172.21.15.106:0/505629806 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d7806a3d0 con 0x7f9d78071220 2024-04-25T10:30:28.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:28.846+0000 7f9d6dffb640 1 -- 172.21.15.106:0/505629806 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9d74044020 con 0x7f9d78071220 2024-04-25T10:30:28.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:28.847+0000 7f9d6dffb640 1 --2- 172.21.15.106:0/505629806 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f9d5c0744b0 0x7f9d5c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:28.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:28.847+0000 7f9d7c83a640 1 --2- 172.21.15.106:0/505629806 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f9d5c0744b0 0x7f9d5c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:28.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:28.847+0000 7f9d6dffb640 1 -- 172.21.15.106:0/505629806 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f9d740b5700 con 0x7f9d78071220 2024-04-25T10:30:28.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:28.847+0000 7f9d7c83a640 1 --2- 172.21.15.106:0/505629806 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f9d5c0744b0 0x7f9d5c0769c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9d781504a0 tx=0x7f9d68009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:28.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:28.852+0000 7f9d6dffb640 1 -- 172.21.15.106:0/505629806 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9d7404b030 con 0x7f9d78071220 2024-04-25T10:30:29.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.063+0000 7f9d7f29f640 1 -- 172.21.15.106:0/505629806 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9d780cbe80 con 0x7f9d78071220 2024-04-25T10:30:29.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.064+0000 7f9d6dffb640 1 -- 172.21.15.106:0/505629806 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+5527 (secure 0 0 0) 0x7f9d74080270 con 0x7f9d78071220 2024-04-25T10:30:29.066 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:30:29.067 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/2834155852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":2834155852},{"type":"v1","addr":"172.21.15.106:6835","nonce":2834155852}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/2632121774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":2632121774},{"type":"v1","addr":"172.21.15.106:6837","nonce":2632121774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4269,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6839/486339834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":486339834},{"type":"v1","addr":"172.21.15.106:6839","nonce":486339834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4284,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/2781656864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6833","nonce":2781656864},{"type":"v1","addr":"172.21.15.134:6836","nonce":2781656864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4287,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/1993918351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1993918351},{"type":"v1","addr":"172.21.15.134:6835","nonce":1993918351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4290,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/2662003420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2662003420},{"type":"v1","addr":"172.21.15.134:6837","nonce":2662003420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T10:30:29.067 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 3 2024-04-25T10:30:29.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.066+0000 7f9d7f29f640 1 -- 172.21.15.106:0/505629806 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f9d5c0744b0 msgr2=0x7f9d5c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:29.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.066+0000 7f9d7f29f640 1 --2- 172.21.15.106:0/505629806 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f9d5c0744b0 0x7f9d5c0769c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9d781504a0 tx=0x7f9d68009040 comp rx=0 tx=0).stop 2024-04-25T10:30:29.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.066+0000 7f9d7f29f640 1 -- 172.21.15.106:0/505629806 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d78071220 msgr2=0x7f9d781f4ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:29.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.066+0000 7f9d7f29f640 1 --2- 172.21.15.106:0/505629806 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d78071220 0x7f9d781f4ad0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9d7402a2b0 tx=0x7f9d740046c0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.066+0000 7f9d7f29f640 1 -- 172.21.15.106:0/505629806 shutdown_connections 2024-04-25T10:30:29.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.066+0000 7f9d7f29f640 1 --2- 172.21.15.106:0/505629806 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f9d5c0744b0 0x7f9d5c0769c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.066+0000 7f9d7f29f640 1 --2- 172.21.15.106:0/505629806 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d7815e020 0x7f9d7815d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.066+0000 7f9d7f29f640 1 --2- 172.21.15.106:0/505629806 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9d78154c60 0x7f9d7814fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.066+0000 7f9d7f29f640 1 --2- 172.21.15.106:0/505629806 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d78071220 0x7f9d781f4ad0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.066+0000 7f9d7f29f640 1 -- 172.21.15.106:0/505629806 >> 172.21.15.106:0/505629806 conn(0x7f9d780cff60 msgr2=0x7f9d781521c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:29.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.066+0000 7f9d7f29f640 1 -- 172.21.15.106:0/505629806 shutdown_connections 2024-04-25T10:30:29.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.066+0000 7f9d7f29f640 1 -- 172.21.15.106:0/505629806 wait complete. 2024-04-25T10:30:29.079 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:30:29.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.184+0000 7f9bc31b7640 1 Processor -- start 2024-04-25T10:30:29.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.184+0000 7f9bc31b7640 1 -- start start 2024-04-25T10:30:29.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.185+0000 7f9bc31b7640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9bbc06dc00 0x7f9bbc156f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:29.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.185+0000 7f9bc31b7640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f9bbc15d690 0x7f9bbc0cce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:29.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.185+0000 7f9bc31b7640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bbc15cbd0 con 0x7f9bbc0c61a0 2024-04-25T10:30:29.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.185+0000 7f9bc31b7640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f9bbc15d200 con 0x7f9bbc06dc00 2024-04-25T10:30:29.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.185+0000 7f9bc31b7640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f9bbc160990 con 0x7f9bbc15d690 2024-04-25T10:30:29.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.185+0000 7f9bbbfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9bbc06dc00 0x7f9bbc156f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:29.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.185+0000 7f9bc0f53640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f9bbc0c61a0 0x7f9bbc1601c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:51266/0 (socket says 172.21.15.106:51266) 2024-04-25T10:30:29.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.185+0000 7f9bc0f53640 1 -- 172.21.15.106:0/1067084143 learned_addr learned my addr 172.21.15.106:0/1067084143 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:30:29.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.185+0000 7f9bc1754640 1 --2- 172.21.15.106:0/1067084143 >> v2:172.21.15.134:3300/0 conn(0x7f9bbc15d690 0x7f9bbc0cce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:29.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.186+0000 7f9bbbfff640 1 -- 172.21.15.106:0/1067084143 >> v2:172.21.15.134:3300/0 conn(0x7f9bbc15d690 msgr2=0x7f9bbc0cce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:29.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.186+0000 7f9bbbfff640 1 --2- 172.21.15.106:0/1067084143 >> v2:172.21.15.134:3300/0 conn(0x7f9bbc15d690 0x7f9bbc0cce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.186+0000 7f9bbbfff640 1 -- 172.21.15.106:0/1067084143 >> v1:172.21.15.106:6789/0 conn(0x7f9bbc0c61a0 legacy=0x7f9bbc1601c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:29.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.186+0000 7f9bbb7fe640 1 -- 172.21.15.106:0/1067084143 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2000569907 0 0) 0x7f9bbc15cbd0 con 0x7f9bbc0c61a0 2024-04-25T10:30:29.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.186+0000 7f9bbbfff640 1 -- 172.21.15.106:0/1067084143 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bbc160c70 con 0x7f9bbc06dc00 2024-04-25T10:30:29.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.186+0000 7f9bbbfff640 1 --2- 172.21.15.106:0/1067084143 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9bbc06dc00 0x7f9bbc156f30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc157470 tx=0x7f9ba802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a86f43741fb0957e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:29.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.186+0000 7f9bbb7fe640 1 -- 172.21.15.106:0/1067084143 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba8037070 con 0x7f9bbc06dc00 2024-04-25T10:30:29.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.186+0000 7f9bbb7fe640 1 -- 172.21.15.106:0/1067084143 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ba8032440 con 0x7f9bbc06dc00 2024-04-25T10:30:29.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.186+0000 7f9bbb7fe640 1 -- 172.21.15.106:0/1067084143 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba8032810 con 0x7f9bbc06dc00 2024-04-25T10:30:29.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.187+0000 7f9bc31b7640 1 -- 172.21.15.106:0/1067084143 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9bbc06dc00 msgr2=0x7f9bbc156f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:29.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.187+0000 7f9bc31b7640 1 --2- 172.21.15.106:0/1067084143 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9bbc06dc00 0x7f9bbc156f30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc157470 tx=0x7f9ba802a1a0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.187+0000 7f9bc31b7640 1 -- 172.21.15.106:0/1067084143 shutdown_connections 2024-04-25T10:30:29.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.187+0000 7f9bc31b7640 1 --2- 172.21.15.106:0/1067084143 >> v2:172.21.15.134:3300/0 conn(0x7f9bbc15d690 0x7f9bbc0cce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.187+0000 7f9bc31b7640 1 --2- 172.21.15.106:0/1067084143 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9bbc06dc00 0x7f9bbc156f30 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.187+0000 7f9bc31b7640 1 -- 172.21.15.106:0/1067084143 >> 172.21.15.106:0/1067084143 conn(0x7f9bbc0727d0 msgr2=0x7f9bbc072c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:29.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.187+0000 7f9bc31b7640 1 -- 172.21.15.106:0/1067084143 shutdown_connections 2024-04-25T10:30:29.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.187+0000 7f9bc31b7640 1 -- 172.21.15.106:0/1067084143 wait complete. 2024-04-25T10:30:29.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.189+0000 7f9bc31b7640 1 Processor -- start 2024-04-25T10:30:29.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.189+0000 7f9bc31b7640 1 -- start start 2024-04-25T10:30:29.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.190+0000 7f9bc31b7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9bbc06dc00 0x7f9bbc1f6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:29.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.190+0000 7f9bc31b7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9bbc0c61a0 0x7f9bbc15f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:29.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.190+0000 7f9bc31b7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9bbc15d690 0x7f9bbc155960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:29.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.190+0000 7f9bc31b7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9bbc15daa0 con 0x7f9bbc0c61a0 2024-04-25T10:30:29.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.190+0000 7f9bc31b7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9bbc15ce90 con 0x7f9bbc06dc00 2024-04-25T10:30:29.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.190+0000 7f9bc31b7640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f9bbc06d6d0 con 0x7f9bbc15d690 2024-04-25T10:30:29.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.190+0000 7f9bbbfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9bbc0c61a0 0x7f9bbc15f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:29.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.190+0000 7f9bbbfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9bbc0c61a0 0x7f9bbc15f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57938/0 (socket says 172.21.15.106:57938) 2024-04-25T10:30:29.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.190+0000 7f9bbbfff640 1 -- 172.21.15.106:0/2988199138 learned_addr learned my addr 172.21.15.106:0/2988199138 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:29.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.191+0000 7f9bc0f53640 1 --2- 172.21.15.106:0/2988199138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9bbc06dc00 0x7f9bbc1f6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:29.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.191+0000 7f9bc1754640 1 --2- 172.21.15.106:0/2988199138 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9bbc15d690 0x7f9bbc155960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:29.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.191+0000 7f9bbbfff640 1 -- 172.21.15.106:0/2988199138 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9bbc15d690 msgr2=0x7f9bbc155960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:29.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.191+0000 7f9bbbfff640 1 --2- 172.21.15.106:0/2988199138 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9bbc15d690 0x7f9bbc155960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.191+0000 7f9bbbfff640 1 -- 172.21.15.106:0/2988199138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9bbc06dc00 msgr2=0x7f9bbc1f6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:29.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.191+0000 7f9bbbfff640 1 --2- 172.21.15.106:0/2988199138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9bbc06dc00 0x7f9bbc1f6550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.191+0000 7f9bbbfff640 1 -- 172.21.15.106:0/2988199138 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bbc151490 con 0x7f9bbc0c61a0 2024-04-25T10:30:29.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.191+0000 7f9bc1754640 1 --2- 172.21.15.106:0/2988199138 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9bbc15d690 0x7f9bbc155960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:29.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.191+0000 7f9bbbfff640 1 --2- 172.21.15.106:0/2988199138 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9bbc0c61a0 0x7f9bbc15f770 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9ba8000c00 tx=0x7f9ba80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:29.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.191+0000 7f9bb97fa640 1 -- 172.21.15.106:0/2988199138 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba8037040 con 0x7f9bbc0c61a0 2024-04-25T10:30:29.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.191+0000 7f9bb97fa640 1 -- 172.21.15.106:0/2988199138 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ba8042070 con 0x7f9bbc0c61a0 2024-04-25T10:30:29.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.191+0000 7f9bc31b7640 1 -- 172.21.15.106:0/2988199138 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bbc15abf0 con 0x7f9bbc0c61a0 2024-04-25T10:30:29.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.191+0000 7f9bb97fa640 1 -- 172.21.15.106:0/2988199138 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba8030bd0 con 0x7f9bbc0c61a0 2024-04-25T10:30:29.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.191+0000 7f9bc31b7640 1 -- 172.21.15.106:0/2988199138 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bbc1f3360 con 0x7f9bbc0c61a0 2024-04-25T10:30:29.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.192+0000 7f9bc31b7640 1 -- 172.21.15.106:0/2988199138 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bbc06a3d0 con 0x7f9bbc0c61a0 2024-04-25T10:30:29.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.193+0000 7f9bb97fa640 1 -- 172.21.15.106:0/2988199138 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9ba8047460 con 0x7f9bbc0c61a0 2024-04-25T10:30:29.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.193+0000 7f9bb97fa640 1 --2- 172.21.15.106:0/2988199138 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f9ba00744b0 0x7f9ba00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:29.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.193+0000 7f9bb97fa640 1 -- 172.21.15.106:0/2988199138 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f9ba80b5ba0 con 0x7f9bbc0c61a0 2024-04-25T10:30:29.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.193+0000 7f9bc0f53640 1 --2- 172.21.15.106:0/2988199138 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f9ba00744b0 0x7f9ba00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:29.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.193+0000 7f9bc0f53640 1 --2- 172.21.15.106:0/2988199138 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f9ba00744b0 0x7f9ba00769c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9bb0000f60 tx=0x7f9bb0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:29.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.197+0000 7f9bb97fa640 1 -- 172.21.15.106:0/2988199138 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9ba8032e20 con 0x7f9bbc0c61a0 2024-04-25T10:30:29.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.372+0000 7f9bc31b7640 1 -- 172.21.15.106:0/2988199138 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9bbc0c67f0 con 0x7f9bbc0c61a0 2024-04-25T10:30:29.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.373+0000 7f9bb97fa640 1 -- 172.21.15.106:0/2988199138 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12920 (secure 0 0 0) 0x7f9ba8080680 con 0x7f9bbc0c61a0 2024-04-25T10:30:29.375 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:30:29.375 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":16,"fsid":"3ee77985-e8e4-41cf-81e9-7c774cf7891f","created":"2024-04-25T10:30:07.981054+0000","modified":"2024-04-25T10:30:22.366852+0000","last_up_change":"2024-04-25T10:30:19.184325+0000","last_in_change":"2024-04-25T10:30:10.217553+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:30:20.111046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a5962cb1-dbbd-47b1-8d15-9bae114bfdf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1240717079},{"type":"v1","addr":"172.21.15.106:6801","nonce":1240717079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1240717079},{"type":"v1","addr":"172.21.15.106:6803","nonce":1240717079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1240717079},{"type":"v1","addr":"172.21.15.106:6807","nonce":1240717079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1240717079},{"type":"v1","addr":"172.21.15.106:6805","nonce":1240717079}]},"public_addr":"172.21.15.106:6801/1240717079","cluster_addr":"172.21.15.106:6803/1240717079","heartbeat_back_addr":"172.21.15.106:6807/1240717079","heartbeat_front_addr":"172.21.15.106:6805/1240717079","state":["exists","up"]},{"osd":1,"uuid":"94436e13-4769-4752-8397-b5e396a6f7ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4089653051},{"type":"v1","addr":"172.21.15.106:6825","nonce":4089653051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4089653051},{"type":"v1","addr":"172.21.15.106:6827","nonce":4089653051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4089653051},{"type":"v1","addr":"172.21.15.106:6831","nonce":4089653051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4089653051},{"type":"v1","addr":"172.21.15.106:6829","nonce":4089653051}]},"public_addr":"172.21.15.106:6825/4089653051","cluster_addr":"172.21.15.106:6827/4089653051","heartbeat_back_addr":"172.21.15.106:6831/4089653051","heartbeat_front_addr":"172.21.15.106:6829/4089653051","state":["exists","up"]},{"osd":2,"uuid":"429aae0b-70c6-4e7b-9043-6a11334c7efc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":1441018239},{"type":"v1","addr":"172.21.15.106:6809","nonce":1441018239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1441018239},{"type":"v1","addr":"172.21.15.106:6811","nonce":1441018239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1441018239},{"type":"v1","addr":"172.21.15.106:6815","nonce":1441018239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":1441018239},{"type":"v1","addr":"172.21.15.106:6813","nonce":1441018239}]},"public_addr":"172.21.15.106:6809/1441018239","cluster_addr":"172.21.15.106:6811/1441018239","heartbeat_back_addr":"172.21.15.106:6815/1441018239","heartbeat_front_addr":"172.21.15.106:6813/1441018239","state":["exists","up"]},{"osd":3,"uuid":"9593a4d1-772f-4667-ab0d-2982f841e342","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":1846968711},{"type":"v1","addr":"172.21.15.106:6817","nonce":1846968711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1846968711},{"type":"v1","addr":"172.21.15.106:6819","nonce":1846968711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":1846968711},{"type":"v1","addr":"172.21.15.106:6823","nonce":1846968711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":1846968711},{"type":"v1","addr":"172.21.15.106:6821","nonce":1846968711}]},"public_addr":"172.21.15.106:6817/1846968711","cluster_addr":"172.21.15.106:6819/1846968711","heartbeat_back_addr":"172.21.15.106:6823/1846968711","heartbeat_front_addr":"172.21.15.106:6821/1846968711","state":["exists","up"]},{"osd":4,"uuid":"1b61f16d-5c60-480a-b449-c745aa3f6b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":729224739},{"type":"v1","addr":"172.21.15.134:6825","nonce":729224739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":729224739},{"type":"v1","addr":"172.21.15.134:6827","nonce":729224739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":729224739},{"type":"v1","addr":"172.21.15.134:6831","nonce":729224739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":729224739},{"type":"v1","addr":"172.21.15.134:6829","nonce":729224739}]},"public_addr":"172.21.15.134:6825/729224739","cluster_addr":"172.21.15.134:6827/729224739","heartbeat_back_addr":"172.21.15.134:6831/729224739","heartbeat_front_addr":"172.21.15.134:6829/729224739","state":["exists","up"]},{"osd":5,"uuid":"944cdd7f-47c0-4b71-b7e0-d093542f4528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3132275652},{"type":"v1","addr":"172.21.15.134:6809","nonce":3132275652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3132275652},{"type":"v1","addr":"172.21.15.134:6811","nonce":3132275652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3132275652},{"type":"v1","addr":"172.21.15.134:6815","nonce":3132275652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3132275652},{"type":"v1","addr":"172.21.15.134:6813","nonce":3132275652}]},"public_addr":"172.21.15.134:6809/3132275652","cluster_addr":"172.21.15.134:6811/3132275652","heartbeat_back_addr":"172.21.15.134:6815/3132275652","heartbeat_front_addr":"172.21.15.134:6813/3132275652","state":["exists","up"]},{"osd":6,"uuid":"cc2446e2-1d21-4135-ac04-6af7c990058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":992702668},{"type":"v1","addr":"172.21.15.134:6801","nonce":992702668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":992702668},{"type":"v1","addr":"172.21.15.134:6803","nonce":992702668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":992702668},{"type":"v1","addr":"172.21.15.134:6807","nonce":992702668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":992702668},{"type":"v1","addr":"172.21.15.134:6805","nonce":992702668}]},"public_addr":"172.21.15.134:6801/992702668","cluster_addr":"172.21.15.134:6803/992702668","heartbeat_back_addr":"172.21.15.134:6807/992702668","heartbeat_front_addr":"172.21.15.134:6805/992702668","state":["exists","up"]},{"osd":7,"uuid":"3df1fc7a-82fc-4257-8629-98027b6e2aed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":3291062284},{"type":"v1","addr":"172.21.15.134:6817","nonce":3291062284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":3291062284},{"type":"v1","addr":"172.21.15.134:6819","nonce":3291062284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":3291062284},{"type":"v1","addr":"172.21.15.134:6823","nonce":3291062284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":3291062284},{"type":"v1","addr":"172.21.15.134:6821","nonce":3291062284}]},"public_addr":"172.21.15.134:6817/3291062284","cluster_addr":"172.21.15.134:6819/3291062284","heartbeat_back_addr":"172.21.15.134:6823/3291062284","heartbeat_front_addr":"172.21.15.134:6821/3291062284","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.403941+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.382563+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.411544+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.330444+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.405430+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.387737+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.408922+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.454450+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:30:29.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.374+0000 7f9bc31b7640 1 -- 172.21.15.106:0/2988199138 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f9ba00744b0 msgr2=0x7f9ba00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:29.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.374+0000 7f9bc31b7640 1 --2- 172.21.15.106:0/2988199138 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f9ba00744b0 0x7f9ba00769c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9bb0000f60 tx=0x7f9bb0009040 comp rx=0 tx=0).stop 2024-04-25T10:30:29.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.374+0000 7f9bc31b7640 1 -- 172.21.15.106:0/2988199138 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9bbc0c61a0 msgr2=0x7f9bbc15f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:29.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.374+0000 7f9bc31b7640 1 --2- 172.21.15.106:0/2988199138 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9bbc0c61a0 0x7f9bbc15f770 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9ba8000c00 tx=0x7f9ba80046c0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.375+0000 7f9bc31b7640 1 -- 172.21.15.106:0/2988199138 shutdown_connections 2024-04-25T10:30:29.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.375+0000 7f9bc31b7640 1 --2- 172.21.15.106:0/2988199138 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f9ba00744b0 0x7f9ba00769c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.375+0000 7f9bc31b7640 1 --2- 172.21.15.106:0/2988199138 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9bbc15d690 0x7f9bbc155960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.375+0000 7f9bc31b7640 1 --2- 172.21.15.106:0/2988199138 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9bbc0c61a0 0x7f9bbc15f770 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.375+0000 7f9bc31b7640 1 --2- 172.21.15.106:0/2988199138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9bbc06dc00 0x7f9bbc1f6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.375+0000 7f9bc31b7640 1 -- 172.21.15.106:0/2988199138 >> 172.21.15.106:0/2988199138 conn(0x7f9bbc0727d0 msgr2=0x7f9bbc06e890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:29.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.375+0000 7f9bc31b7640 1 -- 172.21.15.106:0/2988199138 shutdown_connections 2024-04-25T10:30:29.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.375+0000 7f9bc31b7640 1 -- 172.21.15.106:0/2988199138 wait complete. 2024-04-25T10:30:29.389 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:30:20.111046+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:30:29.389 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:30:29.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.498+0000 7f6b24630640 1 Processor -- start 2024-04-25T10:30:29.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.498+0000 7f6b24630640 1 -- start start 2024-04-25T10:30:29.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.499+0000 7f6b24630640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b1c0c6cd0 0x7f6b1c0cace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:29.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.499+0000 7f6b24630640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b1c0722d0 con 0x7f6b1c0c9cc0 2024-04-25T10:30:29.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.499+0000 7f6b24630640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b1c0c7430 con 0x7f6b1c0ca790 2024-04-25T10:30:29.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.499+0000 7f6b24630640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6b1c0c6430 con 0x7f6b1c0c6cd0 2024-04-25T10:30:29.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.499+0000 7f6b22bcd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f6b1c0c9cc0 0x7f6b1c164400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:51268/0 (socket says 172.21.15.106:51268) 2024-04-25T10:30:29.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.499+0000 7f6b22bcd640 1 -- 172.21.15.106:0/644170501 learned_addr learned my addr 172.21.15.106:0/644170501 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:30:29.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.499+0000 7f6b21bcb640 1 --2- 172.21.15.106:0/644170501 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b1c0c6cd0 0x7f6b1c0cace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:29.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.499+0000 7f6b213ca640 1 -- 172.21.15.106:0/644170501 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 262131996 0 0) 0x7f6b1c0722d0 con 0x7f6b1c0c9cc0 2024-04-25T10:30:29.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.499+0000 7f6b213ca640 1 -- 172.21.15.106:0/644170501 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6b08003520 con 0x7f6b1c0c9cc0 2024-04-25T10:30:29.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.500+0000 7f6b213ca640 1 -- 172.21.15.106:0/644170501 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1140677150 0 0) 0x7f6b1c0c7430 con 0x7f6b1c0ca790 2024-04-25T10:30:29.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.500+0000 7f6b213ca640 1 -- 172.21.15.106:0/644170501 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6b1c0722d0 con 0x7f6b1c0ca790 2024-04-25T10:30:29.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.500+0000 7f6b21bcb640 1 -- 172.21.15.106:0/644170501 >> v1:172.21.15.134:6789/0 conn(0x7f6b1c0ca790 legacy=0x7f6b1c153b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:29.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.500+0000 7f6b21bcb640 1 -- 172.21.15.106:0/644170501 >> v1:172.21.15.106:6789/0 conn(0x7f6b1c0c9cc0 legacy=0x7f6b1c164400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:29.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.500+0000 7f6b21bcb640 1 -- 172.21.15.106:0/644170501 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b1c0c6710 con 0x7f6b1c0c6cd0 2024-04-25T10:30:29.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.500+0000 7f6b21bcb640 1 --2- 172.21.15.106:0/644170501 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b1c0c6cd0 0x7f6b1c0cace0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c0097d0 tx=0x7f6b0c02a290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d37e96a0e9b57e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:29.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.500+0000 7f6b213ca640 1 -- 172.21.15.106:0/644170501 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c037070 con 0x7f6b1c0c6cd0 2024-04-25T10:30:29.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.501+0000 7f6b213ca640 1 -- 172.21.15.106:0/644170501 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b0c032440 con 0x7f6b1c0c6cd0 2024-04-25T10:30:29.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.501+0000 7f6b213ca640 1 -- 172.21.15.106:0/644170501 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c0327f0 con 0x7f6b1c0c6cd0 2024-04-25T10:30:29.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.501+0000 7f6b24630640 1 -- 172.21.15.106:0/644170501 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b1c0c6cd0 msgr2=0x7f6b1c0cace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:29.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.501+0000 7f6b24630640 1 --2- 172.21.15.106:0/644170501 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b1c0c6cd0 0x7f6b1c0cace0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c0097d0 tx=0x7f6b0c02a290 comp rx=0 tx=0).stop 2024-04-25T10:30:29.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.501+0000 7f6b24630640 1 -- 172.21.15.106:0/644170501 shutdown_connections 2024-04-25T10:30:29.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.501+0000 7f6b24630640 1 --2- 172.21.15.106:0/644170501 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b1c0c6cd0 0x7f6b1c0cace0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.501+0000 7f6b24630640 1 -- 172.21.15.106:0/644170501 >> 172.21.15.106:0/644170501 conn(0x7f6b1c06daf0 msgr2=0x7f6b1c06df00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:29.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.501+0000 7f6b24630640 1 -- 172.21.15.106:0/644170501 shutdown_connections 2024-04-25T10:30:29.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.501+0000 7f6b24630640 1 -- 172.21.15.106:0/644170501 wait complete. 2024-04-25T10:30:29.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.503+0000 7f6b24630640 1 Processor -- start 2024-04-25T10:30:29.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.503+0000 7f6b24630640 1 -- start start 2024-04-25T10:30:29.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.504+0000 7f6b24630640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b1c0c6cd0 0x7f6b1c1580f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:29.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.504+0000 7f6b24630640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b1c0c9cc0 0x7f6b1c158fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:29.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.504+0000 7f6b24630640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b1c0ca790 0x7f6b1c159510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:29.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.504+0000 7f6b24630640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1c0c70e0 con 0x7f6b1c0ca790 2024-04-25T10:30:29.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.504+0000 7f6b24630640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1c0cbf00 con 0x7f6b1c0c9cc0 2024-04-25T10:30:29.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.504+0000 7f6b24630640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6b1c0cb220 con 0x7f6b1c0c6cd0 2024-04-25T10:30:29.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.504+0000 7f6b22bcd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b1c0ca790 0x7f6b1c159510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:29.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.504+0000 7f6b22bcd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b1c0ca790 0x7f6b1c159510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57940/0 (socket says 172.21.15.106:57940) 2024-04-25T10:30:29.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.504+0000 7f6b22bcd640 1 -- 172.21.15.106:0/860255651 learned_addr learned my addr 172.21.15.106:0/860255651 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:29.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.504+0000 7f6b223cc640 1 --2- 172.21.15.106:0/860255651 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b1c0c6cd0 0x7f6b1c1580f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:29.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.504+0000 7f6b21bcb640 1 --2- 172.21.15.106:0/860255651 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b1c0c9cc0 0x7f6b1c158fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:29.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.505+0000 7f6b22bcd640 1 -- 172.21.15.106:0/860255651 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b1c0c6cd0 msgr2=0x7f6b1c1580f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:29.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.505+0000 7f6b22bcd640 1 --2- 172.21.15.106:0/860255651 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b1c0c6cd0 0x7f6b1c1580f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.505+0000 7f6b22bcd640 1 -- 172.21.15.106:0/860255651 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b1c0c9cc0 msgr2=0x7f6b1c158fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:29.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.505+0000 7f6b22bcd640 1 --2- 172.21.15.106:0/860255651 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b1c0c9cc0 0x7f6b1c158fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.505+0000 7f6b22bcd640 1 -- 172.21.15.106:0/860255651 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b1c144700 con 0x7f6b1c0ca790 2024-04-25T10:30:29.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.505+0000 7f6b22bcd640 1 --2- 172.21.15.106:0/860255651 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b1c0ca790 0x7f6b1c159510 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6b1400cad0 tx=0x7f6b1400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:29.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.505+0000 7f6b12ffd640 1 -- 172.21.15.106:0/860255651 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b14017070 con 0x7f6b1c0ca790 2024-04-25T10:30:29.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.505+0000 7f6b12ffd640 1 -- 172.21.15.106:0/860255651 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b1400abd0 con 0x7f6b1c0ca790 2024-04-25T10:30:29.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.505+0000 7f6b24630640 1 -- 172.21.15.106:0/860255651 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b1c15a5a0 con 0x7f6b1c0ca790 2024-04-25T10:30:29.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.505+0000 7f6b12ffd640 1 -- 172.21.15.106:0/860255651 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b14012400 con 0x7f6b1c0ca790 2024-04-25T10:30:29.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.505+0000 7f6b24630640 1 -- 172.21.15.106:0/860255651 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b1c145170 con 0x7f6b1c0ca790 2024-04-25T10:30:29.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.506+0000 7f6b24630640 1 -- 172.21.15.106:0/860255651 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b1c06a3d0 con 0x7f6b1c0ca790 2024-04-25T10:30:29.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.507+0000 7f6b12ffd640 1 -- 172.21.15.106:0/860255651 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6b1401a070 con 0x7f6b1c0ca790 2024-04-25T10:30:29.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.507+0000 7f6b12ffd640 1 --2- 172.21.15.106:0/860255651 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f6b000744b0 0x7f6b000769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:29.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.507+0000 7f6b223cc640 1 --2- 172.21.15.106:0/860255651 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f6b000744b0 0x7f6b000769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:29.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.507+0000 7f6b12ffd640 1 -- 172.21.15.106:0/860255651 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f6b14094a80 con 0x7f6b1c0ca790 2024-04-25T10:30:29.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.508+0000 7f6b223cc640 1 --2- 172.21.15.106:0/860255651 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f6b000744b0 0x7f6b000769c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6b18007fd0 tx=0x7f6b18009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:29.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.512+0000 7f6b12ffd640 1 -- 172.21.15.106:0/860255651 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6b1405efa0 con 0x7f6b1c0ca790 2024-04-25T10:30:29.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.684+0000 7f6b24630640 1 -- 172.21.15.106:0/860255651 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6b1c151960 con 0x7f6b1c0ca790 2024-04-25T10:30:29.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.685+0000 7f6b12ffd640 1 -- 172.21.15.106:0/860255651 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7f6b140632a0 con 0x7f6b1c0ca790 2024-04-25T10:30:29.687 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-25T10:30:29.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.687+0000 7f6b24630640 1 -- 172.21.15.106:0/860255651 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f6b000744b0 msgr2=0x7f6b000769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:29.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.687+0000 7f6b24630640 1 --2- 172.21.15.106:0/860255651 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f6b000744b0 0x7f6b000769c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6b18007fd0 tx=0x7f6b18009040 comp rx=0 tx=0).stop 2024-04-25T10:30:29.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.687+0000 7f6b24630640 1 -- 172.21.15.106:0/860255651 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b1c0ca790 msgr2=0x7f6b1c159510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:29.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.687+0000 7f6b24630640 1 --2- 172.21.15.106:0/860255651 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b1c0ca790 0x7f6b1c159510 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6b1400cad0 tx=0x7f6b1400cf90 comp rx=0 tx=0).stop 2024-04-25T10:30:29.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.687+0000 7f6b24630640 1 -- 172.21.15.106:0/860255651 shutdown_connections 2024-04-25T10:30:29.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.687+0000 7f6b24630640 1 --2- 172.21.15.106:0/860255651 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f6b000744b0 0x7f6b000769c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.687+0000 7f6b24630640 1 --2- 172.21.15.106:0/860255651 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b1c0ca790 0x7f6b1c159510 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.687+0000 7f6b24630640 1 --2- 172.21.15.106:0/860255651 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b1c0c9cc0 0x7f6b1c158fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.687+0000 7f6b24630640 1 --2- 172.21.15.106:0/860255651 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b1c0c6cd0 0x7f6b1c1580f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.687+0000 7f6b24630640 1 -- 172.21.15.106:0/860255651 >> 172.21.15.106:0/860255651 conn(0x7f6b1c06daf0 msgr2=0x7f6b1c148c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:29.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.687+0000 7f6b24630640 1 -- 172.21.15.106:0/860255651 shutdown_connections 2024-04-25T10:30:29.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.687+0000 7f6b24630640 1 -- 172.21.15.106:0/860255651 wait complete. 2024-04-25T10:30:29.700 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:30:29.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.807+0000 7f8eca16c640 1 Processor -- start 2024-04-25T10:30:29.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.807+0000 7f8eca16c640 1 -- start start 2024-04-25T10:30:29.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.808+0000 7f8eca16c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f8ec40c7ad0 0x7f8ec406ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:29.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.808+0000 7f8eca16c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ec40c8120 0x7f8ec40c8530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:29.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.808+0000 7f8eca16c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ec415d6d0 con 0x7f8ec4156f70 2024-04-25T10:30:29.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.808+0000 7f8eca16c640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f8ec415d9b0 con 0x7f8ec40c7ad0 2024-04-25T10:30:29.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.808+0000 7f8eca16c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f8ec40c7150 con 0x7f8ec40c8120 2024-04-25T10:30:29.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.808+0000 7f8ec37fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f8ec40c7ad0 0x7f8ec406ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:29.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.808+0000 7f8ec2ffd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f8ec4156f70 0x7f8ec406d560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:51284/0 (socket says 172.21.15.106:51284) 2024-04-25T10:30:29.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.808+0000 7f8ec2ffd640 1 -- 172.21.15.106:0/3594206657 learned_addr learned my addr 172.21.15.106:0/3594206657 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:30:29.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.808+0000 7f8ec3fff640 1 --2- 172.21.15.106:0/3594206657 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ec40c8120 0x7f8ec40c8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:29.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.809+0000 7f8ec37fe640 1 -- 172.21.15.106:0/3594206657 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ec40c8120 msgr2=0x7f8ec40c8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:29.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.809+0000 7f8ec37fe640 1 --2- 172.21.15.106:0/3594206657 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ec40c8120 0x7f8ec40c8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.809+0000 7f8ec37fe640 1 -- 172.21.15.106:0/3594206657 >> v1:172.21.15.106:6789/0 conn(0x7f8ec4156f70 legacy=0x7f8ec406d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:29.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.809+0000 7f8ec37fe640 1 -- 172.21.15.106:0/3594206657 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ec40c7430 con 0x7f8ec40c7ad0 2024-04-25T10:30:29.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.809+0000 7f8ec27fc640 1 -- 172.21.15.106:0/3594206657 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 857556419 0 0) 0x7f8ec415d6d0 con 0x7f8ec4156f70 2024-04-25T10:30:29.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.809+0000 7f8ec37fe640 1 --2- 172.21.15.106:0/3594206657 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8ec40c7ad0 0x7f8ec406ea60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8eb40097a0 tx=0x7f8eb402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5259609d8a47e118 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:29.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.809+0000 7f8ec27fc640 1 -- 172.21.15.106:0/3594206657 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb4037070 con 0x7f8ec40c7ad0 2024-04-25T10:30:29.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.809+0000 7f8ec27fc640 1 -- 172.21.15.106:0/3594206657 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8eb4032440 con 0x7f8ec40c7ad0 2024-04-25T10:30:29.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.809+0000 7f8ec27fc640 1 -- 172.21.15.106:0/3594206657 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb4032810 con 0x7f8ec40c7ad0 2024-04-25T10:30:29.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.810+0000 7f8eca16c640 1 -- 172.21.15.106:0/3594206657 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8ec40c7ad0 msgr2=0x7f8ec406ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:29.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.810+0000 7f8eca16c640 1 --2- 172.21.15.106:0/3594206657 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8ec40c7ad0 0x7f8ec406ea60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8eb40097a0 tx=0x7f8eb402a180 comp rx=0 tx=0).stop 2024-04-25T10:30:29.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.810+0000 7f8eca16c640 1 -- 172.21.15.106:0/3594206657 shutdown_connections 2024-04-25T10:30:29.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.810+0000 7f8eca16c640 1 --2- 172.21.15.106:0/3594206657 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ec40c8120 0x7f8ec40c8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.810+0000 7f8eca16c640 1 --2- 172.21.15.106:0/3594206657 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8ec40c7ad0 0x7f8ec406ea60 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.810+0000 7f8eca16c640 1 -- 172.21.15.106:0/3594206657 >> 172.21.15.106:0/3594206657 conn(0x7f8ec4068ed0 msgr2=0x7f8ec40d20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:29.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.810+0000 7f8eca16c640 1 -- 172.21.15.106:0/3594206657 shutdown_connections 2024-04-25T10:30:29.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.810+0000 7f8eca16c640 1 -- 172.21.15.106:0/3594206657 wait complete. 2024-04-25T10:30:29.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.812+0000 7f8eca16c640 1 Processor -- start 2024-04-25T10:30:29.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.812+0000 7f8eca16c640 1 -- start start 2024-04-25T10:30:29.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.813+0000 7f8eca16c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ec40c7ad0 0x7f8ec415ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:29.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.813+0000 7f8eca16c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ec40c8120 0x7f8ec40cff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:29.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.813+0000 7f8eca16c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8ec4156f70 0x7f8ec40cc920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:29.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.813+0000 7f8eca16c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec40ce360 con 0x7f8ec4156f70 2024-04-25T10:30:29.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.813+0000 7f8eca16c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec40cdea0 con 0x7f8ec40c8120 2024-04-25T10:30:29.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.813+0000 7f8eca16c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f8ec40ce6c0 con 0x7f8ec40c7ad0 2024-04-25T10:30:29.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.813+0000 7f8ec3fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8ec4156f70 0x7f8ec40cc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:29.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.813+0000 7f8ec3fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8ec4156f70 0x7f8ec40cc920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57968/0 (socket says 172.21.15.106:57968) 2024-04-25T10:30:29.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.813+0000 7f8ec3fff640 1 -- 172.21.15.106:0/3932477033 learned_addr learned my addr 172.21.15.106:0/3932477033 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:29.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.813+0000 7f8ec37fe640 1 --2- 172.21.15.106:0/3932477033 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ec40c7ad0 0x7f8ec415ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:29.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.813+0000 7f8ec2ffd640 1 --2- 172.21.15.106:0/3932477033 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ec40c8120 0x7f8ec40cff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:29.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.813+0000 7f8ec3fff640 1 -- 172.21.15.106:0/3932477033 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ec40c7ad0 msgr2=0x7f8ec415ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:29.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.813+0000 7f8ec3fff640 1 --2- 172.21.15.106:0/3932477033 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ec40c7ad0 0x7f8ec415ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.814+0000 7f8ec3fff640 1 -- 172.21.15.106:0/3932477033 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ec40c8120 msgr2=0x7f8ec40cff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:29.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.814+0000 7f8ec3fff640 1 --2- 172.21.15.106:0/3932477033 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ec40c8120 0x7f8ec40cff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.814+0000 7f8ec3fff640 1 -- 172.21.15.106:0/3932477033 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ec4160f90 con 0x7f8ec4156f70 2024-04-25T10:30:29.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.814+0000 7f8ec37fe640 1 --2- 172.21.15.106:0/3932477033 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ec40c7ad0 0x7f8ec415ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:29.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.814+0000 7f8ec3fff640 1 --2- 172.21.15.106:0/3932477033 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8ec4156f70 0x7f8ec40cc920 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc00efc0 tx=0x7f8ebc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:29.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.814+0000 7f8eabfff640 1 -- 172.21.15.106:0/3932477033 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc018020 con 0x7f8ec4156f70 2024-04-25T10:30:29.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.814+0000 7f8eabfff640 1 -- 172.21.15.106:0/3932477033 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ebc014480 con 0x7f8ec4156f70 2024-04-25T10:30:29.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.814+0000 7f8eca16c640 1 -- 172.21.15.106:0/3932477033 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ec40d17c0 con 0x7f8ec4156f70 2024-04-25T10:30:29.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.814+0000 7f8eabfff640 1 -- 172.21.15.106:0/3932477033 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc009e60 con 0x7f8ec4156f70 2024-04-25T10:30:29.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.814+0000 7f8eca16c640 1 -- 172.21.15.106:0/3932477033 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ec40d1d80 con 0x7f8ec4156f70 2024-04-25T10:30:29.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.815+0000 7f8eca16c640 1 -- 172.21.15.106:0/3932477033 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ec406a3d0 con 0x7f8ec4156f70 2024-04-25T10:30:29.819 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.815+0000 7f8eabfff640 1 -- 172.21.15.106:0/3932477033 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8ebc005d10 con 0x7f8ec4156f70 2024-04-25T10:30:29.819 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.816+0000 7f8eabfff640 1 --2- 172.21.15.106:0/3932477033 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f8eb00744b0 0x7f8eb00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:29.819 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.816+0000 7f8eabfff640 1 -- 172.21.15.106:0/3932477033 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f8ebc095c10 con 0x7f8ec4156f70 2024-04-25T10:30:29.819 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.816+0000 7f8ec37fe640 1 --2- 172.21.15.106:0/3932477033 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f8eb00744b0 0x7f8eb00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:29.819 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.816+0000 7f8ec37fe640 1 --2- 172.21.15.106:0/3932477033 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f8eb00744b0 0x7f8eb00769c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4034000 tx=0x7f8eb402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:29.822 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.820+0000 7f8eabfff640 1 -- 172.21.15.106:0/3932477033 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8ebc022030 con 0x7f8ec4156f70 2024-04-25T10:30:30.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.998+0000 7f8eca16c640 1 -- 172.21.15.106:0/3932477033 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f8ec406e540 con 0x7f8ec4156f70 2024-04-25T10:30:30.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:29.999+0000 7f8eabfff640 1 -- 172.21.15.106:0/3932477033 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) ==== 84+0+65 (secure 0 0 0) 0x7f8ec4160f90 con 0x7f8ec4156f70 2024-04-25T10:30:30.001 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:30:30.001 INFO:teuthology.orchestra.run.smithi106.stdout:[ 2024-04-25T10:30:30.001 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:30:30.002 INFO:teuthology.orchestra.run.smithi106.stdout: "poolnum": 1, 2024-04-25T10:30:30.002 INFO:teuthology.orchestra.run.smithi106.stdout: "poolname": ".mgr" 2024-04-25T10:30:30.002 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:30:30.002 INFO:teuthology.orchestra.run.smithi106.stdout:] 2024-04-25T10:30:30.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.000+0000 7f8eca16c640 1 -- 172.21.15.106:0/3932477033 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f8eb00744b0 msgr2=0x7f8eb00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:30.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.000+0000 7f8eca16c640 1 --2- 172.21.15.106:0/3932477033 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f8eb00744b0 0x7f8eb00769c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4034000 tx=0x7f8eb402e000 comp rx=0 tx=0).stop 2024-04-25T10:30:30.003 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.001+0000 7f8eca16c640 1 -- 172.21.15.106:0/3932477033 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8ec4156f70 msgr2=0x7f8ec40cc920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:30.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.001+0000 7f8eca16c640 1 --2- 172.21.15.106:0/3932477033 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8ec4156f70 0x7f8ec40cc920 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc00efc0 tx=0x7f8ebc00c490 comp rx=0 tx=0).stop 2024-04-25T10:30:30.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.001+0000 7f8eca16c640 1 -- 172.21.15.106:0/3932477033 shutdown_connections 2024-04-25T10:30:30.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.001+0000 7f8eca16c640 1 --2- 172.21.15.106:0/3932477033 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f8eb00744b0 0x7f8eb00769c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:30.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.001+0000 7f8eca16c640 1 --2- 172.21.15.106:0/3932477033 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8ec4156f70 0x7f8ec40cc920 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:30.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.001+0000 7f8eca16c640 1 --2- 172.21.15.106:0/3932477033 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ec40c8120 0x7f8ec40cff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:30.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.001+0000 7f8eca16c640 1 --2- 172.21.15.106:0/3932477033 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ec40c7ad0 0x7f8ec415ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:30.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.001+0000 7f8eca16c640 1 -- 172.21.15.106:0/3932477033 >> 172.21.15.106:0/3932477033 conn(0x7f8ec4068ed0 msgr2=0x7f8ec406d560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:30.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.001+0000 7f8eca16c640 1 -- 172.21.15.106:0/3932477033 shutdown_connections 2024-04-25T10:30:30.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.001+0000 7f8eca16c640 1 -- 172.21.15.106:0/3932477033 wait complete. 2024-04-25T10:30:30.014 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:30:30.015 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:30:30.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.124+0000 7f46ed7d5640 1 Processor -- start 2024-04-25T10:30:30.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.124+0000 7f46ed7d5640 1 -- start start 2024-04-25T10:30:30.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.125+0000 7f46ed7d5640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f46e80d0790 0x7f46e80d0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:30.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.125+0000 7f46ed7d5640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f46e8156f00 0x7f46e806eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:30.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.125+0000 7f46ed7d5640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f46e806f2e0 0x7f46e8160130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:30.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.125+0000 7f46ed7d5640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f46e80ce5e0 con 0x7f46e80d0790 2024-04-25T10:30:30.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.125+0000 7f46ed7d5640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f46e815d580 con 0x7f46e806f2e0 2024-04-25T10:30:30.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.125+0000 7f46ed7d5640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f46e8161050 con 0x7f46e8156f00 2024-04-25T10:30:30.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.126+0000 7f46e6ffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f46e80d0790 0x7f46e80d0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:30.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.126+0000 7f46e6ffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f46e80d0790 0x7f46e80d0de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57984/0 (socket says 172.21.15.106:57984) 2024-04-25T10:30:30.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.126+0000 7f46e6ffd640 1 -- 172.21.15.106:0/2292371907 learned_addr learned my addr 172.21.15.106:0/2292371907 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:30.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.126+0000 7f46e67fc640 1 --2- 172.21.15.106:0/2292371907 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f46e8156f00 0x7f46e806eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:30.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.126+0000 7f46e77fe640 1 --2- 172.21.15.106:0/2292371907 >> v2:172.21.15.134:3300/0 conn(0x7f46e806f2e0 0x7f46e8160130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:30.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.126+0000 7f46e6ffd640 1 -- 172.21.15.106:0/2292371907 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f46e8156f00 msgr2=0x7f46e806eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:30.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.126+0000 7f46e6ffd640 1 --2- 172.21.15.106:0/2292371907 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f46e8156f00 0x7f46e806eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:30.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.126+0000 7f46e6ffd640 1 -- 172.21.15.106:0/2292371907 >> v2:172.21.15.134:3300/0 conn(0x7f46e806f2e0 msgr2=0x7f46e8160130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:30.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.126+0000 7f46e6ffd640 1 --2- 172.21.15.106:0/2292371907 >> v2:172.21.15.134:3300/0 conn(0x7f46e806f2e0 0x7f46e8160130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:30.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.126+0000 7f46e6ffd640 1 -- 172.21.15.106:0/2292371907 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46e8160d70 con 0x7f46e80d0790 2024-04-25T10:30:30.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.126+0000 7f46e77fe640 1 --2- 172.21.15.106:0/2292371907 >> v2:172.21.15.134:3300/0 conn(0x7f46e806f2e0 0x7f46e8160130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:30.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.127+0000 7f46e6ffd640 1 --2- 172.21.15.106:0/2292371907 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46e80d0790 0x7f46e80d0de0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f46e80903c0 tx=0x7f46d802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4323c69436f66e3f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:30.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.127+0000 7f46e5ffb640 1 -- 172.21.15.106:0/2292371907 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d8036070 con 0x7f46e80d0790 2024-04-25T10:30:30.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.127+0000 7f46e5ffb640 1 -- 172.21.15.106:0/2292371907 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46d8031600 con 0x7f46e80d0790 2024-04-25T10:30:30.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.127+0000 7f46e5ffb640 1 -- 172.21.15.106:0/2292371907 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d80319d0 con 0x7f46e80d0790 2024-04-25T10:30:30.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.127+0000 7f46ed7d5640 1 -- 172.21.15.106:0/2292371907 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46e80d0790 msgr2=0x7f46e80d0de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:30.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.127+0000 7f46ed7d5640 1 --2- 172.21.15.106:0/2292371907 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46e80d0790 0x7f46e80d0de0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f46e80903c0 tx=0x7f46d802a730 comp rx=0 tx=0).stop 2024-04-25T10:30:30.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.127+0000 7f46ed7d5640 1 -- 172.21.15.106:0/2292371907 shutdown_connections 2024-04-25T10:30:30.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.127+0000 7f46ed7d5640 1 --2- 172.21.15.106:0/2292371907 >> v2:172.21.15.134:3300/0 conn(0x7f46e806f2e0 0x7f46e8160130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:30.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.127+0000 7f46ed7d5640 1 --2- 172.21.15.106:0/2292371907 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f46e8156f00 0x7f46e806eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:30.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.127+0000 7f46ed7d5640 1 --2- 172.21.15.106:0/2292371907 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46e80d0790 0x7f46e80d0de0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:30.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.127+0000 7f46ed7d5640 1 -- 172.21.15.106:0/2292371907 >> 172.21.15.106:0/2292371907 conn(0x7f46e80684e0 msgr2=0x7f46e80688f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:30.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.127+0000 7f46ed7d5640 1 -- 172.21.15.106:0/2292371907 shutdown_connections 2024-04-25T10:30:30.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.128+0000 7f46ed7d5640 1 -- 172.21.15.106:0/2292371907 wait complete. 2024-04-25T10:30:30.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.129+0000 7f46ed7d5640 1 Processor -- start 2024-04-25T10:30:30.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.129+0000 7f46ed7d5640 1 -- start start 2024-04-25T10:30:30.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.130+0000 7f46ed7d5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46e806f2e0 0x7f46e81f34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:30.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.131+0000 7f46ed7d5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46e80d0790 0x7f46e8151d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:30.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.131+0000 7f46ed7d5640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f46e8156f00 0x7f46e8150950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:30.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.131+0000 7f46ed7d5640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f46e815d210 con 0x7f46e806f2e0 2024-04-25T10:30:30.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.131+0000 7f46ed7d5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f46e815cf30 con 0x7f46e80d0790 2024-04-25T10:30:30.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.131+0000 7f46ed7d5640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f46e815da10 con 0x7f46e8156f00 2024-04-25T10:30:30.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.131+0000 7f46e6ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46e806f2e0 0x7f46e81f34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:30.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.131+0000 7f46e6ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46e806f2e0 0x7f46e81f34c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57994/0 (socket says 172.21.15.106:57994) 2024-04-25T10:30:30.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.131+0000 7f46e6ffd640 1 -- 172.21.15.106:0/3915989105 learned_addr learned my addr 172.21.15.106:0/3915989105 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:30.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.131+0000 7f46e77fe640 1 --2- 172.21.15.106:0/3915989105 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f46e8156f00 0x7f46e8150950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:30.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.131+0000 7f46e67fc640 1 --2- 172.21.15.106:0/3915989105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46e80d0790 0x7f46e8151d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:30.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.131+0000 7f46e6ffd640 1 -- 172.21.15.106:0/3915989105 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f46e8156f00 msgr2=0x7f46e8150950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:30.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.131+0000 7f46e6ffd640 1 --2- 172.21.15.106:0/3915989105 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f46e8156f00 0x7f46e8150950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:30.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.131+0000 7f46e6ffd640 1 -- 172.21.15.106:0/3915989105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46e80d0790 msgr2=0x7f46e8151d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:30.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.131+0000 7f46e6ffd640 1 --2- 172.21.15.106:0/3915989105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46e80d0790 0x7f46e8151d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:30.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.131+0000 7f46e6ffd640 1 -- 172.21.15.106:0/3915989105 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46e8150e90 con 0x7f46e806f2e0 2024-04-25T10:30:30.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.131+0000 7f46e6ffd640 1 --2- 172.21.15.106:0/3915989105 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46e806f2e0 0x7f46e81f34c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f46d8034040 tx=0x7f46d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:30.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.131+0000 7f46cf7fe640 1 -- 172.21.15.106:0/3915989105 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d8036050 con 0x7f46e806f2e0 2024-04-25T10:30:30.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.131+0000 7f46cf7fe640 1 -- 172.21.15.106:0/3915989105 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46d8042070 con 0x7f46e806f2e0 2024-04-25T10:30:30.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.131+0000 7f46ed7d5640 1 -- 172.21.15.106:0/3915989105 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46e8151440 con 0x7f46e806f2e0 2024-04-25T10:30:30.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.131+0000 7f46cf7fe640 1 -- 172.21.15.106:0/3915989105 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d8009510 con 0x7f46e806f2e0 2024-04-25T10:30:30.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.131+0000 7f46ed7d5640 1 -- 172.21.15.106:0/3915989105 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46e8159fb0 con 0x7f46e806f2e0 2024-04-25T10:30:30.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.132+0000 7f46ed7d5640 1 -- 172.21.15.106:0/3915989105 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46e806a3d0 con 0x7f46e806f2e0 2024-04-25T10:30:30.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.133+0000 7f46cf7fe640 1 -- 172.21.15.106:0/3915989105 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f46d8007c50 con 0x7f46e806f2e0 2024-04-25T10:30:30.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.133+0000 7f46cf7fe640 1 --2- 172.21.15.106:0/3915989105 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f46d40744b0 0x7f46d40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:30.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.133+0000 7f46e67fc640 1 --2- 172.21.15.106:0/3915989105 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f46d40744b0 0x7f46d40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:30.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.134+0000 7f46cf7fe640 1 -- 172.21.15.106:0/3915989105 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f46d80b5d30 con 0x7f46e806f2e0 2024-04-25T10:30:30.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.134+0000 7f46e67fc640 1 --2- 172.21.15.106:0/3915989105 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f46d40744b0 0x7f46d40769c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f46dc006fd0 tx=0x7f46dc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:30.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.138+0000 7f46cf7fe640 1 -- 172.21.15.106:0/3915989105 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f46e8150e90 con 0x7f46e806f2e0 2024-04-25T10:30:30.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.310+0000 7f46ed7d5640 1 -- 172.21.15.106:0/3915989105 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f46e80d2460 con 0x7f46e806f2e0 2024-04-25T10:30:30.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.389+0000 7f46cf7fe640 1 -- 172.21.15.106:0/3915989105 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) ==== 138+0+0 (secure 0 0 0) 0x7f46d80808a0 con 0x7f46e806f2e0 2024-04-25T10:30:30.391 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_metadata' created 2024-04-25T10:30:30.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.391+0000 7f46ed7d5640 1 -- 172.21.15.106:0/3915989105 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f46d40744b0 msgr2=0x7f46d40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:30.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.391+0000 7f46ed7d5640 1 --2- 172.21.15.106:0/3915989105 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f46d40744b0 0x7f46d40769c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f46dc006fd0 tx=0x7f46dc008040 comp rx=0 tx=0).stop 2024-04-25T10:30:30.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.391+0000 7f46ed7d5640 1 -- 172.21.15.106:0/3915989105 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46e806f2e0 msgr2=0x7f46e81f34c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:30.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.391+0000 7f46ed7d5640 1 --2- 172.21.15.106:0/3915989105 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46e806f2e0 0x7f46e81f34c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f46d8034040 tx=0x7f46d8004a40 comp rx=0 tx=0).stop 2024-04-25T10:30:30.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.391+0000 7f46ed7d5640 1 -- 172.21.15.106:0/3915989105 shutdown_connections 2024-04-25T10:30:30.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.391+0000 7f46ed7d5640 1 --2- 172.21.15.106:0/3915989105 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f46d40744b0 0x7f46d40769c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:30.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.391+0000 7f46ed7d5640 1 --2- 172.21.15.106:0/3915989105 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f46e8156f00 0x7f46e8150950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:30.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.391+0000 7f46ed7d5640 1 --2- 172.21.15.106:0/3915989105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46e80d0790 0x7f46e8151d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:30.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.391+0000 7f46ed7d5640 1 --2- 172.21.15.106:0/3915989105 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46e806f2e0 0x7f46e81f34c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:30.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.391+0000 7f46ed7d5640 1 -- 172.21.15.106:0/3915989105 >> 172.21.15.106:0/3915989105 conn(0x7f46e80684e0 msgr2=0x7f46e815a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:30.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.391+0000 7f46ed7d5640 1 -- 172.21.15.106:0/3915989105 shutdown_connections 2024-04-25T10:30:30.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.391+0000 7f46ed7d5640 1 -- 172.21.15.106:0/3915989105 wait complete. 2024-04-25T10:30:30.408 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:30:30.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.519+0000 7f9eba405640 1 Processor -- start 2024-04-25T10:30:30.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.519+0000 7f9eba405640 1 -- start start 2024-04-25T10:30:30.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.521+0000 7f9eba405640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f9eb4156f70 0x7f9eb415ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:30.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.521+0000 7f9eba405640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9eb415d340 0x7f9eb4167680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:30.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.521+0000 7f9eba405640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9eb4073980 con 0x7f9eb40ca170 2024-04-25T10:30:30.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.521+0000 7f9eba405640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f9eb4073f00 con 0x7f9eb4156f70 2024-04-25T10:30:30.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.521+0000 7f9eba405640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f9eb4074990 con 0x7f9eb415d340 2024-04-25T10:30:30.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.521+0000 7f9eb89a2640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9eb415d340 0x7f9eb4167680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:30.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.521+0000 7f9eb37fe640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f9eb4156f70 0x7f9eb415ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:30.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.521+0000 7f9eb37fe640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f9eb4156f70 0x7f9eb415ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.106:41056/0 (socket says 172.21.15.106:41056) 2024-04-25T10:30:30.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.521+0000 7f9eb37fe640 1 -- 172.21.15.106:0/2588792391 learned_addr learned my addr 172.21.15.106:0/2588792391 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:30.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.522+0000 7f9eb89a2640 1 -- 172.21.15.106:0/2588792391 >> v2:172.21.15.134:3300/0 conn(0x7f9eb4156f70 msgr2=0x7f9eb415ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:30.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.522+0000 7f9eb89a2640 1 --2- 172.21.15.106:0/2588792391 >> v2:172.21.15.134:3300/0 conn(0x7f9eb4156f70 0x7f9eb415ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:30.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.522+0000 7f9eb89a2640 1 -- 172.21.15.106:0/2588792391 >> v1:172.21.15.134:6789/0 conn(0x7f9eb40ca170 legacy=0x7f9eb406c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:30.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.522+0000 7f9eb89a2640 1 -- 172.21.15.106:0/2588792391 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eb4160ea0 con 0x7f9eb415d340 2024-04-25T10:30:30.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.522+0000 7f9eb37fe640 1 --2- 172.21.15.106:0/2588792391 >> v2:172.21.15.134:3300/0 conn(0x7f9eb4156f70 0x7f9eb415ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:30.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.522+0000 7f9eb89a2640 1 --2- 172.21.15.106:0/2588792391 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9eb415d340 0x7f9eb4167680 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4074b10 tx=0x7f9eac02a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=119edb4b62235a0d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:30.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.523+0000 7f9eb2ffd640 1 -- 172.21.15.106:0/2588792391 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eac007740 con 0x7f9eb415d340 2024-04-25T10:30:30.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.523+0000 7f9eb2ffd640 1 -- 172.21.15.106:0/2588792391 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9eac0078e0 con 0x7f9eb415d340 2024-04-25T10:30:30.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.523+0000 7f9eb2ffd640 1 -- 172.21.15.106:0/2588792391 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eac007cb0 con 0x7f9eb415d340 2024-04-25T10:30:30.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.523+0000 7f9eba405640 1 -- 172.21.15.106:0/2588792391 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9eb415d340 msgr2=0x7f9eb4167680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:30.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.523+0000 7f9eba405640 1 --2- 172.21.15.106:0/2588792391 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9eb415d340 0x7f9eb4167680 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4074b10 tx=0x7f9eac02a6d0 comp rx=0 tx=0).stop 2024-04-25T10:30:30.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.523+0000 7f9eba405640 1 -- 172.21.15.106:0/2588792391 shutdown_connections 2024-04-25T10:30:30.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.523+0000 7f9eba405640 1 --2- 172.21.15.106:0/2588792391 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9eb415d340 0x7f9eb4167680 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:30.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.523+0000 7f9eba405640 1 --2- 172.21.15.106:0/2588792391 >> v2:172.21.15.134:3300/0 conn(0x7f9eb4156f70 0x7f9eb415ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:30.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.523+0000 7f9eba405640 1 -- 172.21.15.106:0/2588792391 >> 172.21.15.106:0/2588792391 conn(0x7f9eb40cef20 msgr2=0x7f9eb40cf330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:30.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.523+0000 7f9eba405640 1 -- 172.21.15.106:0/2588792391 shutdown_connections 2024-04-25T10:30:30.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.523+0000 7f9eba405640 1 -- 172.21.15.106:0/2588792391 wait complete. 2024-04-25T10:30:30.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.525+0000 7f9eba405640 1 Processor -- start 2024-04-25T10:30:30.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.525+0000 7f9eba405640 1 -- start start 2024-04-25T10:30:30.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.526+0000 7f9eba405640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9eb40ca170 0x7f9eb40bf950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:30.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.526+0000 7f9eba405640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9eb4156f70 0x7f9eb4152a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:30.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.526+0000 7f9eba405640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9eb415d340 0x7f9eb41f2d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:30.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.526+0000 7f9eba405640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb4073c20 con 0x7f9eb4156f70 2024-04-25T10:30:30.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.526+0000 7f9eba405640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb4074640 con 0x7f9eb40ca170 2024-04-25T10:30:30.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.526+0000 7f9eba405640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f9eb4161140 con 0x7f9eb415d340 2024-04-25T10:30:30.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.527+0000 7f9eb37fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9eb4156f70 0x7f9eb4152a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:30.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.527+0000 7f9eb37fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9eb4156f70 0x7f9eb4152a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58002/0 (socket says 172.21.15.106:58002) 2024-04-25T10:30:30.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.527+0000 7f9eb37fe640 1 -- 172.21.15.106:0/352130431 learned_addr learned my addr 172.21.15.106:0/352130431 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:30.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.527+0000 7f9eb89a2640 1 --2- 172.21.15.106:0/352130431 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9eb415d340 0x7f9eb41f2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:30.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.527+0000 7f9eb3fff640 1 --2- 172.21.15.106:0/352130431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9eb40ca170 0x7f9eb40bf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:30.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.527+0000 7f9eb37fe640 1 -- 172.21.15.106:0/352130431 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9eb415d340 msgr2=0x7f9eb41f2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:30.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.527+0000 7f9eb37fe640 1 --2- 172.21.15.106:0/352130431 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9eb415d340 0x7f9eb41f2d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:30.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.527+0000 7f9eb37fe640 1 -- 172.21.15.106:0/352130431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9eb40ca170 msgr2=0x7f9eb40bf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:30.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.527+0000 7f9eb37fe640 1 --2- 172.21.15.106:0/352130431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9eb40ca170 0x7f9eb40bf950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:30.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.527+0000 7f9eb37fe640 1 -- 172.21.15.106:0/352130431 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eb4152fb0 con 0x7f9eb4156f70 2024-04-25T10:30:30.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.527+0000 7f9eb37fe640 1 --2- 172.21.15.106:0/352130431 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9eb4156f70 0x7f9eb4152a70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9ea800ea10 tx=0x7f9ea800eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:30.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.527+0000 7f9eb0ff9640 1 -- 172.21.15.106:0/352130431 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea8017070 con 0x7f9eb4156f70 2024-04-25T10:30:30.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.527+0000 7f9eb0ff9640 1 -- 172.21.15.106:0/352130431 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ea800c9a0 con 0x7f9eb4156f70 2024-04-25T10:30:30.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.527+0000 7f9eba405640 1 -- 172.21.15.106:0/352130431 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9eb41f3260 con 0x7f9eb4156f70 2024-04-25T10:30:30.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.527+0000 7f9eb0ff9640 1 -- 172.21.15.106:0/352130431 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea800cd50 con 0x7f9eb4156f70 2024-04-25T10:30:30.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.527+0000 7f9eba405640 1 -- 172.21.15.106:0/352130431 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9eb4157970 con 0x7f9eb4156f70 2024-04-25T10:30:30.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.528+0000 7f9eba405640 1 -- 172.21.15.106:0/352130431 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9eb406a3d0 con 0x7f9eb4156f70 2024-04-25T10:30:30.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.529+0000 7f9eb0ff9640 1 -- 172.21.15.106:0/352130431 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9ea801a070 con 0x7f9eb4156f70 2024-04-25T10:30:30.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.529+0000 7f9eb0ff9640 1 --2- 172.21.15.106:0/352130431 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f9ea00744e0 0x7f9ea00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:30.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.530+0000 7f9eb3fff640 1 --2- 172.21.15.106:0/352130431 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f9ea00744e0 0x7f9ea00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:30.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.530+0000 7f9eb0ff9640 1 -- 172.21.15.106:0/352130431 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 5227+0+0 (secure 0 0 0) 0x7f9ea8094e00 con 0x7f9eb4156f70 2024-04-25T10:30:30.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.530+0000 7f9eb3fff640 1 --2- 172.21.15.106:0/352130431 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f9ea00744e0 0x7f9ea00769f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4007fd0 tx=0x7f9ea4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:30.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.534+0000 7f9eb0ff9640 1 -- 172.21.15.106:0/352130431 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9ea805f800 con 0x7f9eb4156f70 2024-04-25T10:30:30.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:30.711+0000 7f9eba405640 1 -- 172.21.15.106:0/352130431 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f9eb4074420 con 0x7f9eb4156f70 2024-04-25T10:30:31.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:31.391+0000 7f9eb0ff9640 1 -- 172.21.15.106:0/352130431 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) ==== 170+0+0 (secure 0 0 0) 0x7f9ea805f100 con 0x7f9eb4156f70 2024-04-25T10:30:31.393 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_data' created 2024-04-25T10:30:31.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:31.394+0000 7f9eba405640 1 -- 172.21.15.106:0/352130431 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f9ea00744e0 msgr2=0x7f9ea00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:31.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:31.394+0000 7f9eba405640 1 --2- 172.21.15.106:0/352130431 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f9ea00744e0 0x7f9ea00769f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4007fd0 tx=0x7f9ea4009040 comp rx=0 tx=0).stop 2024-04-25T10:30:31.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:31.394+0000 7f9eba405640 1 -- 172.21.15.106:0/352130431 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9eb4156f70 msgr2=0x7f9eb4152a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:31.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:31.394+0000 7f9eba405640 1 --2- 172.21.15.106:0/352130431 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9eb4156f70 0x7f9eb4152a70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9ea800ea10 tx=0x7f9ea800eed0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:31.395+0000 7f9eba405640 1 -- 172.21.15.106:0/352130431 shutdown_connections 2024-04-25T10:30:31.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:31.395+0000 7f9eba405640 1 --2- 172.21.15.106:0/352130431 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f9ea00744e0 0x7f9ea00769f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:31.395+0000 7f9eba405640 1 --2- 172.21.15.106:0/352130431 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9eb415d340 0x7f9eb41f2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:31.395+0000 7f9eba405640 1 --2- 172.21.15.106:0/352130431 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9eb4156f70 0x7f9eb4152a70 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:31.395+0000 7f9eba405640 1 --2- 172.21.15.106:0/352130431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9eb40ca170 0x7f9eb40bf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:31.395+0000 7f9eba405640 1 -- 172.21.15.106:0/352130431 >> 172.21.15.106:0/352130431 conn(0x7f9eb40cef20 msgr2=0x7f9eb41600e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:31.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:31.395+0000 7f9eba405640 1 -- 172.21.15.106:0/352130431 shutdown_connections 2024-04-25T10:30:31.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:31.395+0000 7f9eba405640 1 -- 172.21.15.106:0/352130431 wait complete. 2024-04-25T10:30:31.422 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:30:31.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:31.531+0000 7effd6092640 1 Processor -- start 2024-04-25T10:30:31.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:31.531+0000 7effd6092640 1 -- start start 2024-04-25T10:30:31.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:31.532+0000 7effd6092640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7effd00cb2b0 0x7effd00cce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:31.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:31.532+0000 7effd6092640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effd015b8c0 con 0x7effd00cc600 2024-04-25T10:30:31.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:31.532+0000 7effd6092640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effd015b5e0 con 0x7effd0154ac0 2024-04-25T10:30:31.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:31.532+0000 7effd6092640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7effd015eea0 con 0x7effd00cb2b0 2024-04-25T10:30:31.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:31.533+0000 7effcf7fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7effd00cc600 0x7effd00c5390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:51298/0 (socket says 172.21.15.106:51298) 2024-04-25T10:30:31.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:31.533+0000 7effcf7fe640 1 -- 172.21.15.106:0/1351252600 learned_addr learned my addr 172.21.15.106:0/1351252600 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:30:31.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:31.533+0000 7effceffd640 1 --2- 172.21.15.106:0/1351252600 >> v2:172.21.15.106:3300/0 conn(0x7effd00cb2b0 0x7effd00cce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:31.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:31.533+0000 7effceffd640 1 -- 172.21.15.106:0/1351252600 >> v1:172.21.15.134:6789/0 conn(0x7effd0154ac0 legacy=0x7effd0165340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:31.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:31.533+0000 7effceffd640 1 -- 172.21.15.106:0/1351252600 >> v1:172.21.15.106:6789/0 conn(0x7effd00cc600 legacy=0x7effd00c5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:31.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:31.533+0000 7effceffd640 1 -- 172.21.15.106:0/1351252600 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effd015e0e0 con 0x7effd00cb2b0 2024-04-25T10:30:31.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:31.533+0000 7effce7fc640 1 -- 172.21.15.106:0/1351252600 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4022733643 0 0) 0x7effd015b8c0 con 0x7effd00cc600 2024-04-25T10:30:31.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:31.534+0000 7effceffd640 1 --2- 172.21.15.106:0/1351252600 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7effd00cb2b0 0x7effd00cce90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7effc40097a0 tx=0x7effc402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=282fdb9a9b3636ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:31.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:31.534+0000 7effce7fc640 1 -- 172.21.15.106:0/1351252600 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc4037070 con 0x7effd00cb2b0 2024-04-25T10:30:31.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:31.534+0000 7effce7fc640 1 -- 172.21.15.106:0/1351252600 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effc4032440 con 0x7effd00cb2b0 2024-04-25T10:30:31.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:31.534+0000 7effce7fc640 1 -- 172.21.15.106:0/1351252600 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc4032810 con 0x7effd00cb2b0 2024-04-25T10:30:31.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:31.534+0000 7effd6092640 1 -- 172.21.15.106:0/1351252600 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7effd00cb2b0 msgr2=0x7effd00cce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:31.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:31.534+0000 7effd6092640 1 --2- 172.21.15.106:0/1351252600 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7effd00cb2b0 0x7effd00cce90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7effc40097a0 tx=0x7effc402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:31.534+0000 7effd6092640 1 -- 172.21.15.106:0/1351252600 shutdown_connections 2024-04-25T10:30:31.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:31.534+0000 7effd6092640 1 --2- 172.21.15.106:0/1351252600 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7effd00cb2b0 0x7effd00cce90 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:31.534+0000 7effd6092640 1 -- 172.21.15.106:0/1351252600 >> 172.21.15.106:0/1351252600 conn(0x7effd00727d0 msgr2=0x7effd0072c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:31.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:31.534+0000 7effd6092640 1 -- 172.21.15.106:0/1351252600 shutdown_connections 2024-04-25T10:30:31.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:31.534+0000 7effd6092640 1 -- 172.21.15.106:0/1351252600 wait complete. 2024-04-25T10:30:31.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:31.536+0000 7effd6092640 1 Processor -- start 2024-04-25T10:30:31.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:31.536+0000 7effd6092640 1 -- start start 2024-04-25T10:30:31.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:31.537+0000 7effd6092640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7effd00cb2b0 0x7effd00cf850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:31.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:31.537+0000 7effd6092640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7effd00cc600 0x7effd00c8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:31.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:31.537+0000 7effd6092640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7effd0154ac0 0x7effd01f4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:31.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:31.537+0000 7effd6092640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7effd015e980 con 0x7effd00cc600 2024-04-25T10:30:31.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:31.537+0000 7effd6092640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7effd015e6a0 con 0x7effd0154ac0 2024-04-25T10:30:31.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:31.537+0000 7effd6092640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7effd00cbff0 con 0x7effd00cb2b0 2024-04-25T10:30:31.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:31.538+0000 7effceffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7effd00cc600 0x7effd00c8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:31.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:31.538+0000 7effceffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7effd00cc600 0x7effd00c8520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58032/0 (socket says 172.21.15.106:58032) 2024-04-25T10:30:31.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:31.538+0000 7effceffd640 1 -- 172.21.15.106:0/884440593 learned_addr learned my addr 172.21.15.106:0/884440593 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:31.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:31.538+0000 7effceffd640 1 -- 172.21.15.106:0/884440593 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7effd00cb2b0 msgr2=0x7effd00cf850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:31.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:31.538+0000 7effcf7fe640 1 --2- 172.21.15.106:0/884440593 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7effd00cb2b0 0x7effd00cf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:31.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:31.538+0000 7effcffff640 1 --2- 172.21.15.106:0/884440593 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7effd0154ac0 0x7effd01f4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:31.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:31.538+0000 7effceffd640 1 --2- 172.21.15.106:0/884440593 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7effd00cb2b0 0x7effd00cf850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:31.538+0000 7effceffd640 1 -- 172.21.15.106:0/884440593 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7effd0154ac0 msgr2=0x7effd01f4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:31.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:31.538+0000 7effceffd640 1 --2- 172.21.15.106:0/884440593 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7effd0154ac0 0x7effd01f4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:31.538+0000 7effceffd640 1 -- 172.21.15.106:0/884440593 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effd01f4cc0 con 0x7effd00cc600 2024-04-25T10:30:31.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:31.538+0000 7effcf7fe640 1 --2- 172.21.15.106:0/884440593 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7effd00cb2b0 0x7effd00cf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:31.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:31.538+0000 7effcffff640 1 --2- 172.21.15.106:0/884440593 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7effd0154ac0 0x7effd01f4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:31.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:31.538+0000 7effceffd640 1 --2- 172.21.15.106:0/884440593 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7effd00cc600 0x7effd00c8520 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7effc40098c0 tx=0x7effc40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:31.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:31.538+0000 7effb7fff640 1 -- 172.21.15.106:0/884440593 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc4041070 con 0x7effd00cc600 2024-04-25T10:30:31.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:31.538+0000 7effb7fff640 1 -- 172.21.15.106:0/884440593 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effc403c870 con 0x7effd00cc600 2024-04-25T10:30:31.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:31.538+0000 7effd6092640 1 -- 172.21.15.106:0/884440593 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effd01f1940 con 0x7effd00cc600 2024-04-25T10:30:31.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:31.538+0000 7effb7fff640 1 -- 172.21.15.106:0/884440593 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc4037040 con 0x7effd00cc600 2024-04-25T10:30:31.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:31.538+0000 7effd6092640 1 -- 172.21.15.106:0/884440593 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effd01f4ed0 con 0x7effd00cc600 2024-04-25T10:30:31.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:31.539+0000 7effd6092640 1 -- 172.21.15.106:0/884440593 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effd01f16b0 con 0x7effd00cc600 2024-04-25T10:30:31.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:31.540+0000 7effb7fff640 1 -- 172.21.15.106:0/884440593 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7effc4044070 con 0x7effd00cc600 2024-04-25T10:30:31.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:31.540+0000 7effb7fff640 1 --2- 172.21.15.106:0/884440593 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7effbc0744b0 0x7effbc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:31.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:31.541+0000 7effb7fff640 1 -- 172.21.15.106:0/884440593 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5609+0+0 (secure 0 0 0) 0x7effc40b5d10 con 0x7effd00cc600 2024-04-25T10:30:31.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:31.541+0000 7effcf7fe640 1 --2- 172.21.15.106:0/884440593 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7effbc0744b0 0x7effbc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:31.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:31.541+0000 7effcf7fe640 1 --2- 172.21.15.106:0/884440593 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7effbc0744b0 0x7effbc0769c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7effc0000f60 tx=0x7effc0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:31.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:31.545+0000 7effb7fff640 1 -- 172.21.15.106:0/884440593 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7effc4080700 con 0x7effd00cc600 2024-04-25T10:30:31.653 DEBUG:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:30:31.655 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:30:31.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:31.759+0000 7effd6092640 1 -- 172.21.15.106:0/884440593 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7effd006a3d0 con 0x7effd00cc600 2024-04-25T10:30:32.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.423+0000 7effb7fff640 1 -- 172.21.15.106:0/884440593 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7effc4080000 con 0x7effd00cc600 2024-04-25T10:30:32.425 INFO:teuthology.orchestra.run.smithi106.stderr:new fs with metadata pool 2 and data pool 3 2024-04-25T10:30:32.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.427+0000 7effd6092640 1 -- 172.21.15.106:0/884440593 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7effbc0744b0 msgr2=0x7effbc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:32.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.427+0000 7effd6092640 1 --2- 172.21.15.106:0/884440593 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7effbc0744b0 0x7effbc0769c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7effc0000f60 tx=0x7effc0009040 comp rx=0 tx=0).stop 2024-04-25T10:30:32.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.427+0000 7effd6092640 1 -- 172.21.15.106:0/884440593 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7effd00cc600 msgr2=0x7effd00c8520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:32.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.427+0000 7effd6092640 1 --2- 172.21.15.106:0/884440593 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7effd00cc600 0x7effd00c8520 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7effc40098c0 tx=0x7effc40046c0 comp rx=0 tx=0).stop 2024-04-25T10:30:32.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.427+0000 7effd6092640 1 -- 172.21.15.106:0/884440593 shutdown_connections 2024-04-25T10:30:32.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.427+0000 7effd6092640 1 --2- 172.21.15.106:0/884440593 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7effbc0744b0 0x7effbc0769c0 secure :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7effc0000f60 tx=0x7effc0009040 comp rx=0 tx=0).stop 2024-04-25T10:30:32.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.427+0000 7effd6092640 1 --2- 172.21.15.106:0/884440593 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7effd0154ac0 0x7effd01f4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:32.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.427+0000 7effd6092640 1 --2- 172.21.15.106:0/884440593 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7effd00cc600 0x7effd00c8520 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:32.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.427+0000 7effd6092640 1 --2- 172.21.15.106:0/884440593 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7effd00cb2b0 0x7effd00cf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:32.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.427+0000 7effd6092640 1 -- 172.21.15.106:0/884440593 >> 172.21.15.106:0/884440593 conn(0x7effd00727d0 msgr2=0x7effd00b4fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:32.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.428+0000 7effd6092640 1 -- 172.21.15.106:0/884440593 shutdown_connections 2024-04-25T10:30:32.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.428+0000 7effd6092640 1 -- 172.21.15.106:0/884440593 wait complete. 2024-04-25T10:30:32.449 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:30:32.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.559+0000 7f06d334d640 1 Processor -- start 2024-04-25T10:30:32.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.559+0000 7f06d334d640 1 -- start start 2024-04-25T10:30:32.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.560+0000 7f06d334d640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f06cc074910 0x7f06cc0c9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:32.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.560+0000 7f06d334d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f06cc157080 0x7f06cc15cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:32.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.560+0000 7f06d334d640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06cc070af0 con 0x7f06cc0684e0 2024-04-25T10:30:32.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.560+0000 7f06d334d640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f06cc0733a0 con 0x7f06cc074910 2024-04-25T10:30:32.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.560+0000 7f06d334d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f06cc0708a0 con 0x7f06cc157080 2024-04-25T10:30:32.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.561+0000 7f06d10e9640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f06cc074910 0x7f06cc0c9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:32.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.561+0000 7f06d08e8640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f06cc157080 0x7f06cc15cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:32.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.561+0000 7f06d10e9640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f06cc074910 0x7f06cc0c9100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.106:41070/0 (socket says 172.21.15.106:41070) 2024-04-25T10:30:32.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.561+0000 7f06d08e8640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f06cc157080 0x7f06cc15cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.106:43478/0 (socket says 172.21.15.106:43478) 2024-04-25T10:30:32.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.561+0000 7f06d10e9640 1 -- 172.21.15.106:0/3492996783 learned_addr learned my addr 172.21.15.106:0/3492996783 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:32.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.561+0000 7f06d08e8640 1 -- 172.21.15.106:0/3492996783 >> v2:172.21.15.134:3300/0 conn(0x7f06cc074910 msgr2=0x7f06cc0c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:32.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.561+0000 7f06d08e8640 1 --2- 172.21.15.106:0/3492996783 >> v2:172.21.15.134:3300/0 conn(0x7f06cc074910 0x7f06cc0c9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:32.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.561+0000 7f06d08e8640 1 -- 172.21.15.106:0/3492996783 >> v1:172.21.15.134:6789/0 conn(0x7f06cc0684e0 legacy=0x7f06cc0c9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:32.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.561+0000 7f06d08e8640 1 -- 172.21.15.106:0/3492996783 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06cc070dd0 con 0x7f06cc157080 2024-04-25T10:30:32.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.561+0000 7f06d10e9640 1 --2- 172.21.15.106:0/3492996783 >> v2:172.21.15.134:3300/0 conn(0x7f06cc074910 0x7f06cc0c9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:32.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.562+0000 7f06d08e8640 1 --2- 172.21.15.106:0/3492996783 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f06cc157080 0x7f06cc15cc30 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f06bc0097a0 tx=0x7f06bc02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4de97ca30046c6a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:32.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.562+0000 7f06c3fff640 1 -- 172.21.15.106:0/3492996783 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06bc037070 con 0x7f06cc157080 2024-04-25T10:30:32.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.562+0000 7f06c3fff640 1 -- 172.21.15.106:0/3492996783 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06bc032440 con 0x7f06cc157080 2024-04-25T10:30:32.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.562+0000 7f06c3fff640 1 -- 172.21.15.106:0/3492996783 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06bc032810 con 0x7f06cc157080 2024-04-25T10:30:32.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.563+0000 7f06d334d640 1 -- 172.21.15.106:0/3492996783 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f06cc157080 msgr2=0x7f06cc15cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:32.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.563+0000 7f06d334d640 1 --2- 172.21.15.106:0/3492996783 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f06cc157080 0x7f06cc15cc30 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f06bc0097a0 tx=0x7f06bc02a180 comp rx=0 tx=0).stop 2024-04-25T10:30:32.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.563+0000 7f06d334d640 1 -- 172.21.15.106:0/3492996783 shutdown_connections 2024-04-25T10:30:32.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.563+0000 7f06d334d640 1 --2- 172.21.15.106:0/3492996783 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f06cc157080 0x7f06cc15cc30 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:32.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.563+0000 7f06d334d640 1 --2- 172.21.15.106:0/3492996783 >> v2:172.21.15.134:3300/0 conn(0x7f06cc074910 0x7f06cc0c9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:32.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.563+0000 7f06d334d640 1 -- 172.21.15.106:0/3492996783 >> 172.21.15.106:0/3492996783 conn(0x7f06cc0ce570 msgr2=0x7f06cc0ce980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:32.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.563+0000 7f06d334d640 1 -- 172.21.15.106:0/3492996783 shutdown_connections 2024-04-25T10:30:32.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.563+0000 7f06d334d640 1 -- 172.21.15.106:0/3492996783 wait complete. 2024-04-25T10:30:32.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.564+0000 7f06d334d640 1 Processor -- start 2024-04-25T10:30:32.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.565+0000 7f06d334d640 1 -- start start 2024-04-25T10:30:32.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.566+0000 7f06d334d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f06cc0684e0 0x7f06cc1f34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:32.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.566+0000 7f06d334d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06cc074910 0x7f06cc14fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:32.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.566+0000 7f06d334d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06cc157080 0x7f06cc150400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:32.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.566+0000 7f06d334d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f06cc074110 con 0x7f06cc157080 2024-04-25T10:30:32.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.566+0000 7f06d334d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f06cc073ec0 con 0x7f06cc074910 2024-04-25T10:30:32.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.566+0000 7f06d334d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f06cc0743f0 con 0x7f06cc0684e0 2024-04-25T10:30:32.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.566+0000 7f06d18ea640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06cc157080 0x7f06cc150400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:32.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.566+0000 7f06d18ea640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06cc157080 0x7f06cc150400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58034/0 (socket says 172.21.15.106:58034) 2024-04-25T10:30:32.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.566+0000 7f06d18ea640 1 -- 172.21.15.106:0/3436160645 learned_addr learned my addr 172.21.15.106:0/3436160645 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:32.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.566+0000 7f06d10e9640 1 --2- 172.21.15.106:0/3436160645 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f06cc0684e0 0x7f06cc1f34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:32.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.566+0000 7f06d08e8640 1 --2- 172.21.15.106:0/3436160645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06cc074910 0x7f06cc14fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:32.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.566+0000 7f06d18ea640 1 -- 172.21.15.106:0/3436160645 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f06cc0684e0 msgr2=0x7f06cc1f34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:32.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.566+0000 7f06d18ea640 1 --2- 172.21.15.106:0/3436160645 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f06cc0684e0 0x7f06cc1f34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:32.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.566+0000 7f06d18ea640 1 -- 172.21.15.106:0/3436160645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06cc074910 msgr2=0x7f06cc14fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:32.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.566+0000 7f06d18ea640 1 --2- 172.21.15.106:0/3436160645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06cc074910 0x7f06cc14fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:32.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.566+0000 7f06d18ea640 1 -- 172.21.15.106:0/3436160645 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06cc0c89f0 con 0x7f06cc157080 2024-04-25T10:30:32.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.566+0000 7f06d18ea640 1 --2- 172.21.15.106:0/3436160645 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06cc157080 0x7f06cc150400 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f06c400c790 tx=0x7f06c400cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:32.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.566+0000 7f06c1ffb640 1 -- 172.21.15.106:0/3436160645 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c4003b70 con 0x7f06cc157080 2024-04-25T10:30:32.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.567+0000 7f06c1ffb640 1 -- 172.21.15.106:0/3436160645 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06c4003d10 con 0x7f06cc157080 2024-04-25T10:30:32.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.567+0000 7f06c1ffb640 1 -- 172.21.15.106:0/3436160645 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c400aa90 con 0x7f06cc157080 2024-04-25T10:30:32.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.567+0000 7f06d334d640 1 -- 172.21.15.106:0/3436160645 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06cc1f39f0 con 0x7f06cc157080 2024-04-25T10:30:32.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.567+0000 7f06d334d640 1 -- 172.21.15.106:0/3436160645 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06cc1f2cd0 con 0x7f06cc157080 2024-04-25T10:30:32.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.568+0000 7f06d334d640 1 -- 172.21.15.106:0/3436160645 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06cc1586f0 con 0x7f06cc157080 2024-04-25T10:30:32.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.569+0000 7f06c1ffb640 1 -- 172.21.15.106:0/3436160645 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f06c400ac30 con 0x7f06cc157080 2024-04-25T10:30:32.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.569+0000 7f06c1ffb640 1 --2- 172.21.15.106:0/3436160645 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f06b00744e0 0x7f06b00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:32.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.569+0000 7f06c1ffb640 1 -- 172.21.15.106:0/3436160645 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f06c4094e70 con 0x7f06cc157080 2024-04-25T10:30:32.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.569+0000 7f06d10e9640 1 --2- 172.21.15.106:0/3436160645 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f06b00744e0 0x7f06b00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:32.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.570+0000 7f06d10e9640 1 --2- 172.21.15.106:0/3436160645 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f06b00744e0 0x7f06b00769f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f06c8006ca0 tx=0x7f06c8009240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:32.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.574+0000 7f06c1ffb640 1 -- 172.21.15.106:0/3436160645 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f06c405f040 con 0x7f06cc157080 2024-04-25T10:30:32.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.748+0000 7f06d334d640 1 -- 172.21.15.106:0/3436160645 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f06cc06a3d0 con 0x7f06cc157080 2024-04-25T10:30:32.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.749+0000 7f06c1ffb640 1 -- 172.21.15.106:0/3436160645 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16654 (secure 0 0 0) 0x7f06c4063340 con 0x7f06cc157080 2024-04-25T10:30:32.752 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:30:32.752 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":19,"fsid":"3ee77985-e8e4-41cf-81e9-7c774cf7891f","created":"2024-04-25T10:30:07.981054+0000","modified":"2024-04-25T10:30:31.761147+0000","last_up_change":"2024-04-25T10:30:19.184325+0000","last_in_change":"2024-04-25T10:30:10.217553+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:30:20.111046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:30:30.314914+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:30:30.715500+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a5962cb1-dbbd-47b1-8d15-9bae114bfdf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1240717079},{"type":"v1","addr":"172.21.15.106:6801","nonce":1240717079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1240717079},{"type":"v1","addr":"172.21.15.106:6803","nonce":1240717079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1240717079},{"type":"v1","addr":"172.21.15.106:6807","nonce":1240717079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1240717079},{"type":"v1","addr":"172.21.15.106:6805","nonce":1240717079}]},"public_addr":"172.21.15.106:6801/1240717079","cluster_addr":"172.21.15.106:6803/1240717079","heartbeat_back_addr":"172.21.15.106:6807/1240717079","heartbeat_front_addr":"172.21.15.106:6805/1240717079","state":["exists","up"]},{"osd":1,"uuid":"94436e13-4769-4752-8397-b5e396a6f7ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4089653051},{"type":"v1","addr":"172.21.15.106:6825","nonce":4089653051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4089653051},{"type":"v1","addr":"172.21.15.106:6827","nonce":4089653051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4089653051},{"type":"v1","addr":"172.21.15.106:6831","nonce":4089653051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4089653051},{"type":"v1","addr":"172.21.15.106:6829","nonce":4089653051}]},"public_addr":"172.21.15.106:6825/4089653051","cluster_addr":"172.21.15.106:6827/4089653051","heartbeat_back_addr":"172.21.15.106:6831/4089653051","heartbeat_front_addr":"172.21.15.106:6829/4089653051","state":["exists","up"]},{"osd":2,"uuid":"429aae0b-70c6-4e7b-9043-6a11334c7efc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":1441018239},{"type":"v1","addr":"172.21.15.106:6809","nonce":1441018239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1441018239},{"type":"v1","addr":"172.21.15.106:6811","nonce":1441018239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1441018239},{"type":"v1","addr":"172.21.15.106:6815","nonce":1441018239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":1441018239},{"type":"v1","addr":"172.21.15.106:6813","nonce":1441018239}]},"public_addr":"172.21.15.106:6809/1441018239","cluster_addr":"172.21.15.106:6811/1441018239","heartbeat_back_addr":"172.21.15.106:6815/1441018239","heartbeat_front_addr":"172.21.15.106:6813/1441018239","state":["exists","up"]},{"osd":3,"uuid":"9593a4d1-772f-4667-ab0d-2982f841e342","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":1846968711},{"type":"v1","addr":"172.21.15.106:6817","nonce":1846968711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1846968711},{"type":"v1","addr":"172.21.15.106:6819","nonce":1846968711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":1846968711},{"type":"v1","addr":"172.21.15.106:6823","nonce":1846968711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":1846968711},{"type":"v1","addr":"172.21.15.106:6821","nonce":1846968711}]},"public_addr":"172.21.15.106:6817/1846968711","cluster_addr":"172.21.15.106:6819/1846968711","heartbeat_back_addr":"172.21.15.106:6823/1846968711","heartbeat_front_addr":"172.21.15.106:6821/1846968711","state":["exists","up"]},{"osd":4,"uuid":"1b61f16d-5c60-480a-b449-c745aa3f6b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":729224739},{"type":"v1","addr":"172.21.15.134:6825","nonce":729224739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":729224739},{"type":"v1","addr":"172.21.15.134:6827","nonce":729224739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":729224739},{"type":"v1","addr":"172.21.15.134:6831","nonce":729224739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":729224739},{"type":"v1","addr":"172.21.15.134:6829","nonce":729224739}]},"public_addr":"172.21.15.134:6825/729224739","cluster_addr":"172.21.15.134:6827/729224739","heartbeat_back_addr":"172.21.15.134:6831/729224739","heartbeat_front_addr":"172.21.15.134:6829/729224739","state":["exists","up"]},{"osd":5,"uuid":"944cdd7f-47c0-4b71-b7e0-d093542f4528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3132275652},{"type":"v1","addr":"172.21.15.134:6809","nonce":3132275652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3132275652},{"type":"v1","addr":"172.21.15.134:6811","nonce":3132275652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3132275652},{"type":"v1","addr":"172.21.15.134:6815","nonce":3132275652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3132275652},{"type":"v1","addr":"172.21.15.134:6813","nonce":3132275652}]},"public_addr":"172.21.15.134:6809/3132275652","cluster_addr":"172.21.15.134:6811/3132275652","heartbeat_back_addr":"172.21.15.134:6815/3132275652","heartbeat_front_addr":"172.21.15.134:6813/3132275652","state":["exists","up"]},{"osd":6,"uuid":"cc2446e2-1d21-4135-ac04-6af7c990058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":992702668},{"type":"v1","addr":"172.21.15.134:6801","nonce":992702668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":992702668},{"type":"v1","addr":"172.21.15.134:6803","nonce":992702668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":992702668},{"type":"v1","addr":"172.21.15.134:6807","nonce":992702668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":992702668},{"type":"v1","addr":"172.21.15.134:6805","nonce":992702668}]},"public_addr":"172.21.15.134:6801/992702668","cluster_addr":"172.21.15.134:6803/992702668","heartbeat_back_addr":"172.21.15.134:6807/992702668","heartbeat_front_addr":"172.21.15.134:6805/992702668","state":["exists","up"]},{"osd":7,"uuid":"3df1fc7a-82fc-4257-8629-98027b6e2aed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":3291062284},{"type":"v1","addr":"172.21.15.134:6817","nonce":3291062284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":3291062284},{"type":"v1","addr":"172.21.15.134:6819","nonce":3291062284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":3291062284},{"type":"v1","addr":"172.21.15.134:6823","nonce":3291062284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":3291062284},{"type":"v1","addr":"172.21.15.134:6821","nonce":3291062284}]},"public_addr":"172.21.15.134:6817/3291062284","cluster_addr":"172.21.15.134:6819/3291062284","heartbeat_back_addr":"172.21.15.134:6823/3291062284","heartbeat_front_addr":"172.21.15.134:6821/3291062284","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.403941+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.382563+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.411544+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.330444+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.405430+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.387737+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.408922+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.454450+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:30:32.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.751+0000 7f06d334d640 1 -- 172.21.15.106:0/3436160645 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f06b00744e0 msgr2=0x7f06b00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:32.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.751+0000 7f06d334d640 1 --2- 172.21.15.106:0/3436160645 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f06b00744e0 0x7f06b00769f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f06c8006ca0 tx=0x7f06c8009240 comp rx=0 tx=0).stop 2024-04-25T10:30:32.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.751+0000 7f06d334d640 1 -- 172.21.15.106:0/3436160645 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06cc157080 msgr2=0x7f06cc150400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:32.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.751+0000 7f06d334d640 1 --2- 172.21.15.106:0/3436160645 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06cc157080 0x7f06cc150400 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f06c400c790 tx=0x7f06c400cc50 comp rx=0 tx=0).stop 2024-04-25T10:30:32.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.751+0000 7f06d334d640 1 -- 172.21.15.106:0/3436160645 shutdown_connections 2024-04-25T10:30:32.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.751+0000 7f06d334d640 1 --2- 172.21.15.106:0/3436160645 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f06b00744e0 0x7f06b00769f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:32.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.751+0000 7f06d334d640 1 --2- 172.21.15.106:0/3436160645 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06cc157080 0x7f06cc150400 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:32.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.751+0000 7f06d334d640 1 --2- 172.21.15.106:0/3436160645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06cc074910 0x7f06cc14fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:32.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.751+0000 7f06d334d640 1 --2- 172.21.15.106:0/3436160645 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f06cc0684e0 0x7f06cc1f34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:32.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.751+0000 7f06d334d640 1 -- 172.21.15.106:0/3436160645 >> 172.21.15.106:0/3436160645 conn(0x7f06cc0ce570 msgr2=0x7f06cc154430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:32.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.751+0000 7f06d334d640 1 -- 172.21.15.106:0/3436160645 shutdown_connections 2024-04-25T10:30:32.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.751+0000 7f06d334d640 1 -- 172.21.15.106:0/3436160645 wait complete. 2024-04-25T10:30:32.765 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:30:32.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.868+0000 7f15fe9c7640 1 Processor -- start 2024-04-25T10:30:32.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.868+0000 7f15fe9c7640 1 -- start start 2024-04-25T10:30:32.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.870+0000 7f15fe9c7640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f15f80cbff0 0x7f15f80cb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:32.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.870+0000 7f15fe9c7640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f15f8156ec0 0x7f15f80c61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:32.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.870+0000 7f15fe9c7640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15f815dae0 con 0x7f15f80c66e0 2024-04-25T10:30:32.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.870+0000 7f15fe9c7640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f15f8161270 con 0x7f15f80cbff0 2024-04-25T10:30:32.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.870+0000 7f15fe9c7640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f15f81604b0 con 0x7f15f8156ec0 2024-04-25T10:30:32.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.870+0000 7f15f7fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f15f80cbff0 0x7f15f80cb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:32.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.870+0000 7f15fcf64640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f15f80c66e0 0x7f15f80cce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:51312/0 (socket says 172.21.15.106:51312) 2024-04-25T10:30:32.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.870+0000 7f15fcf64640 1 -- 172.21.15.106:0/754521183 learned_addr learned my addr 172.21.15.106:0/754521183 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:30:32.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.870+0000 7f15f77fe640 1 --2- 172.21.15.106:0/754521183 >> v2:172.21.15.134:3300/0 conn(0x7f15f8156ec0 0x7f15f80c61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:32.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.870+0000 7f15f7fff640 1 -- 172.21.15.106:0/754521183 >> v2:172.21.15.134:3300/0 conn(0x7f15f8156ec0 msgr2=0x7f15f80c61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:32.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.870+0000 7f15f7fff640 1 --2- 172.21.15.106:0/754521183 >> v2:172.21.15.134:3300/0 conn(0x7f15f8156ec0 0x7f15f80c61a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:32.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.870+0000 7f15f7fff640 1 -- 172.21.15.106:0/754521183 >> v1:172.21.15.106:6789/0 conn(0x7f15f80c66e0 legacy=0x7f15f80cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:32.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.870+0000 7f15f6ffd640 1 -- 172.21.15.106:0/754521183 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 644004523 0 0) 0x7f15f815dae0 con 0x7f15f80c66e0 2024-04-25T10:30:32.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.870+0000 7f15f7fff640 1 -- 172.21.15.106:0/754521183 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15f8160790 con 0x7f15f80cbff0 2024-04-25T10:30:32.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.871+0000 7f15f7fff640 1 --2- 172.21.15.106:0/754521183 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f15f80cbff0 0x7f15f80cb270 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f15f80749c0 tx=0x7f15e802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2db11f6a58474674 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:32.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.871+0000 7f15f6ffd640 1 -- 172.21.15.106:0/754521183 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e8037070 con 0x7f15f80cbff0 2024-04-25T10:30:32.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.871+0000 7f15f6ffd640 1 -- 172.21.15.106:0/754521183 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15e8032440 con 0x7f15f80cbff0 2024-04-25T10:30:32.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.871+0000 7f15f6ffd640 1 -- 172.21.15.106:0/754521183 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e8032810 con 0x7f15f80cbff0 2024-04-25T10:30:32.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.872+0000 7f15fe9c7640 1 -- 172.21.15.106:0/754521183 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f15f80cbff0 msgr2=0x7f15f80cb270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:32.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.872+0000 7f15fe9c7640 1 --2- 172.21.15.106:0/754521183 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f15f80cbff0 0x7f15f80cb270 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f15f80749c0 tx=0x7f15e802a230 comp rx=0 tx=0).stop 2024-04-25T10:30:32.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.872+0000 7f15fe9c7640 1 -- 172.21.15.106:0/754521183 shutdown_connections 2024-04-25T10:30:32.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.872+0000 7f15fe9c7640 1 --2- 172.21.15.106:0/754521183 >> v2:172.21.15.134:3300/0 conn(0x7f15f8156ec0 0x7f15f80c61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:32.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.872+0000 7f15fe9c7640 1 --2- 172.21.15.106:0/754521183 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f15f80cbff0 0x7f15f80cb270 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:32.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.872+0000 7f15fe9c7640 1 -- 172.21.15.106:0/754521183 >> 172.21.15.106:0/754521183 conn(0x7f15f80727d0 msgr2=0x7f15f8072c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:32.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.872+0000 7f15fe9c7640 1 -- 172.21.15.106:0/754521183 shutdown_connections 2024-04-25T10:30:32.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.872+0000 7f15fe9c7640 1 -- 172.21.15.106:0/754521183 wait complete. 2024-04-25T10:30:32.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.873+0000 7f15fe9c7640 1 Processor -- start 2024-04-25T10:30:32.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.874+0000 7f15fe9c7640 1 -- start start 2024-04-25T10:30:32.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.875+0000 7f15fe9c7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15f80c66e0 0x7f15f815a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:32.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.875+0000 7f15fe9c7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15f8156ec0 0x7f15f8152bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:32.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.875+0000 7f15fe9c7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f15f81592b0 0x7f15f815f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:32.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.875+0000 7f15fe9c7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f15f81572d0 con 0x7f15f81592b0 2024-04-25T10:30:32.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.875+0000 7f15fe9c7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f15f8160d50 con 0x7f15f8156ec0 2024-04-25T10:30:32.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.875+0000 7f15fe9c7640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f15f815d730 con 0x7f15f80c66e0 2024-04-25T10:30:32.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.875+0000 7f15fcf64640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f15f81592b0 0x7f15f815f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:32.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.875+0000 7f15fcf64640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f15f81592b0 0x7f15f815f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58052/0 (socket says 172.21.15.106:58052) 2024-04-25T10:30:32.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.875+0000 7f15fcf64640 1 -- 172.21.15.106:0/1730154609 learned_addr learned my addr 172.21.15.106:0/1730154609 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:32.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.875+0000 7f15f77fe640 1 --2- 172.21.15.106:0/1730154609 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15f8156ec0 0x7f15f8152bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:32.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.875+0000 7f15f7fff640 1 --2- 172.21.15.106:0/1730154609 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15f80c66e0 0x7f15f815a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:32.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.875+0000 7f15fcf64640 1 -- 172.21.15.106:0/1730154609 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15f80c66e0 msgr2=0x7f15f815a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:32.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.875+0000 7f15fcf64640 1 --2- 172.21.15.106:0/1730154609 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15f80c66e0 0x7f15f815a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:32.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.875+0000 7f15fcf64640 1 -- 172.21.15.106:0/1730154609 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15f8156ec0 msgr2=0x7f15f8152bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:32.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.875+0000 7f15fcf64640 1 --2- 172.21.15.106:0/1730154609 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15f8156ec0 0x7f15f8152bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:32.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.875+0000 7f15fcf64640 1 -- 172.21.15.106:0/1730154609 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15f8158210 con 0x7f15f81592b0 2024-04-25T10:30:32.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.875+0000 7f15f77fe640 1 --2- 172.21.15.106:0/1730154609 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15f8156ec0 0x7f15f8152bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:32.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.875+0000 7f15fcf64640 1 --2- 172.21.15.106:0/1730154609 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f15f81592b0 0x7f15f815f0d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f15f000cb20 tx=0x7f15f00021c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:32.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.875+0000 7f15f4ff9640 1 -- 172.21.15.106:0/1730154609 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f000ac00 con 0x7f15f81592b0 2024-04-25T10:30:32.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.876+0000 7f15f4ff9640 1 -- 172.21.15.106:0/1730154609 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15f000adc0 con 0x7f15f81592b0 2024-04-25T10:30:32.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.876+0000 7f15fe9c7640 1 -- 172.21.15.106:0/1730154609 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15f81584f0 con 0x7f15f81592b0 2024-04-25T10:30:32.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.876+0000 7f15f4ff9640 1 -- 172.21.15.106:0/1730154609 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f0012640 con 0x7f15f81592b0 2024-04-25T10:30:32.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.876+0000 7f15fe9c7640 1 -- 172.21.15.106:0/1730154609 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15f8158ab0 con 0x7f15f81592b0 2024-04-25T10:30:32.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.876+0000 7f15fe9c7640 1 -- 172.21.15.106:0/1730154609 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15f8153110 con 0x7f15f81592b0 2024-04-25T10:30:32.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.877+0000 7f15f4ff9640 1 -- 172.21.15.106:0/1730154609 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f15f00127e0 con 0x7f15f81592b0 2024-04-25T10:30:32.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.877+0000 7f15f4ff9640 1 --2- 172.21.15.106:0/1730154609 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f15e40744e0 0x7f15e40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:32.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.878+0000 7f15f7fff640 1 --2- 172.21.15.106:0/1730154609 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f15e40744e0 0x7f15e40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:32.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.878+0000 7f15f4ff9640 1 -- 172.21.15.106:0/1730154609 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f15f0095790 con 0x7f15f81592b0 2024-04-25T10:30:32.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.878+0000 7f15f7fff640 1 --2- 172.21.15.106:0/1730154609 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f15e40744e0 0x7f15e40769f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f15e8034000 tx=0x7f15e802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:32.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:32.882+0000 7f15f4ff9640 1 -- 172.21.15.106:0/1730154609 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f15f8153110 con 0x7f15f81592b0 2024-04-25T10:30:33.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.057+0000 7f15fe9c7640 1 -- 172.21.15.106:0/1730154609 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f15f80cbdd0 con 0x7f15f81592b0 2024-04-25T10:30:33.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.059+0000 7f15f4ff9640 1 -- 172.21.15.106:0/1730154609 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16654 (secure 0 0 0) 0x7f15f005ffb0 con 0x7f15f81592b0 2024-04-25T10:30:33.061 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:30:33.061 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":19,"fsid":"3ee77985-e8e4-41cf-81e9-7c774cf7891f","created":"2024-04-25T10:30:07.981054+0000","modified":"2024-04-25T10:30:31.761147+0000","last_up_change":"2024-04-25T10:30:19.184325+0000","last_in_change":"2024-04-25T10:30:10.217553+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:30:20.111046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:30:30.314914+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:30:30.715500+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a5962cb1-dbbd-47b1-8d15-9bae114bfdf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1240717079},{"type":"v1","addr":"172.21.15.106:6801","nonce":1240717079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1240717079},{"type":"v1","addr":"172.21.15.106:6803","nonce":1240717079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1240717079},{"type":"v1","addr":"172.21.15.106:6807","nonce":1240717079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1240717079},{"type":"v1","addr":"172.21.15.106:6805","nonce":1240717079}]},"public_addr":"172.21.15.106:6801/1240717079","cluster_addr":"172.21.15.106:6803/1240717079","heartbeat_back_addr":"172.21.15.106:6807/1240717079","heartbeat_front_addr":"172.21.15.106:6805/1240717079","state":["exists","up"]},{"osd":1,"uuid":"94436e13-4769-4752-8397-b5e396a6f7ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4089653051},{"type":"v1","addr":"172.21.15.106:6825","nonce":4089653051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4089653051},{"type":"v1","addr":"172.21.15.106:6827","nonce":4089653051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4089653051},{"type":"v1","addr":"172.21.15.106:6831","nonce":4089653051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4089653051},{"type":"v1","addr":"172.21.15.106:6829","nonce":4089653051}]},"public_addr":"172.21.15.106:6825/4089653051","cluster_addr":"172.21.15.106:6827/4089653051","heartbeat_back_addr":"172.21.15.106:6831/4089653051","heartbeat_front_addr":"172.21.15.106:6829/4089653051","state":["exists","up"]},{"osd":2,"uuid":"429aae0b-70c6-4e7b-9043-6a11334c7efc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":1441018239},{"type":"v1","addr":"172.21.15.106:6809","nonce":1441018239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1441018239},{"type":"v1","addr":"172.21.15.106:6811","nonce":1441018239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1441018239},{"type":"v1","addr":"172.21.15.106:6815","nonce":1441018239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":1441018239},{"type":"v1","addr":"172.21.15.106:6813","nonce":1441018239}]},"public_addr":"172.21.15.106:6809/1441018239","cluster_addr":"172.21.15.106:6811/1441018239","heartbeat_back_addr":"172.21.15.106:6815/1441018239","heartbeat_front_addr":"172.21.15.106:6813/1441018239","state":["exists","up"]},{"osd":3,"uuid":"9593a4d1-772f-4667-ab0d-2982f841e342","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":1846968711},{"type":"v1","addr":"172.21.15.106:6817","nonce":1846968711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1846968711},{"type":"v1","addr":"172.21.15.106:6819","nonce":1846968711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":1846968711},{"type":"v1","addr":"172.21.15.106:6823","nonce":1846968711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":1846968711},{"type":"v1","addr":"172.21.15.106:6821","nonce":1846968711}]},"public_addr":"172.21.15.106:6817/1846968711","cluster_addr":"172.21.15.106:6819/1846968711","heartbeat_back_addr":"172.21.15.106:6823/1846968711","heartbeat_front_addr":"172.21.15.106:6821/1846968711","state":["exists","up"]},{"osd":4,"uuid":"1b61f16d-5c60-480a-b449-c745aa3f6b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":729224739},{"type":"v1","addr":"172.21.15.134:6825","nonce":729224739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":729224739},{"type":"v1","addr":"172.21.15.134:6827","nonce":729224739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":729224739},{"type":"v1","addr":"172.21.15.134:6831","nonce":729224739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":729224739},{"type":"v1","addr":"172.21.15.134:6829","nonce":729224739}]},"public_addr":"172.21.15.134:6825/729224739","cluster_addr":"172.21.15.134:6827/729224739","heartbeat_back_addr":"172.21.15.134:6831/729224739","heartbeat_front_addr":"172.21.15.134:6829/729224739","state":["exists","up"]},{"osd":5,"uuid":"944cdd7f-47c0-4b71-b7e0-d093542f4528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3132275652},{"type":"v1","addr":"172.21.15.134:6809","nonce":3132275652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3132275652},{"type":"v1","addr":"172.21.15.134:6811","nonce":3132275652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3132275652},{"type":"v1","addr":"172.21.15.134:6815","nonce":3132275652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3132275652},{"type":"v1","addr":"172.21.15.134:6813","nonce":3132275652}]},"public_addr":"172.21.15.134:6809/3132275652","cluster_addr":"172.21.15.134:6811/3132275652","heartbeat_back_addr":"172.21.15.134:6815/3132275652","heartbeat_front_addr":"172.21.15.134:6813/3132275652","state":["exists","up"]},{"osd":6,"uuid":"cc2446e2-1d21-4135-ac04-6af7c990058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":992702668},{"type":"v1","addr":"172.21.15.134:6801","nonce":992702668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":992702668},{"type":"v1","addr":"172.21.15.134:6803","nonce":992702668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":992702668},{"type":"v1","addr":"172.21.15.134:6807","nonce":992702668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":992702668},{"type":"v1","addr":"172.21.15.134:6805","nonce":992702668}]},"public_addr":"172.21.15.134:6801/992702668","cluster_addr":"172.21.15.134:6803/992702668","heartbeat_back_addr":"172.21.15.134:6807/992702668","heartbeat_front_addr":"172.21.15.134:6805/992702668","state":["exists","up"]},{"osd":7,"uuid":"3df1fc7a-82fc-4257-8629-98027b6e2aed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":3291062284},{"type":"v1","addr":"172.21.15.134:6817","nonce":3291062284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":3291062284},{"type":"v1","addr":"172.21.15.134:6819","nonce":3291062284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":3291062284},{"type":"v1","addr":"172.21.15.134:6823","nonce":3291062284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":3291062284},{"type":"v1","addr":"172.21.15.134:6821","nonce":3291062284}]},"public_addr":"172.21.15.134:6817/3291062284","cluster_addr":"172.21.15.134:6819/3291062284","heartbeat_back_addr":"172.21.15.134:6823/3291062284","heartbeat_front_addr":"172.21.15.134:6821/3291062284","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.403941+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.382563+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.411544+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.330444+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.405430+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.387737+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.408922+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.454450+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:30:33.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.061+0000 7f15fe9c7640 1 -- 172.21.15.106:0/1730154609 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f15e40744e0 msgr2=0x7f15e40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:33.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.061+0000 7f15fe9c7640 1 --2- 172.21.15.106:0/1730154609 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f15e40744e0 0x7f15e40769f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f15e8034000 tx=0x7f15e802e000 comp rx=0 tx=0).stop 2024-04-25T10:30:33.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.061+0000 7f15fe9c7640 1 -- 172.21.15.106:0/1730154609 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f15f81592b0 msgr2=0x7f15f815f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:33.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.061+0000 7f15fe9c7640 1 --2- 172.21.15.106:0/1730154609 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f15f81592b0 0x7f15f815f0d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f15f000cb20 tx=0x7f15f00021c0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.061+0000 7f15fe9c7640 1 -- 172.21.15.106:0/1730154609 shutdown_connections 2024-04-25T10:30:33.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.061+0000 7f15fe9c7640 1 --2- 172.21.15.106:0/1730154609 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f15e40744e0 0x7f15e40769f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.061+0000 7f15fe9c7640 1 --2- 172.21.15.106:0/1730154609 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f15f81592b0 0x7f15f815f0d0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.061+0000 7f15fe9c7640 1 --2- 172.21.15.106:0/1730154609 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15f8156ec0 0x7f15f8152bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.061+0000 7f15fe9c7640 1 --2- 172.21.15.106:0/1730154609 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15f80c66e0 0x7f15f815a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.061+0000 7f15fe9c7640 1 -- 172.21.15.106:0/1730154609 >> 172.21.15.106:0/1730154609 conn(0x7f15f80727d0 msgr2=0x7f15f806f120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:33.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.061+0000 7f15fe9c7640 1 -- 172.21.15.106:0/1730154609 shutdown_connections 2024-04-25T10:30:33.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.061+0000 7f15fe9c7640 1 -- 172.21.15.106:0/1730154609 wait complete. 2024-04-25T10:30:33.074 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:30:33.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.182+0000 7fad0a496640 1 Processor -- start 2024-04-25T10:30:33.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.182+0000 7fad0a496640 1 -- start start 2024-04-25T10:30:33.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.183+0000 7fad0a496640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fad040cb570 0x7fad040711c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.183+0000 7fad0a496640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad040ca2e0 con 0x7fad040c6420 2024-04-25T10:30:33.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.183+0000 7fad0a496640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad040cadb0 con 0x7fad04071fe0 2024-04-25T10:30:33.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.183+0000 7fad0a496640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fad040c6960 con 0x7fad040cb570 2024-04-25T10:30:33.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.183+0000 7fad08a33640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fad040cb570 0x7fad040711c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.183+0000 7fad08a33640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fad040cb570 0x7fad040711c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58068/0 (socket says 172.21.15.106:58068) 2024-04-25T10:30:33.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.183+0000 7fad03fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fad040c6420 0x7fad04159a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:51314/0 (socket says 172.21.15.106:51314) 2024-04-25T10:30:33.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.183+0000 7fad08a33640 1 -- 172.21.15.106:0/3082813267 learned_addr learned my addr 172.21.15.106:0/3082813267 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:33.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.184+0000 7fad08a33640 1 -- 172.21.15.106:0/3082813267 >> v1:172.21.15.134:6789/0 conn(0x7fad04071fe0 legacy=0x7fad04153bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:33.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.184+0000 7fad08a33640 1 -- 172.21.15.106:0/3082813267 >> v1:172.21.15.106:6789/0 conn(0x7fad040c6420 legacy=0x7fad04159a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:33.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.184+0000 7fad08a33640 1 -- 172.21.15.106:0/3082813267 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad040c6e30 con 0x7fad040cb570 2024-04-25T10:30:33.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.184+0000 7fad02ffd640 1 -- 172.21.15.106:0/3082813267 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1972734782 0 0) 0x7fad040ca2e0 con 0x7fad040c6420 2024-04-25T10:30:33.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.184+0000 7fad08a33640 1 --2- 172.21.15.106:0/3082813267 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad040cb570 0x7fad040711c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7facfc00d150 tx=0x7facfc02c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd1bd802e999cfa1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:33.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.185+0000 7fad02ffd640 1 -- 172.21.15.106:0/3082813267 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facfc00b2e0 con 0x7fad040cb570 2024-04-25T10:30:33.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.185+0000 7fad02ffd640 1 -- 172.21.15.106:0/3082813267 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facfc009600 con 0x7fad040cb570 2024-04-25T10:30:33.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.185+0000 7fad02ffd640 1 -- 172.21.15.106:0/3082813267 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facfc00b480 con 0x7fad040cb570 2024-04-25T10:30:33.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.185+0000 7fad0a496640 1 -- 172.21.15.106:0/3082813267 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad040cb570 msgr2=0x7fad040711c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:33.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.185+0000 7fad0a496640 1 --2- 172.21.15.106:0/3082813267 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad040cb570 0x7fad040711c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7facfc00d150 tx=0x7facfc02c8e0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.185+0000 7fad0a496640 1 -- 172.21.15.106:0/3082813267 shutdown_connections 2024-04-25T10:30:33.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.185+0000 7fad0a496640 1 --2- 172.21.15.106:0/3082813267 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad040cb570 0x7fad040711c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.185+0000 7fad0a496640 1 -- 172.21.15.106:0/3082813267 >> 172.21.15.106:0/3082813267 conn(0x7fad040cef20 msgr2=0x7fad040cf330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:33.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.185+0000 7fad0a496640 1 -- 172.21.15.106:0/3082813267 shutdown_connections 2024-04-25T10:30:33.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.185+0000 7fad0a496640 1 -- 172.21.15.106:0/3082813267 wait complete. 2024-04-25T10:30:33.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.187+0000 7fad0a496640 1 Processor -- start 2024-04-25T10:30:33.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.187+0000 7fad0a496640 1 -- start start 2024-04-25T10:30:33.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.188+0000 7fad0a496640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad04071fe0 0x7fad0414d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.188+0000 7fad0a496640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad040c6420 0x7fad04150dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.188+0000 7fad0a496640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad040cb570 0x7fad04145730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.188+0000 7fad0a496640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fad040c73f0 con 0x7fad040c6420 2024-04-25T10:30:33.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.188+0000 7fad0a496640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fad040cb050 con 0x7fad04071fe0 2024-04-25T10:30:33.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.188+0000 7fad0a496640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fad040c7570 con 0x7fad040cb570 2024-04-25T10:30:33.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.188+0000 7fad037fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad040c6420 0x7fad04150dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.188+0000 7fad037fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad040c6420 0x7fad04150dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58074/0 (socket says 172.21.15.106:58074) 2024-04-25T10:30:33.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.189+0000 7fad037fe640 1 -- 172.21.15.106:0/1217622061 learned_addr learned my addr 172.21.15.106:0/1217622061 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:33.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.189+0000 7fad03fff640 1 --2- 172.21.15.106:0/1217622061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad04071fe0 0x7fad0414d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.189+0000 7fad08a33640 1 --2- 172.21.15.106:0/1217622061 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad040cb570 0x7fad04145730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.189+0000 7fad037fe640 1 -- 172.21.15.106:0/1217622061 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad040cb570 msgr2=0x7fad04145730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:33.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.189+0000 7fad037fe640 1 --2- 172.21.15.106:0/1217622061 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad040cb570 0x7fad04145730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.189+0000 7fad037fe640 1 -- 172.21.15.106:0/1217622061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad04071fe0 msgr2=0x7fad0414d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:33.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.189+0000 7fad037fe640 1 --2- 172.21.15.106:0/1217622061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad04071fe0 0x7fad0414d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.189+0000 7fad037fe640 1 -- 172.21.15.106:0/1217622061 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad0414dac0 con 0x7fad040c6420 2024-04-25T10:30:33.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.189+0000 7fad03fff640 1 --2- 172.21.15.106:0/1217622061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad04071fe0 0x7fad0414d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:33.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.189+0000 7fad037fe640 1 --2- 172.21.15.106:0/1217622061 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad040c6420 0x7fad04150dd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7facf800b3e0 tx=0x7facf800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:33.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.189+0000 7fad00ff9640 1 -- 172.21.15.106:0/1217622061 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf8015020 con 0x7fad040c6420 2024-04-25T10:30:33.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.189+0000 7fad00ff9640 1 -- 172.21.15.106:0/1217622061 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facf8011850 con 0x7fad040c6420 2024-04-25T10:30:33.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.189+0000 7fad00ff9640 1 -- 172.21.15.106:0/1217622061 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf8011c80 con 0x7fad040c6420 2024-04-25T10:30:33.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.189+0000 7fad0a496640 1 -- 172.21.15.106:0/1217622061 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad04149900 con 0x7fad040c6420 2024-04-25T10:30:33.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.189+0000 7fad0a496640 1 -- 172.21.15.106:0/1217622061 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad04156ba0 con 0x7fad040c6420 2024-04-25T10:30:33.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.190+0000 7fad0a496640 1 -- 172.21.15.106:0/1217622061 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad0406a3d0 con 0x7fad040c6420 2024-04-25T10:30:33.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.191+0000 7fad00ff9640 1 -- 172.21.15.106:0/1217622061 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7facf8018020 con 0x7fad040c6420 2024-04-25T10:30:33.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.191+0000 7fad00ff9640 1 --2- 172.21.15.106:0/1217622061 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7facf00744b0 0x7facf00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.191+0000 7fad03fff640 1 --2- 172.21.15.106:0/1217622061 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7facf00744b0 0x7facf00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.191+0000 7fad00ff9640 1 -- 172.21.15.106:0/1217622061 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7facf8094f50 con 0x7fad040c6420 2024-04-25T10:30:33.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.192+0000 7fad03fff640 1 --2- 172.21.15.106:0/1217622061 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7facf00744b0 0x7facf00769c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7facf4000f60 tx=0x7facf4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:33.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.196+0000 7fad00ff9640 1 -- 172.21.15.106:0/1217622061 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7facf8011e20 con 0x7fad040c6420 2024-04-25T10:30:33.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.408+0000 7fad0a496640 1 -- 172.21.15.106:0/1217622061 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fad040c79b0 con 0x7fad040c6420 2024-04-25T10:30:33.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.433+0000 7fad00ff9640 1 -- 172.21.15.106:0/1217622061 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v5) ==== 118+0+0 (secure 0 0 0) 0x7facf805f800 con 0x7fad040c6420 2024-04-25T10:30:33.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.435+0000 7fad0a496640 1 -- 172.21.15.106:0/1217622061 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7facf00744b0 msgr2=0x7facf00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:33.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.435+0000 7fad0a496640 1 --2- 172.21.15.106:0/1217622061 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7facf00744b0 0x7facf00769c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7facf4000f60 tx=0x7facf4009040 comp rx=0 tx=0).stop 2024-04-25T10:30:33.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.435+0000 7fad0a496640 1 -- 172.21.15.106:0/1217622061 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad040c6420 msgr2=0x7fad04150dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:33.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.435+0000 7fad0a496640 1 --2- 172.21.15.106:0/1217622061 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad040c6420 0x7fad04150dd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7facf800b3e0 tx=0x7facf800b8a0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.435+0000 7fad0a496640 1 -- 172.21.15.106:0/1217622061 shutdown_connections 2024-04-25T10:30:33.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.435+0000 7fad0a496640 1 --2- 172.21.15.106:0/1217622061 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7facf00744b0 0x7facf00769c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.435+0000 7fad0a496640 1 --2- 172.21.15.106:0/1217622061 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad040cb570 0x7fad04145730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.435+0000 7fad0a496640 1 --2- 172.21.15.106:0/1217622061 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad040c6420 0x7fad04150dd0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.435+0000 7fad0a496640 1 --2- 172.21.15.106:0/1217622061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad04071fe0 0x7fad0414d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.435+0000 7fad0a496640 1 -- 172.21.15.106:0/1217622061 >> 172.21.15.106:0/1217622061 conn(0x7fad040cef20 msgr2=0x7fad040c9100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:33.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.435+0000 7fad0a496640 1 -- 172.21.15.106:0/1217622061 shutdown_connections 2024-04-25T10:30:33.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.435+0000 7fad0a496640 1 -- 172.21.15.106:0/1217622061 wait complete. 2024-04-25T10:30:33.448 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300} 2024-04-25T10:30:33.449 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-04-25T10:30:33.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.557+0000 7f49f1b8f640 1 Processor -- start 2024-04-25T10:30:33.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.557+0000 7f49f1b8f640 1 -- start start 2024-04-25T10:30:33.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.558+0000 7f49f1b8f640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f49ec0cbff0 0x7f49ec0cb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.558+0000 7f49f1b8f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f49ec156ec0 0x7f49ec0c61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.558+0000 7f49f1b8f640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49ec15dae0 con 0x7f49ec0c66e0 2024-04-25T10:30:33.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.558+0000 7f49f1b8f640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f49ec161270 con 0x7f49ec0cbff0 2024-04-25T10:30:33.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.558+0000 7f49f1b8f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f49ec1604b0 con 0x7f49ec156ec0 2024-04-25T10:30:33.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.558+0000 7f49eaffd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f49ec156ec0 0x7f49ec0c61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.558+0000 7f49eb7fe640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f49ec0cbff0 0x7f49ec0cb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.558+0000 7f49eb7fe640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f49ec0cbff0 0x7f49ec0cb270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.106:47564/0 (socket says 172.21.15.106:47564) 2024-04-25T10:30:33.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.558+0000 7f49eaffd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f49ec156ec0 0x7f49ec0c61a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.106:48640/0 (socket says 172.21.15.106:48640) 2024-04-25T10:30:33.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.558+0000 7f49eb7fe640 1 -- 172.21.15.106:0/1300990294 learned_addr learned my addr 172.21.15.106:0/1300990294 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:33.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.558+0000 7f49ebfff640 1 --1- 172.21.15.106:0/1300990294 >> v1:172.21.15.134:6789/0 conn(0x7f49ec0c66e0 0x7f49ec0cce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.106:41114/0 (socket says 172.21.15.106:41114) 2024-04-25T10:30:33.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.559+0000 7f49eb7fe640 1 -- 172.21.15.106:0/1300990294 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f49ec156ec0 msgr2=0x7f49ec0c61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:33.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.559+0000 7f49eb7fe640 1 --2- 172.21.15.106:0/1300990294 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f49ec156ec0 0x7f49ec0c61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.559+0000 7f49eb7fe640 1 -- 172.21.15.106:0/1300990294 >> v1:172.21.15.134:6789/0 conn(0x7f49ec0c66e0 legacy=0x7f49ec0cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:33.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.559+0000 7f49ea7fc640 1 -- 172.21.15.106:0/1300990294 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1481889646 0 0) 0x7f49ec15dae0 con 0x7f49ec0c66e0 2024-04-25T10:30:33.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.559+0000 7f49eb7fe640 1 -- 172.21.15.106:0/1300990294 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49ec160790 con 0x7f49ec0cbff0 2024-04-25T10:30:33.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.559+0000 7f49eaffd640 1 --2- 172.21.15.106:0/1300990294 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f49ec156ec0 0x7f49ec0c61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:33.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.560+0000 7f49eb7fe640 1 --2- 172.21.15.106:0/1300990294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f49ec0cbff0 0x7f49ec0cb270 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f49dc009870 tx=0x7f49dc02a300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73963ca2ca76aeed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:33.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.560+0000 7f49ea7fc640 1 -- 172.21.15.106:0/1300990294 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49dc037070 con 0x7f49ec0cbff0 2024-04-25T10:30:33.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.560+0000 7f49ea7fc640 1 -- 172.21.15.106:0/1300990294 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49dc02ae50 con 0x7f49ec0cbff0 2024-04-25T10:30:33.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.560+0000 7f49ea7fc640 1 -- 172.21.15.106:0/1300990294 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49dc032860 con 0x7f49ec0cbff0 2024-04-25T10:30:33.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.560+0000 7f49f1b8f640 1 -- 172.21.15.106:0/1300990294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f49ec0cbff0 msgr2=0x7f49ec0cb270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:33.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.560+0000 7f49f1b8f640 1 --2- 172.21.15.106:0/1300990294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f49ec0cbff0 0x7f49ec0cb270 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f49dc009870 tx=0x7f49dc02a300 comp rx=0 tx=0).stop 2024-04-25T10:30:33.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.560+0000 7f49f1b8f640 1 -- 172.21.15.106:0/1300990294 shutdown_connections 2024-04-25T10:30:33.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.560+0000 7f49f1b8f640 1 --2- 172.21.15.106:0/1300990294 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f49ec156ec0 0x7f49ec0c61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.560+0000 7f49f1b8f640 1 --2- 172.21.15.106:0/1300990294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f49ec0cbff0 0x7f49ec0cb270 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.560+0000 7f49f1b8f640 1 -- 172.21.15.106:0/1300990294 >> 172.21.15.106:0/1300990294 conn(0x7f49ec0727d0 msgr2=0x7f49ec072c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:33.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.560+0000 7f49f1b8f640 1 -- 172.21.15.106:0/1300990294 shutdown_connections 2024-04-25T10:30:33.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.561+0000 7f49f1b8f640 1 -- 172.21.15.106:0/1300990294 wait complete. 2024-04-25T10:30:33.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.562+0000 7f49f1b8f640 1 Processor -- start 2024-04-25T10:30:33.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.562+0000 7f49f1b8f640 1 -- start start 2024-04-25T10:30:33.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.564+0000 7f49f1b8f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49ec0c66e0 0x7f49ec1f2800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.564+0000 7f49f1b8f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f49ec156ec0 0x7f49ec155530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.564+0000 7f49f1b8f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f49ec14cd00 0x7f49ec14b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.564+0000 7f49f1b8f640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f49ec1572d0 con 0x7f49ec0c66e0 2024-04-25T10:30:33.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.564+0000 7f49f1b8f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f49ec160d50 con 0x7f49ec156ec0 2024-04-25T10:30:33.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.564+0000 7f49f1b8f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f49ec15d730 con 0x7f49ec14cd00 2024-04-25T10:30:33.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.564+0000 7f49eb7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49ec0c66e0 0x7f49ec1f2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.564+0000 7f49eb7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49ec0c66e0 0x7f49ec1f2800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47404/0 (socket says 172.21.15.106:47404) 2024-04-25T10:30:33.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.564+0000 7f49eb7fe640 1 -- 172.21.15.106:0/3778004343 learned_addr learned my addr 172.21.15.106:0/3778004343 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:33.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.564+0000 7f49eaffd640 1 --2- 172.21.15.106:0/3778004343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f49ec156ec0 0x7f49ec155530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.564+0000 7f49ebfff640 1 --2- 172.21.15.106:0/3778004343 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f49ec14cd00 0x7f49ec14b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.564+0000 7f49eb7fe640 1 -- 172.21.15.106:0/3778004343 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f49ec14cd00 msgr2=0x7f49ec14b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:33.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.564+0000 7f49eb7fe640 1 --2- 172.21.15.106:0/3778004343 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f49ec14cd00 0x7f49ec14b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.564+0000 7f49eb7fe640 1 -- 172.21.15.106:0/3778004343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f49ec156ec0 msgr2=0x7f49ec155530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:33.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.564+0000 7f49eb7fe640 1 --2- 172.21.15.106:0/3778004343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f49ec156ec0 0x7f49ec155530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.564+0000 7f49eb7fe640 1 -- 172.21.15.106:0/3778004343 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49ec14f1b0 con 0x7f49ec0c66e0 2024-04-25T10:30:33.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.564+0000 7f49eb7fe640 1 --2- 172.21.15.106:0/3778004343 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49ec0c66e0 0x7f49ec1f2800 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f49dc009870 tx=0x7f49dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:33.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.564+0000 7f49d3fff640 1 -- 172.21.15.106:0/3778004343 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49dc037040 con 0x7f49ec0c66e0 2024-04-25T10:30:33.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.564+0000 7f49d3fff640 1 -- 172.21.15.106:0/3778004343 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49dc004250 con 0x7f49ec0c66e0 2024-04-25T10:30:33.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.564+0000 7f49f1b8f640 1 -- 172.21.15.106:0/3778004343 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49ec1f6c40 con 0x7f49ec0c66e0 2024-04-25T10:30:33.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.564+0000 7f49d3fff640 1 -- 172.21.15.106:0/3778004343 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49dc003610 con 0x7f49ec0c66e0 2024-04-25T10:30:33.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.564+0000 7f49f1b8f640 1 -- 172.21.15.106:0/3778004343 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49ec151660 con 0x7f49ec0c66e0 2024-04-25T10:30:33.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.565+0000 7f49f1b8f640 1 -- 172.21.15.106:0/3778004343 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49ec06a3d0 con 0x7f49ec0c66e0 2024-04-25T10:30:33.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.566+0000 7f49d3fff640 1 -- 172.21.15.106:0/3778004343 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f49dc03a070 con 0x7f49ec0c66e0 2024-04-25T10:30:33.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.566+0000 7f49d3fff640 1 --2- 172.21.15.106:0/3778004343 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f49d80744b0 0x7f49d80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.566+0000 7f49eaffd640 1 --2- 172.21.15.106:0/3778004343 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f49d80744b0 0x7f49d80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.566+0000 7f49d3fff640 1 -- 172.21.15.106:0/3778004343 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f49dc0b5690 con 0x7f49ec0c66e0 2024-04-25T10:30:33.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.567+0000 7f49eaffd640 1 --2- 172.21.15.106:0/3778004343 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f49d80744b0 0x7f49d80769c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f49ec0ca630 tx=0x7f49e0009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:33.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.571+0000 7f49d3fff640 1 -- 172.21.15.106:0/3778004343 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f49dc041020 con 0x7f49ec0c66e0 2024-04-25T10:30:33.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:33.784+0000 7f49f1b8f640 1 -- 172.21.15.106:0/3778004343 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f49ec06e170 con 0x7f49ec0c66e0 2024-04-25T10:30:34.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.438+0000 7f49d3fff640 1 -- 172.21.15.106:0/3778004343 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v6) ==== 105+0+0 (secure 0 0 0) 0x7f49dc07f8f0 con 0x7f49ec0c66e0 2024-04-25T10:30:34.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.441+0000 7f49f1b8f640 1 -- 172.21.15.106:0/3778004343 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f49d80744b0 msgr2=0x7f49d80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:34.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.441+0000 7f49f1b8f640 1 --2- 172.21.15.106:0/3778004343 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f49d80744b0 0x7f49d80769c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f49ec0ca630 tx=0x7f49e0009190 comp rx=0 tx=0).stop 2024-04-25T10:30:34.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.441+0000 7f49f1b8f640 1 -- 172.21.15.106:0/3778004343 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49ec0c66e0 msgr2=0x7f49ec1f2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:34.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.441+0000 7f49f1b8f640 1 --2- 172.21.15.106:0/3778004343 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49ec0c66e0 0x7f49ec1f2800 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f49dc009870 tx=0x7f49dc0046c0 comp rx=0 tx=0).stop 2024-04-25T10:30:34.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.441+0000 7f49f1b8f640 1 -- 172.21.15.106:0/3778004343 shutdown_connections 2024-04-25T10:30:34.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.441+0000 7f49f1b8f640 1 --2- 172.21.15.106:0/3778004343 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f49d80744b0 0x7f49d80769c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:34.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.441+0000 7f49f1b8f640 1 --2- 172.21.15.106:0/3778004343 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f49ec14cd00 0x7f49ec14b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:34.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.441+0000 7f49f1b8f640 1 --2- 172.21.15.106:0/3778004343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f49ec156ec0 0x7f49ec155530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:34.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.441+0000 7f49f1b8f640 1 --2- 172.21.15.106:0/3778004343 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49ec0c66e0 0x7f49ec1f2800 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:34.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.441+0000 7f49f1b8f640 1 -- 172.21.15.106:0/3778004343 >> 172.21.15.106:0/3778004343 conn(0x7f49ec0727d0 msgr2=0x7f49ec06e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:34.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.441+0000 7f49f1b8f640 1 -- 172.21.15.106:0/3778004343 shutdown_connections 2024-04-25T10:30:34.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.441+0000 7f49f1b8f640 1 -- 172.21.15.106:0/3778004343 wait complete. 2024-04-25T10:30:34.457 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:30:34.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.572+0000 7fe6740d1640 1 Processor -- start 2024-04-25T10:30:34.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.572+0000 7fe6740d1640 1 -- start start 2024-04-25T10:30:34.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.573+0000 7fe6740d1640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fe66c154ac0 0x7fe66c166350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:34.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.573+0000 7fe6740d1640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe66c15b8c0 con 0x7fe66c0cb2b0 2024-04-25T10:30:34.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.573+0000 7fe6740d1640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe66c15b5e0 con 0x7fe66c0cc600 2024-04-25T10:30:34.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.573+0000 7fe6740d1640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fe66c15eea0 con 0x7fe66c154ac0 2024-04-25T10:30:34.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.573+0000 7fe67166c640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fe66c0cb2b0 0x7fe66c0cce90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:44212/0 (socket says 172.21.15.106:44212) 2024-04-25T10:30:34.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.573+0000 7fe67166c640 1 -- 172.21.15.106:0/132982512 learned_addr learned my addr 172.21.15.106:0/132982512 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:30:34.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.573+0000 7fe67266e640 1 --2- 172.21.15.106:0/132982512 >> v2:172.21.15.134:3300/0 conn(0x7fe66c154ac0 0x7fe66c166350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:34.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.574+0000 7fe670e6b640 1 -- 172.21.15.106:0/132982512 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2715094862 0 0) 0x7fe66c15b8c0 con 0x7fe66c0cb2b0 2024-04-25T10:30:34.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.574+0000 7fe670e6b640 1 -- 172.21.15.106:0/132982512 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe658003520 con 0x7fe66c0cb2b0 2024-04-25T10:30:34.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.574+0000 7fe670e6b640 1 -- 172.21.15.106:0/132982512 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2250677056 0 0) 0x7fe658003520 con 0x7fe66c0cb2b0 2024-04-25T10:30:34.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.574+0000 7fe670e6b640 1 -- 172.21.15.106:0/132982512 >> v2:172.21.15.134:3300/0 conn(0x7fe66c154ac0 msgr2=0x7fe66c166350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:34.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.574+0000 7fe670e6b640 1 --2- 172.21.15.106:0/132982512 >> v2:172.21.15.134:3300/0 conn(0x7fe66c154ac0 0x7fe66c166350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:34.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.574+0000 7fe670e6b640 1 -- 172.21.15.106:0/132982512 >> v1:172.21.15.134:6789/0 conn(0x7fe66c0cc600 legacy=0x7fe66c0c5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:34.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.574+0000 7fe670e6b640 1 -- 172.21.15.106:0/132982512 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe66c15e0e0 con 0x7fe66c0cb2b0 2024-04-25T10:30:34.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.574+0000 7fe670e6b640 1 -- 172.21.15.106:0/132982512 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3776459350 0 0) 0x7fe65c001670 con 0x7fe66c0cb2b0 2024-04-25T10:30:34.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.574+0000 7fe670e6b640 1 -- 172.21.15.106:0/132982512 >> v1:172.21.15.106:6789/0 conn(0x7fe66c0cb2b0 legacy=0x7fe66c0cce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:34.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.574+0000 7fe670e6b640 1 --2- 172.21.15.106:0/132982512 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6580039b0 0x7fe658003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:34.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.574+0000 7fe670e6b640 1 --2- 172.21.15.106:0/132982512 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe658004800 0x7fe658004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:34.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.574+0000 7fe670e6b640 1 --2- 172.21.15.106:0/132982512 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe6580051c0 0x7fe65800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:34.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.574+0000 7fe670e6b640 1 -- 172.21.15.106:0/132982512 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe66c15eea0 con 0x7fe6580051c0 2024-04-25T10:30:34.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.574+0000 7fe670e6b640 1 -- 172.21.15.106:0/132982512 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe65800c160 con 0x7fe658004800 2024-04-25T10:30:34.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.574+0000 7fe670e6b640 1 -- 172.21.15.106:0/132982512 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe65800c360 con 0x7fe6580039b0 2024-04-25T10:30:34.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.574+0000 7fe67266e640 1 --2- 172.21.15.106:0/132982512 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe6580051c0 0x7fe65800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:34.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.574+0000 7fe67166c640 1 --2- 172.21.15.106:0/132982512 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe658004800 0x7fe658004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:34.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.574+0000 7fe671e6d640 1 --2- 172.21.15.106:0/132982512 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6580039b0 0x7fe658003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:34.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.574+0000 7fe67266e640 1 -- 172.21.15.106:0/132982512 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6580039b0 msgr2=0x7fe658003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:34.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.574+0000 7fe67266e640 1 --2- 172.21.15.106:0/132982512 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6580039b0 0x7fe658003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:34.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.574+0000 7fe67266e640 1 -- 172.21.15.106:0/132982512 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe658004800 msgr2=0x7fe658004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:34.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.574+0000 7fe67266e640 1 --2- 172.21.15.106:0/132982512 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe658004800 0x7fe658004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:34.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.574+0000 7fe67266e640 1 -- 172.21.15.106:0/132982512 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe658003520 con 0x7fe6580051c0 2024-04-25T10:30:34.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.575+0000 7fe67266e640 1 -- 172.21.15.106:0/132982512 reap_dead start 2024-04-25T10:30:34.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.575+0000 7fe67266e640 1 --2- 172.21.15.106:0/132982512 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe6580051c0 0x7fe65800b9c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe664009f90 tx=0x7fe658005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79fd41045b7bde2d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:34.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.575+0000 7fe670e6b640 1 -- 172.21.15.106:0/132982512 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe664037070 con 0x7fe6580051c0 2024-04-25T10:30:34.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.575+0000 7fe670e6b640 1 -- 172.21.15.106:0/132982512 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe664032620 con 0x7fe6580051c0 2024-04-25T10:30:34.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.575+0000 7fe670e6b640 1 -- 172.21.15.106:0/132982512 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe664009c10 con 0x7fe6580051c0 2024-04-25T10:30:34.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.576+0000 7fe6740d1640 1 -- 172.21.15.106:0/132982512 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe6580051c0 msgr2=0x7fe65800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:34.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.576+0000 7fe6740d1640 1 --2- 172.21.15.106:0/132982512 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe6580051c0 0x7fe65800b9c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe664009f90 tx=0x7fe658005830 comp rx=0 tx=0).stop 2024-04-25T10:30:34.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.576+0000 7fe6740d1640 1 -- 172.21.15.106:0/132982512 shutdown_connections 2024-04-25T10:30:34.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.576+0000 7fe6740d1640 1 --2- 172.21.15.106:0/132982512 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe6580051c0 0x7fe65800b9c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:34.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.576+0000 7fe6740d1640 1 -- 172.21.15.106:0/132982512 >> 172.21.15.106:0/132982512 conn(0x7fe66c0727d0 msgr2=0x7fe66c072c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:34.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.576+0000 7fe6740d1640 1 -- 172.21.15.106:0/132982512 shutdown_connections 2024-04-25T10:30:34.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.576+0000 7fe6740d1640 1 -- 172.21.15.106:0/132982512 wait complete. 2024-04-25T10:30:34.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.577+0000 7fe6740d1640 1 Processor -- start 2024-04-25T10:30:34.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.577+0000 7fe6740d1640 1 -- start start 2024-04-25T10:30:34.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.579+0000 7fe6740d1640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6580051c0 0x7fe66c1562e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:34.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.579+0000 7fe6740d1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe66c14d740 0x7fe66c1f0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:34.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.579+0000 7fe6740d1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe66c150960 0x7fe66c147a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:34.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.579+0000 7fe6740d1640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe66c15e980 con 0x7fe66c14d740 2024-04-25T10:30:34.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.579+0000 7fe6740d1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe66c15e6a0 con 0x7fe66c150960 2024-04-25T10:30:34.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.579+0000 7fe6740d1640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe66c0cbff0 con 0x7fe6580051c0 2024-04-25T10:30:34.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.579+0000 7fe67166c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe66c14d740 0x7fe66c1f0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:34.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.579+0000 7fe67166c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe66c14d740 0x7fe66c1f0490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47418/0 (socket says 172.21.15.106:47418) 2024-04-25T10:30:34.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.579+0000 7fe67166c640 1 -- 172.21.15.106:0/2823627960 learned_addr learned my addr 172.21.15.106:0/2823627960 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:34.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.579+0000 7fe67166c640 1 -- 172.21.15.106:0/2823627960 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6580051c0 msgr2=0x7fe66c1562e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:34.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.579+0000 7fe67166c640 1 --2- 172.21.15.106:0/2823627960 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6580051c0 0x7fe66c1562e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:34.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.579+0000 7fe67166c640 1 -- 172.21.15.106:0/2823627960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe66c150960 msgr2=0x7fe66c147a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:34.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.579+0000 7fe67166c640 1 --2- 172.21.15.106:0/2823627960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe66c150960 0x7fe66c147a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:34.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.579+0000 7fe67166c640 1 -- 172.21.15.106:0/2823627960 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe66c159bf0 con 0x7fe66c14d740 2024-04-25T10:30:34.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.579+0000 7fe67166c640 1 --2- 172.21.15.106:0/2823627960 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe66c14d740 0x7fe66c1f0490 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe65c00e730 tx=0x7fe65c00ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:34.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.579+0000 7fe6627fc640 1 -- 172.21.15.106:0/2823627960 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65c00ee40 con 0x7fe66c14d740 2024-04-25T10:30:34.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.579+0000 7fe6627fc640 1 -- 172.21.15.106:0/2823627960 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe65c018070 con 0x7fe66c14d740 2024-04-25T10:30:34.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.579+0000 7fe6740d1640 1 -- 172.21.15.106:0/2823627960 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe66c14d460 con 0x7fe66c14d740 2024-04-25T10:30:34.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.579+0000 7fe6627fc640 1 -- 172.21.15.106:0/2823627960 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65c00cb10 con 0x7fe66c14d740 2024-04-25T10:30:34.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.579+0000 7fe6740d1640 1 -- 172.21.15.106:0/2823627960 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe66c15a190 con 0x7fe66c14d740 2024-04-25T10:30:34.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.580+0000 7fe6740d1640 1 -- 172.21.15.106:0/2823627960 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe66c14d1b0 con 0x7fe66c14d740 2024-04-25T10:30:34.653 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.581+0000 7fe6627fc640 1 -- 172.21.15.106:0/2823627960 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe65c00ccb0 con 0x7fe66c14d740 2024-04-25T10:30:34.653 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.581+0000 7fe6627fc640 1 --2- 172.21.15.106:0/2823627960 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fe6500744e0 0x7fe6500769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:34.653 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.581+0000 7fe671e6d640 1 --2- 172.21.15.106:0/2823627960 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fe6500744e0 0x7fe6500769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:34.653 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.581+0000 7fe6627fc640 1 -- 172.21.15.106:0/2823627960 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe65c095030 con 0x7fe66c14d740 2024-04-25T10:30:34.653 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.582+0000 7fe671e6d640 1 --2- 172.21.15.106:0/2823627960 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fe6500744e0 0x7fe6500769f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe66c0749e0 tx=0x7fe668009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:34.653 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.586+0000 7fe6627fc640 1 -- 172.21.15.106:0/2823627960 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe65c05f7c0 con 0x7fe66c14d740 2024-04-25T10:30:34.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:34.798+0000 7fe6740d1640 1 -- 172.21.15.106:0/2823627960 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fe66c06a3d0 con 0x7fe66c14d740 2024-04-25T10:30:35.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.440+0000 7fe6627fc640 1 -- 172.21.15.106:0/2823627960 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v7) ==== 122+0+0 (secure 0 0 0) 0x7fe65c05f0c0 con 0x7fe66c14d740 2024-04-25T10:30:35.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.442+0000 7fe6740d1640 1 -- 172.21.15.106:0/2823627960 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fe6500744e0 msgr2=0x7fe6500769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:35.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.442+0000 7fe6740d1640 1 --2- 172.21.15.106:0/2823627960 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fe6500744e0 0x7fe6500769f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe66c0749e0 tx=0x7fe668009040 comp rx=0 tx=0).stop 2024-04-25T10:30:35.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.442+0000 7fe6740d1640 1 -- 172.21.15.106:0/2823627960 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe66c14d740 msgr2=0x7fe66c1f0490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:35.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.443+0000 7fe6740d1640 1 --2- 172.21.15.106:0/2823627960 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe66c14d740 0x7fe66c1f0490 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe65c00e730 tx=0x7fe65c00ebf0 comp rx=0 tx=0).stop 2024-04-25T10:30:35.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.443+0000 7fe6740d1640 1 -- 172.21.15.106:0/2823627960 shutdown_connections 2024-04-25T10:30:35.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.443+0000 7fe6740d1640 1 --2- 172.21.15.106:0/2823627960 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fe6500744e0 0x7fe6500769f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:35.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.443+0000 7fe6740d1640 1 --2- 172.21.15.106:0/2823627960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe66c150960 0x7fe66c147a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:35.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.443+0000 7fe6740d1640 1 --2- 172.21.15.106:0/2823627960 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe66c14d740 0x7fe66c1f0490 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:35.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.443+0000 7fe6740d1640 1 --2- 172.21.15.106:0/2823627960 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6580051c0 0x7fe66c1562e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:35.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.443+0000 7fe6740d1640 1 -- 172.21.15.106:0/2823627960 >> 172.21.15.106:0/2823627960 conn(0x7fe66c0727d0 msgr2=0x7fe66c1537f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:35.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.443+0000 7fe6740d1640 1 -- 172.21.15.106:0/2823627960 shutdown_connections 2024-04-25T10:30:35.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.443+0000 7fe6740d1640 1 -- 172.21.15.106:0/2823627960 wait complete. 2024-04-25T10:30:35.458 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-25T10:30:35.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.571+0000 7f5f0853f640 1 Processor -- start 2024-04-25T10:30:35.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.571+0000 7f5f0853f640 1 -- start start 2024-04-25T10:30:35.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.572+0000 7f5f0853f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5f000cc600 0x7f5f000c5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:35.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.573+0000 7f5f0853f640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f0015b8c0 con 0x7f5f000c58d0 2024-04-25T10:30:35.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.573+0000 7f5f0853f640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f0015b5e0 con 0x7f5f000cb2b0 2024-04-25T10:30:35.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.573+0000 7f5f0853f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f5f0015eea0 con 0x7f5f000cc600 2024-04-25T10:30:35.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.573+0000 7f5f05ada640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f5f000c58d0 0x7f5f000cce90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:44214/0 (socket says 172.21.15.106:44214) 2024-04-25T10:30:35.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.573+0000 7f5f05ada640 1 -- 172.21.15.106:0/681847831 learned_addr learned my addr 172.21.15.106:0/681847831 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:30:35.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.573+0000 7f5f062db640 1 --2- 172.21.15.106:0/681847831 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5f000cc600 0x7f5f000c5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:35.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.573+0000 7f5f052d9640 1 -- 172.21.15.106:0/681847831 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 814180132 0 0) 0x7f5f0015b8c0 con 0x7f5f000c58d0 2024-04-25T10:30:35.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.573+0000 7f5f052d9640 1 -- 172.21.15.106:0/681847831 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5ee8003520 con 0x7f5f000c58d0 2024-04-25T10:30:35.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.573+0000 7f5f052d9640 1 -- 172.21.15.106:0/681847831 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 944413850 0 0) 0x7f5f0015b5e0 con 0x7f5f000cb2b0 2024-04-25T10:30:35.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.573+0000 7f5f052d9640 1 -- 172.21.15.106:0/681847831 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5f0015b8c0 con 0x7f5f000cb2b0 2024-04-25T10:30:35.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.573+0000 7f5f052d9640 1 -- 172.21.15.106:0/681847831 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1914437416 0 0) 0x7f5ee8003520 con 0x7f5f000c58d0 2024-04-25T10:30:35.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.573+0000 7f5f052d9640 1 -- 172.21.15.106:0/681847831 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5f000cc600 msgr2=0x7f5f000c5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:35.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.573+0000 7f5f052d9640 1 --2- 172.21.15.106:0/681847831 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5f000cc600 0x7f5f000c5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:35.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.573+0000 7f5f052d9640 1 -- 172.21.15.106:0/681847831 >> v1:172.21.15.134:6789/0 conn(0x7f5f000cb2b0 legacy=0x7f5f00165340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:35.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.573+0000 7f5f052d9640 1 -- 172.21.15.106:0/681847831 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f0015e0e0 con 0x7f5f000c58d0 2024-04-25T10:30:35.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.573+0000 7f5f052d9640 1 -- 172.21.15.106:0/681847831 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3776459350 0 0) 0x7f5ef4001670 con 0x7f5f000c58d0 2024-04-25T10:30:35.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.573+0000 7f5f052d9640 1 -- 172.21.15.106:0/681847831 >> v1:172.21.15.106:6789/0 conn(0x7f5f000c58d0 legacy=0x7f5f000cce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:35.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.573+0000 7f5f052d9640 1 --2- 172.21.15.106:0/681847831 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ee8003b70 0x7f5ee8003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:35.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.573+0000 7f5f052d9640 1 --2- 172.21.15.106:0/681847831 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ee8004800 0x7f5ee8004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:35.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.573+0000 7f5f052d9640 1 --2- 172.21.15.106:0/681847831 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5f000cc600 0x7f5ee8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:35.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.573+0000 7f5f052d9640 1 -- 172.21.15.106:0/681847831 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5f0015eea0 con 0x7f5ee8003b70 2024-04-25T10:30:35.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.573+0000 7f5f052d9640 1 -- 172.21.15.106:0/681847831 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee800bad0 con 0x7f5ee8004800 2024-04-25T10:30:35.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.573+0000 7f5f052d9640 1 -- 172.21.15.106:0/681847831 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f5ee800bcd0 con 0x7f5f000cc600 2024-04-25T10:30:35.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.573+0000 7f5f062db640 1 --2- 172.21.15.106:0/681847831 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ee8003b70 0x7f5ee8003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:35.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.574+0000 7f5f05ada640 1 --2- 172.21.15.106:0/681847831 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ee8004800 0x7f5ee8004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:35.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.574+0000 7f5f06adc640 1 --2- 172.21.15.106:0/681847831 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5f000cc600 0x7f5ee8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:35.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.574+0000 7f5f062db640 1 -- 172.21.15.106:0/681847831 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5f000cc600 msgr2=0x7f5ee8005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:35.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.574+0000 7f5f062db640 1 --2- 172.21.15.106:0/681847831 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5f000cc600 0x7f5ee8005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:35.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.574+0000 7f5f062db640 1 -- 172.21.15.106:0/681847831 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ee8004800 msgr2=0x7f5ee8004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:35.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.574+0000 7f5f062db640 1 --2- 172.21.15.106:0/681847831 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ee8004800 0x7f5ee8004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:35.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.574+0000 7f5f062db640 1 -- 172.21.15.106:0/681847831 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ee8003520 con 0x7f5ee8003b70 2024-04-25T10:30:35.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.574+0000 7f5f062db640 1 --2- 172.21.15.106:0/681847831 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ee8003b70 0x7f5ee8003fc0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0009f90 tx=0x7f5ee8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdc49aa9f5fc82fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:35.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.575+0000 7f5f052d9640 1 -- 172.21.15.106:0/681847831 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef0009c10 con 0x7f5ee8003b70 2024-04-25T10:30:35.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.575+0000 7f5f052d9640 1 -- 172.21.15.106:0/681847831 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ef0036070 con 0x7f5ee8003b70 2024-04-25T10:30:35.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.575+0000 7f5f052d9640 1 -- 172.21.15.106:0/681847831 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef0031810 con 0x7f5ee8003b70 2024-04-25T10:30:35.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.575+0000 7f5f0853f640 1 -- 172.21.15.106:0/681847831 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ee8003b70 msgr2=0x7f5ee8003fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:35.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.575+0000 7f5f0853f640 1 --2- 172.21.15.106:0/681847831 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ee8003b70 0x7f5ee8003fc0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0009f90 tx=0x7f5ee8005830 comp rx=0 tx=0).stop 2024-04-25T10:30:35.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.575+0000 7f5f06adc640 1 -- 172.21.15.106:0/681847831 reap_dead start 2024-04-25T10:30:35.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.575+0000 7f5f0853f640 1 -- 172.21.15.106:0/681847831 shutdown_connections 2024-04-25T10:30:35.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.575+0000 7f5f0853f640 1 -- 172.21.15.106:0/681847831 >> 172.21.15.106:0/681847831 conn(0x7f5f000727d0 msgr2=0x7f5f00072c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:35.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.575+0000 7f5f0853f640 1 -- 172.21.15.106:0/681847831 shutdown_connections 2024-04-25T10:30:35.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.575+0000 7f5f0853f640 1 -- 172.21.15.106:0/681847831 wait complete. 2024-04-25T10:30:35.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.577+0000 7f5f0853f640 1 Processor -- start 2024-04-25T10:30:35.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.577+0000 7f5f0853f640 1 -- start start 2024-04-25T10:30:35.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.578+0000 7f5f0853f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ee8003b70 0x7f5f001f0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:35.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.578+0000 7f5f0853f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f000c8b80 0x7f5f000c8f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:35.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.578+0000 7f5f0853f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f0015cf10 0x7f5f0015d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:35.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.578+0000 7f5f0853f640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5f0015e980 con 0x7f5f0015cf10 2024-04-25T10:30:35.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.578+0000 7f5f0853f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5f0015e6a0 con 0x7f5f000c8b80 2024-04-25T10:30:35.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.578+0000 7f5f0853f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f5f000cbff0 con 0x7f5ee8003b70 2024-04-25T10:30:35.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.578+0000 7f5f06adc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f0015cf10 0x7f5f0015d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:35.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.578+0000 7f5f06adc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f0015cf10 0x7f5f0015d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47436/0 (socket says 172.21.15.106:47436) 2024-04-25T10:30:35.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.578+0000 7f5f06adc640 1 -- 172.21.15.106:0/542696941 learned_addr learned my addr 172.21.15.106:0/542696941 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:35.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.578+0000 7f5f062db640 1 --2- 172.21.15.106:0/542696941 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ee8003b70 0x7f5f001f0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:35.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.578+0000 7f5f05ada640 1 --2- 172.21.15.106:0/542696941 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f000c8b80 0x7f5f000c8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:35.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.579+0000 7f5f06adc640 1 -- 172.21.15.106:0/542696941 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ee8003b70 msgr2=0x7f5f001f0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:35.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.579+0000 7f5f06adc640 1 --2- 172.21.15.106:0/542696941 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ee8003b70 0x7f5f001f0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:35.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.579+0000 7f5f06adc640 1 -- 172.21.15.106:0/542696941 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f000c8b80 msgr2=0x7f5f000c8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:35.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.579+0000 7f5f06adc640 1 --2- 172.21.15.106:0/542696941 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f000c8b80 0x7f5f000c8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:35.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.579+0000 7f5f06adc640 1 -- 172.21.15.106:0/542696941 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f001f0ed0 con 0x7f5f0015cf10 2024-04-25T10:30:35.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.579+0000 7f5f05ada640 1 --2- 172.21.15.106:0/542696941 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f000c8b80 0x7f5f000c8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:35.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.579+0000 7f5f06adc640 1 --2- 172.21.15.106:0/542696941 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f0015cf10 0x7f5f0015d320 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5f000c5130 tx=0x7f5ef800e820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:35.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.579+0000 7f5efeffd640 1 -- 172.21.15.106:0/542696941 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef80050d0 con 0x7f5f0015cf10 2024-04-25T10:30:35.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.579+0000 7f5efeffd640 1 -- 172.21.15.106:0/542696941 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ef8005290 con 0x7f5f0015cf10 2024-04-25T10:30:35.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.579+0000 7f5f0853f640 1 -- 172.21.15.106:0/542696941 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f001f3ec0 con 0x7f5f0015cf10 2024-04-25T10:30:35.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.579+0000 7f5efeffd640 1 -- 172.21.15.106:0/542696941 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef8006c20 con 0x7f5f0015cf10 2024-04-25T10:30:35.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.579+0000 7f5f0853f640 1 -- 172.21.15.106:0/542696941 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f001f4480 con 0x7f5f0015cf10 2024-04-25T10:30:35.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.580+0000 7f5f0853f640 1 -- 172.21.15.106:0/542696941 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f0006a3d0 con 0x7f5f0015cf10 2024-04-25T10:30:35.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.581+0000 7f5efeffd640 1 -- 172.21.15.106:0/542696941 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5ef8007030 con 0x7f5f0015cf10 2024-04-25T10:30:35.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.581+0000 7f5efeffd640 1 --2- 172.21.15.106:0/542696941 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f5eec0744e0 0x7f5eec0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:35.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.581+0000 7f5f062db640 1 --2- 172.21.15.106:0/542696941 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f5eec0744e0 0x7f5eec0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:35.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.581+0000 7f5efeffd640 1 -- 172.21.15.106:0/542696941 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5ef8094940 con 0x7f5f0015cf10 2024-04-25T10:30:35.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.581+0000 7f5f062db640 1 --2- 172.21.15.106:0/542696941 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f5eec0744e0 0x7f5eec0769f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0035000 tx=0x7f5ef002f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:35.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.586+0000 7f5efeffd640 1 -- 172.21.15.106:0/542696941 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5ef801b060 con 0x7f5f0015cf10 2024-04-25T10:30:35.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:35.797+0000 7f5f0853f640 1 -- 172.21.15.106:0/542696941 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f5f000cbdd0 con 0x7f5f0015cf10 2024-04-25T10:30:36.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.448+0000 7f5efeffd640 1 -- 172.21.15.106:0/542696941 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v8) ==== 115+0+0 (secure 0 0 0) 0x7f5ef805f1f0 con 0x7f5f0015cf10 2024-04-25T10:30:36.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.451+0000 7f5f0853f640 1 -- 172.21.15.106:0/542696941 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f5eec0744e0 msgr2=0x7f5eec0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:36.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.451+0000 7f5f0853f640 1 --2- 172.21.15.106:0/542696941 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f5eec0744e0 0x7f5eec0769f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0035000 tx=0x7f5ef002f000 comp rx=0 tx=0).stop 2024-04-25T10:30:36.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.451+0000 7f5f0853f640 1 -- 172.21.15.106:0/542696941 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f0015cf10 msgr2=0x7f5f0015d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:36.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.451+0000 7f5f0853f640 1 --2- 172.21.15.106:0/542696941 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f0015cf10 0x7f5f0015d320 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5f000c5130 tx=0x7f5ef800e820 comp rx=0 tx=0).stop 2024-04-25T10:30:36.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.451+0000 7f5f0853f640 1 -- 172.21.15.106:0/542696941 shutdown_connections 2024-04-25T10:30:36.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.451+0000 7f5f0853f640 1 --2- 172.21.15.106:0/542696941 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f5eec0744e0 0x7f5eec0769f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:36.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.451+0000 7f5f0853f640 1 --2- 172.21.15.106:0/542696941 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f0015cf10 0x7f5f0015d320 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:36.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.451+0000 7f5f0853f640 1 --2- 172.21.15.106:0/542696941 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f000c8b80 0x7f5f000c8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:36.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.451+0000 7f5f0853f640 1 --2- 172.21.15.106:0/542696941 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ee8003b70 0x7f5f001f0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:36.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.451+0000 7f5f0853f640 1 -- 172.21.15.106:0/542696941 >> 172.21.15.106:0/542696941 conn(0x7f5f000727d0 msgr2=0x7f5f00075ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:36.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.451+0000 7f5f0853f640 1 -- 172.21.15.106:0/542696941 shutdown_connections 2024-04-25T10:30:36.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.451+0000 7f5f0853f640 1 -- 172.21.15.106:0/542696941 wait complete. 2024-04-25T10:30:36.465 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:30:36.466 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:30:36.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.573+0000 7f0fc439d640 1 Processor -- start 2024-04-25T10:30:36.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.573+0000 7f0fc439d640 1 -- start start 2024-04-25T10:30:36.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.574+0000 7f0fc439d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f0fbc072870 0x7f0fbc15a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:36.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.574+0000 7f0fc439d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0fbc0c5280 0x7f0fbc15d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:36.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.574+0000 7f0fc439d640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f0fbc0cae80 0x7f0fbc153710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:36.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.574+0000 7f0fc439d640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f0fbc0c5690 con 0x7f0fbc072870 2024-04-25T10:30:36.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.574+0000 7f0fc439d640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f0fbc0c7c40 con 0x7f0fbc0cae80 2024-04-25T10:30:36.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.574+0000 7f0fc439d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f0fbc0c7f20 con 0x7f0fbc0c5280 2024-04-25T10:30:36.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.575+0000 7f0fc2139640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f0fbc072870 0x7f0fbc15a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:36.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.575+0000 7f0fc2139640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f0fbc072870 0x7f0fbc15a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47444/0 (socket says 172.21.15.106:47444) 2024-04-25T10:30:36.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.575+0000 7f0fc2139640 1 -- 172.21.15.106:0/2036696412 learned_addr learned my addr 172.21.15.106:0/2036696412 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:36.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.575+0000 7f0fc1938640 1 --2- 172.21.15.106:0/2036696412 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0fbc0c5280 0x7f0fbc15d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:36.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.575+0000 7f0fc293a640 1 --2- 172.21.15.106:0/2036696412 >> v2:172.21.15.134:3300/0 conn(0x7f0fbc0cae80 0x7f0fbc153710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:36.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.575+0000 7f0fc2139640 1 -- 172.21.15.106:0/2036696412 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0fbc0c5280 msgr2=0x7f0fbc15d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:36.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.575+0000 7f0fc2139640 1 --2- 172.21.15.106:0/2036696412 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0fbc0c5280 0x7f0fbc15d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:36.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.575+0000 7f0fc2139640 1 -- 172.21.15.106:0/2036696412 >> v2:172.21.15.134:3300/0 conn(0x7f0fbc0cae80 msgr2=0x7f0fbc153710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:36.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.575+0000 7f0fc2139640 1 --2- 172.21.15.106:0/2036696412 >> v2:172.21.15.134:3300/0 conn(0x7f0fbc0cae80 0x7f0fbc153710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:36.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.575+0000 7f0fc2139640 1 -- 172.21.15.106:0/2036696412 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fbc0c8200 con 0x7f0fbc072870 2024-04-25T10:30:36.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.576+0000 7f0fc2139640 1 --2- 172.21.15.106:0/2036696412 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0fbc072870 0x7f0fbc15a590 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc06f460 tx=0x7f0fb402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=594b68efe6808791 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:36.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.576+0000 7f0fc1137640 1 -- 172.21.15.106:0/2036696412 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb4036070 con 0x7f0fbc072870 2024-04-25T10:30:36.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.576+0000 7f0fc1137640 1 -- 172.21.15.106:0/2036696412 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fb4031600 con 0x7f0fbc072870 2024-04-25T10:30:36.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.576+0000 7f0fc1137640 1 -- 172.21.15.106:0/2036696412 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb40319d0 con 0x7f0fbc072870 2024-04-25T10:30:36.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.576+0000 7f0fc439d640 1 -- 172.21.15.106:0/2036696412 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0fbc072870 msgr2=0x7f0fbc15a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:36.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.576+0000 7f0fc439d640 1 --2- 172.21.15.106:0/2036696412 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0fbc072870 0x7f0fbc15a590 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc06f460 tx=0x7f0fb402a610 comp rx=0 tx=0).stop 2024-04-25T10:30:36.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.577+0000 7f0fc439d640 1 -- 172.21.15.106:0/2036696412 shutdown_connections 2024-04-25T10:30:36.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.577+0000 7f0fc439d640 1 --2- 172.21.15.106:0/2036696412 >> v2:172.21.15.134:3300/0 conn(0x7f0fbc0cae80 0x7f0fbc153710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:36.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.577+0000 7f0fc439d640 1 --2- 172.21.15.106:0/2036696412 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0fbc0c5280 0x7f0fbc15d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:36.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.577+0000 7f0fc439d640 1 --2- 172.21.15.106:0/2036696412 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0fbc072870 0x7f0fbc15a590 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:36.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.577+0000 7f0fc439d640 1 -- 172.21.15.106:0/2036696412 >> 172.21.15.106:0/2036696412 conn(0x7f0fbc06c9b0 msgr2=0x7f0fbc06cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:36.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.577+0000 7f0fc439d640 1 -- 172.21.15.106:0/2036696412 shutdown_connections 2024-04-25T10:30:36.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.577+0000 7f0fc439d640 1 -- 172.21.15.106:0/2036696412 wait complete. 2024-04-25T10:30:36.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.578+0000 7f0fc439d640 1 Processor -- start 2024-04-25T10:30:36.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.578+0000 7f0fc439d640 1 -- start start 2024-04-25T10:30:36.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.579+0000 7f0fc439d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0fbc072870 0x7f0fbc14aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:36.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.579+0000 7f0fc439d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0fbc0c5280 0x7f0fbc1e5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:36.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.579+0000 7f0fc439d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0fbc0cae80 0x7f0fbc148890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:36.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.579+0000 7f0fc439d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0fbc15a070 con 0x7f0fbc0c5280 2024-04-25T10:30:36.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.579+0000 7f0fc439d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0fbc159d90 con 0x7f0fbc0cae80 2024-04-25T10:30:36.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.579+0000 7f0fc439d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f0fbc0c7920 con 0x7f0fbc072870 2024-04-25T10:30:36.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.580+0000 7f0fc1938640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0fbc0c5280 0x7f0fbc1e5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:36.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.580+0000 7f0fc1938640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0fbc0c5280 0x7f0fbc1e5410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47458/0 (socket says 172.21.15.106:47458) 2024-04-25T10:30:36.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.580+0000 7f0fc1938640 1 -- 172.21.15.106:0/721365908 learned_addr learned my addr 172.21.15.106:0/721365908 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:36.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.580+0000 7f0fc2139640 1 --2- 172.21.15.106:0/721365908 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0fbc072870 0x7f0fbc14aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:36.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.580+0000 7f0fc293a640 1 --2- 172.21.15.106:0/721365908 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0fbc0cae80 0x7f0fbc148890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:36.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.580+0000 7f0fc1938640 1 -- 172.21.15.106:0/721365908 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0fbc072870 msgr2=0x7f0fbc14aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:36.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.580+0000 7f0fc1938640 1 --2- 172.21.15.106:0/721365908 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0fbc072870 0x7f0fbc14aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:36.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.580+0000 7f0fc1938640 1 -- 172.21.15.106:0/721365908 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0fbc0cae80 msgr2=0x7f0fbc148890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:36.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.580+0000 7f0fc1938640 1 --2- 172.21.15.106:0/721365908 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0fbc0cae80 0x7f0fbc148890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:36.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.580+0000 7f0fc1938640 1 -- 172.21.15.106:0/721365908 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fbc14afd0 con 0x7f0fbc0c5280 2024-04-25T10:30:36.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.580+0000 7f0fc293a640 1 --2- 172.21.15.106:0/721365908 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0fbc0cae80 0x7f0fbc148890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:36.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.580+0000 7f0fc1938640 1 --2- 172.21.15.106:0/721365908 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0fbc0c5280 0x7f0fbc1e5410 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0fb000d5c0 tx=0x7f0fb000da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:36.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.580+0000 7f0fbaffd640 1 -- 172.21.15.106:0/721365908 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb0004240 con 0x7f0fbc0c5280 2024-04-25T10:30:36.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.580+0000 7f0fbaffd640 1 -- 172.21.15.106:0/721365908 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fb0019070 con 0x7f0fbc0c5280 2024-04-25T10:30:36.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.580+0000 7f0fbaffd640 1 -- 172.21.15.106:0/721365908 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb0003610 con 0x7f0fbc0c5280 2024-04-25T10:30:36.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.580+0000 7f0fc439d640 1 -- 172.21.15.106:0/721365908 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fbc1e3940 con 0x7f0fbc0c5280 2024-04-25T10:30:36.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.580+0000 7f0fc439d640 1 -- 172.21.15.106:0/721365908 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fbc1e3ee0 con 0x7f0fbc0c5280 2024-04-25T10:30:36.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.581+0000 7f0fc439d640 1 -- 172.21.15.106:0/721365908 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fbc06a3d0 con 0x7f0fbc0c5280 2024-04-25T10:30:36.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.582+0000 7f0fbaffd640 1 -- 172.21.15.106:0/721365908 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0fb00037b0 con 0x7f0fbc0c5280 2024-04-25T10:30:36.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.582+0000 7f0fbaffd640 1 --2- 172.21.15.106:0/721365908 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f0fa80744e0 0x7f0fa80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:36.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.582+0000 7f0fc2139640 1 --2- 172.21.15.106:0/721365908 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f0fa80744e0 0x7f0fa80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:36.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.582+0000 7f0fbaffd640 1 -- 172.21.15.106:0/721365908 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0fb00953d0 con 0x7f0fbc0c5280 2024-04-25T10:30:36.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.583+0000 7f0fc2139640 1 --2- 172.21.15.106:0/721365908 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f0fa80744e0 0x7f0fa80769f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0fb4033000 tx=0x7f0fb4009000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:36.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.587+0000 7f0fbaffd640 1 -- 172.21.15.106:0/721365908 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0fb005fbf0 con 0x7f0fbc0c5280 2024-04-25T10:30:36.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.796+0000 7f0fc439d640 1 -- 172.21.15.106:0/721365908 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0fbc0c4cf0 con 0x7f0fbc0c5280 2024-04-25T10:30:36.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.797+0000 7f0fbaffd640 1 -- 172.21.15.106:0/721365908 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6834 (secure 0 0 0) 0x7f0fb005f4f0 con 0x7f0fbc0c5280 2024-04-25T10:30:36.799 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:30:36.799 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/2834155852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":2834155852},{"type":"v1","addr":"172.21.15.106:6835","nonce":2834155852}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/2632121774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":2632121774},{"type":"v1","addr":"172.21.15.106:6837","nonce":2632121774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4269,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6839/486339834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":486339834},{"type":"v1","addr":"172.21.15.106:6839","nonce":486339834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4284,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/2781656864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6833","nonce":2781656864},{"type":"v1","addr":"172.21.15.134:6836","nonce":2781656864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4287,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/1993918351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1993918351},{"type":"v1","addr":"172.21.15.134:6835","nonce":1993918351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:30:31.761107+0000","modified":"2024-04-25T10:30:35.799104+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0],"up":{"mds_0":4290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"d","rank":0,"incarnation":4,"state":"up:creating","state_seq":1,"addr":"172.21.15.134:6837/2662003420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2662003420},{"type":"v1","addr":"172.21.15.134:6837","nonce":2662003420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-25T10:30:36.800 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 8 2024-04-25T10:30:36.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.799+0000 7f0fc439d640 1 -- 172.21.15.106:0/721365908 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f0fa80744e0 msgr2=0x7f0fa80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:36.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.799+0000 7f0fc439d640 1 --2- 172.21.15.106:0/721365908 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f0fa80744e0 0x7f0fa80769f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0fb4033000 tx=0x7f0fb4009000 comp rx=0 tx=0).stop 2024-04-25T10:30:36.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.799+0000 7f0fc439d640 1 -- 172.21.15.106:0/721365908 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0fbc0c5280 msgr2=0x7f0fbc1e5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:36.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.799+0000 7f0fc439d640 1 --2- 172.21.15.106:0/721365908 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0fbc0c5280 0x7f0fbc1e5410 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0fb000d5c0 tx=0x7f0fb000da80 comp rx=0 tx=0).stop 2024-04-25T10:30:36.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.799+0000 7f0fc439d640 1 -- 172.21.15.106:0/721365908 shutdown_connections 2024-04-25T10:30:36.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.799+0000 7f0fc439d640 1 --2- 172.21.15.106:0/721365908 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f0fa80744e0 0x7f0fa80769f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:36.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.799+0000 7f0fc439d640 1 --2- 172.21.15.106:0/721365908 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0fbc0cae80 0x7f0fbc148890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:36.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.799+0000 7f0fc439d640 1 --2- 172.21.15.106:0/721365908 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0fbc0c5280 0x7f0fbc1e5410 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:36.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.799+0000 7f0fc439d640 1 --2- 172.21.15.106:0/721365908 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0fbc072870 0x7f0fbc14aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:36.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.799+0000 7f0fc439d640 1 -- 172.21.15.106:0/721365908 >> 172.21.15.106:0/721365908 conn(0x7f0fbc06c9b0 msgr2=0x7f0fbc06cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:36.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.799+0000 7f0fc439d640 1 -- 172.21.15.106:0/721365908 shutdown_connections 2024-04-25T10:30:36.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.799+0000 7f0fc439d640 1 -- 172.21.15.106:0/721365908 wait complete. 2024-04-25T10:30:36.812 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:30:36.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.916+0000 7efd32814640 1 Processor -- start 2024-04-25T10:30:36.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.917+0000 7efd32814640 1 -- start start 2024-04-25T10:30:36.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.918+0000 7efd32814640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7efd2c153180 0x7efd2c06ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:36.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.918+0000 7efd32814640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efd2c153590 0x7efd2c070510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:36.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.918+0000 7efd32814640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd2c074790 con 0x7efd2c0c9c00 2024-04-25T10:30:36.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.918+0000 7efd32814640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7efd2c0741d0 con 0x7efd2c153180 2024-04-25T10:30:36.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.918+0000 7efd32814640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7efd2c0744b0 con 0x7efd2c153590 2024-04-25T10:30:36.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.918+0000 7efd2bfff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7efd2c0c9c00 0x7efd2c0c9190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:44222/0 (socket says 172.21.15.106:44222) 2024-04-25T10:30:36.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.918+0000 7efd2bfff640 1 -- 172.21.15.106:0/748243325 learned_addr learned my addr 172.21.15.106:0/748243325 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:30:36.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.918+0000 7efd30db1640 1 --2- 172.21.15.106:0/748243325 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efd2c153590 0x7efd2c070510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:36.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.918+0000 7efd2b7fe640 1 --2- 172.21.15.106:0/748243325 >> v2:172.21.15.134:3300/0 conn(0x7efd2c153180 0x7efd2c06ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:36.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.918+0000 7efd2affd640 1 -- 172.21.15.106:0/748243325 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 753644959 0 0) 0x7efd2c074790 con 0x7efd2c0c9c00 2024-04-25T10:30:36.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.918+0000 7efd2affd640 1 -- 172.21.15.106:0/748243325 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efd14004630 con 0x7efd2c0c9c00 2024-04-25T10:30:36.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.918+0000 7efd2affd640 1 -- 172.21.15.106:0/748243325 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3036909290 0 0) 0x7efd14004630 con 0x7efd2c0c9c00 2024-04-25T10:30:36.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.918+0000 7efd2affd640 1 -- 172.21.15.106:0/748243325 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efd2c153590 msgr2=0x7efd2c070510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:36.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.919+0000 7efd2affd640 1 --2- 172.21.15.106:0/748243325 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efd2c153590 0x7efd2c070510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:36.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.919+0000 7efd2affd640 1 -- 172.21.15.106:0/748243325 >> v2:172.21.15.134:3300/0 conn(0x7efd2c153180 msgr2=0x7efd2c06ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:36.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.919+0000 7efd2affd640 1 --2- 172.21.15.106:0/748243325 >> v2:172.21.15.134:3300/0 conn(0x7efd2c153180 0x7efd2c06ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:36.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.919+0000 7efd2affd640 1 -- 172.21.15.106:0/748243325 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd2c0c5320 con 0x7efd2c0c9c00 2024-04-25T10:30:36.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.919+0000 7efd2affd640 1 -- 172.21.15.106:0/748243325 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3776459350 0 0) 0x7efd1c001670 con 0x7efd2c0c9c00 2024-04-25T10:30:36.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.919+0000 7efd30db1640 1 --2- 172.21.15.106:0/748243325 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efd2c153590 0x7efd2c070510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:36.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.919+0000 7efd2b7fe640 1 --2- 172.21.15.106:0/748243325 >> v2:172.21.15.134:3300/0 conn(0x7efd2c153180 0x7efd2c06ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:36.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.919+0000 7efd2affd640 1 -- 172.21.15.106:0/748243325 >> v1:172.21.15.106:6789/0 conn(0x7efd2c0c9c00 legacy=0x7efd2c0c9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:36.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.919+0000 7efd2affd640 1 --2- 172.21.15.106:0/748243325 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efd14004ac0 0x7efd14004f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:36.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.919+0000 7efd2affd640 1 --2- 172.21.15.106:0/748243325 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd14005910 0x7efd14005d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:36.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.919+0000 7efd2affd640 1 --2- 172.21.15.106:0/748243325 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efd2c153590 0x7efd14006260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:36.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.919+0000 7efd2affd640 1 -- 172.21.15.106:0/748243325 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7efd2c0741d0 con 0x7efd14005910 2024-04-25T10:30:36.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.919+0000 7efd2affd640 1 -- 172.21.15.106:0/748243325 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efd2c0744b0 con 0x7efd14004ac0 2024-04-25T10:30:36.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.919+0000 7efd2affd640 1 -- 172.21.15.106:0/748243325 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7efd14003490 con 0x7efd2c153590 2024-04-25T10:30:36.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.919+0000 7efd2b7fe640 1 --2- 172.21.15.106:0/748243325 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd14005910 0x7efd14005d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:36.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.919+0000 7efd2bfff640 1 --2- 172.21.15.106:0/748243325 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efd14004ac0 0x7efd14004f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:36.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.919+0000 7efd30db1640 1 --2- 172.21.15.106:0/748243325 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efd2c153590 0x7efd14006260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:36.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.919+0000 7efd2b7fe640 1 -- 172.21.15.106:0/748243325 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efd2c153590 msgr2=0x7efd14006260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:36.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.919+0000 7efd2b7fe640 1 --2- 172.21.15.106:0/748243325 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efd2c153590 0x7efd14006260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:36.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.919+0000 7efd2b7fe640 1 -- 172.21.15.106:0/748243325 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efd14004ac0 msgr2=0x7efd14004f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:36.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.919+0000 7efd2b7fe640 1 --2- 172.21.15.106:0/748243325 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efd14004ac0 0x7efd14004f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:36.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.919+0000 7efd2b7fe640 1 -- 172.21.15.106:0/748243325 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd14004630 con 0x7efd14005910 2024-04-25T10:30:36.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.920+0000 7efd2b7fe640 1 --2- 172.21.15.106:0/748243325 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd14005910 0x7efd14005d20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7efd20009c90 tx=0x7efd14006940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8e20a891cbf39ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:36.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.920+0000 7efd2affd640 1 -- 172.21.15.106:0/748243325 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd20036070 con 0x7efd14005910 2024-04-25T10:30:36.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.920+0000 7efd2affd640 1 -- 172.21.15.106:0/748243325 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd20031400 con 0x7efd14005910 2024-04-25T10:30:36.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.920+0000 7efd2affd640 1 -- 172.21.15.106:0/748243325 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd20009860 con 0x7efd14005910 2024-04-25T10:30:36.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.920+0000 7efd32814640 1 -- 172.21.15.106:0/748243325 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd14005910 msgr2=0x7efd14005d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:36.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.920+0000 7efd32814640 1 --2- 172.21.15.106:0/748243325 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd14005910 0x7efd14005d20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7efd20009c90 tx=0x7efd14006940 comp rx=0 tx=0).stop 2024-04-25T10:30:36.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.920+0000 7efd30db1640 1 -- 172.21.15.106:0/748243325 reap_dead start 2024-04-25T10:30:36.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.921+0000 7efd32814640 1 -- 172.21.15.106:0/748243325 shutdown_connections 2024-04-25T10:30:36.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.921+0000 7efd32814640 1 -- 172.21.15.106:0/748243325 >> 172.21.15.106:0/748243325 conn(0x7efd2c06c9b0 msgr2=0x7efd2c06cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:36.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.921+0000 7efd32814640 1 -- 172.21.15.106:0/748243325 shutdown_connections 2024-04-25T10:30:36.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.921+0000 7efd32814640 1 -- 172.21.15.106:0/748243325 wait complete. 2024-04-25T10:30:36.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.922+0000 7efd32814640 1 Processor -- start 2024-04-25T10:30:36.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.922+0000 7efd32814640 1 -- start start 2024-04-25T10:30:36.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.924+0000 7efd32814640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd14005910 0x7efd2c1539f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:36.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.924+0000 7efd32814640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efd2c148d60 0x7efd2c14a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:36.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.924+0000 7efd32814640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efd2c14eb20 0x7efd2c0ce320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:36.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.924+0000 7efd32814640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7efd2c0ca5e0 con 0x7efd14005910 2024-04-25T10:30:36.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.924+0000 7efd32814640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efd2c0c58e0 con 0x7efd2c14eb20 2024-04-25T10:30:36.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.924+0000 7efd32814640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7efd2c070510 con 0x7efd2c148d60 2024-04-25T10:30:36.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.924+0000 7efd2bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd14005910 0x7efd2c1539f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:36.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.924+0000 7efd2bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd14005910 0x7efd2c1539f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47470/0 (socket says 172.21.15.106:47470) 2024-04-25T10:30:36.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.924+0000 7efd2bfff640 1 -- 172.21.15.106:0/1957007515 learned_addr learned my addr 172.21.15.106:0/1957007515 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:36.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.924+0000 7efd2b7fe640 1 --2- 172.21.15.106:0/1957007515 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efd2c148d60 0x7efd2c14a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:36.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.924+0000 7efd30db1640 1 --2- 172.21.15.106:0/1957007515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efd2c14eb20 0x7efd2c0ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:36.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.924+0000 7efd2bfff640 1 -- 172.21.15.106:0/1957007515 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efd2c148d60 msgr2=0x7efd2c14a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:36.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.924+0000 7efd2bfff640 1 --2- 172.21.15.106:0/1957007515 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efd2c148d60 0x7efd2c14a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:36.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.924+0000 7efd2bfff640 1 -- 172.21.15.106:0/1957007515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efd2c14eb20 msgr2=0x7efd2c0ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:36.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.924+0000 7efd2bfff640 1 --2- 172.21.15.106:0/1957007515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efd2c14eb20 0x7efd2c0ce320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:36.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.924+0000 7efd2bfff640 1 -- 172.21.15.106:0/1957007515 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd2c1f1ea0 con 0x7efd14005910 2024-04-25T10:30:36.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.924+0000 7efd2b7fe640 1 --2- 172.21.15.106:0/1957007515 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efd2c148d60 0x7efd2c14a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:36.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.924+0000 7efd30db1640 1 --2- 172.21.15.106:0/1957007515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efd2c14eb20 0x7efd2c0ce320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:36.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.924+0000 7efd2bfff640 1 --2- 172.21.15.106:0/1957007515 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd14005910 0x7efd2c1539f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7efd1c007340 tx=0x7efd1c007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:36.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.924+0000 7efd28ff9640 1 -- 172.21.15.106:0/1957007515 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd1c01a070 con 0x7efd14005910 2024-04-25T10:30:36.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.924+0000 7efd28ff9640 1 -- 172.21.15.106:0/1957007515 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd1c00f8e0 con 0x7efd14005910 2024-04-25T10:30:36.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.924+0000 7efd32814640 1 -- 172.21.15.106:0/1957007515 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd2c1f2180 con 0x7efd14005910 2024-04-25T10:30:36.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.924+0000 7efd28ff9640 1 -- 172.21.15.106:0/1957007515 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd1c00fcb0 con 0x7efd14005910 2024-04-25T10:30:36.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.924+0000 7efd32814640 1 -- 172.21.15.106:0/1957007515 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd2c1f2740 con 0x7efd14005910 2024-04-25T10:30:36.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.925+0000 7efd32814640 1 -- 172.21.15.106:0/1957007515 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd2c06a3d0 con 0x7efd14005910 2024-04-25T10:30:36.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.926+0000 7efd28ff9640 1 -- 172.21.15.106:0/1957007515 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7efd1c01d070 con 0x7efd14005910 2024-04-25T10:30:36.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.926+0000 7efd28ff9640 1 --2- 172.21.15.106:0/1957007515 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7efd180744b0 0x7efd180769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:36.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.926+0000 7efd2b7fe640 1 --2- 172.21.15.106:0/1957007515 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7efd180744b0 0x7efd180769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:36.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.926+0000 7efd28ff9640 1 -- 172.21.15.106:0/1957007515 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7efd1c094c60 con 0x7efd14005910 2024-04-25T10:30:36.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.927+0000 7efd2b7fe640 1 --2- 172.21.15.106:0/1957007515 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7efd180744b0 0x7efd180769c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7efd20033000 tx=0x7efd20008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:36.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:36.931+0000 7efd28ff9640 1 -- 172.21.15.106:0/1957007515 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efd1c05f480 con 0x7efd14005910 2024-04-25T10:30:37.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.107+0000 7efd32814640 1 -- 172.21.15.106:0/1957007515 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efd2c158d70 con 0x7efd14005910 2024-04-25T10:30:37.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.108+0000 7efd28ff9640 1 -- 172.21.15.106:0/1957007515 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16641 (secure 0 0 0) 0x7efd1c05ed80 con 0x7efd14005910 2024-04-25T10:30:37.111 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:30:37.111 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":20,"fsid":"3ee77985-e8e4-41cf-81e9-7c774cf7891f","created":"2024-04-25T10:30:07.981054+0000","modified":"2024-04-25T10:30:33.409267+0000","last_up_change":"2024-04-25T10:30:19.184325+0000","last_in_change":"2024-04-25T10:30:10.217553+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:30:20.111046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:30:30.314914+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:30:30.715500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a5962cb1-dbbd-47b1-8d15-9bae114bfdf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1240717079},{"type":"v1","addr":"172.21.15.106:6801","nonce":1240717079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1240717079},{"type":"v1","addr":"172.21.15.106:6803","nonce":1240717079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1240717079},{"type":"v1","addr":"172.21.15.106:6807","nonce":1240717079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1240717079},{"type":"v1","addr":"172.21.15.106:6805","nonce":1240717079}]},"public_addr":"172.21.15.106:6801/1240717079","cluster_addr":"172.21.15.106:6803/1240717079","heartbeat_back_addr":"172.21.15.106:6807/1240717079","heartbeat_front_addr":"172.21.15.106:6805/1240717079","state":["exists","up"]},{"osd":1,"uuid":"94436e13-4769-4752-8397-b5e396a6f7ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4089653051},{"type":"v1","addr":"172.21.15.106:6825","nonce":4089653051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4089653051},{"type":"v1","addr":"172.21.15.106:6827","nonce":4089653051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4089653051},{"type":"v1","addr":"172.21.15.106:6831","nonce":4089653051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4089653051},{"type":"v1","addr":"172.21.15.106:6829","nonce":4089653051}]},"public_addr":"172.21.15.106:6825/4089653051","cluster_addr":"172.21.15.106:6827/4089653051","heartbeat_back_addr":"172.21.15.106:6831/4089653051","heartbeat_front_addr":"172.21.15.106:6829/4089653051","state":["exists","up"]},{"osd":2,"uuid":"429aae0b-70c6-4e7b-9043-6a11334c7efc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":1441018239},{"type":"v1","addr":"172.21.15.106:6809","nonce":1441018239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1441018239},{"type":"v1","addr":"172.21.15.106:6811","nonce":1441018239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1441018239},{"type":"v1","addr":"172.21.15.106:6815","nonce":1441018239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":1441018239},{"type":"v1","addr":"172.21.15.106:6813","nonce":1441018239}]},"public_addr":"172.21.15.106:6809/1441018239","cluster_addr":"172.21.15.106:6811/1441018239","heartbeat_back_addr":"172.21.15.106:6815/1441018239","heartbeat_front_addr":"172.21.15.106:6813/1441018239","state":["exists","up"]},{"osd":3,"uuid":"9593a4d1-772f-4667-ab0d-2982f841e342","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":1846968711},{"type":"v1","addr":"172.21.15.106:6817","nonce":1846968711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1846968711},{"type":"v1","addr":"172.21.15.106:6819","nonce":1846968711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":1846968711},{"type":"v1","addr":"172.21.15.106:6823","nonce":1846968711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":1846968711},{"type":"v1","addr":"172.21.15.106:6821","nonce":1846968711}]},"public_addr":"172.21.15.106:6817/1846968711","cluster_addr":"172.21.15.106:6819/1846968711","heartbeat_back_addr":"172.21.15.106:6823/1846968711","heartbeat_front_addr":"172.21.15.106:6821/1846968711","state":["exists","up"]},{"osd":4,"uuid":"1b61f16d-5c60-480a-b449-c745aa3f6b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":729224739},{"type":"v1","addr":"172.21.15.134:6825","nonce":729224739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":729224739},{"type":"v1","addr":"172.21.15.134:6827","nonce":729224739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":729224739},{"type":"v1","addr":"172.21.15.134:6831","nonce":729224739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":729224739},{"type":"v1","addr":"172.21.15.134:6829","nonce":729224739}]},"public_addr":"172.21.15.134:6825/729224739","cluster_addr":"172.21.15.134:6827/729224739","heartbeat_back_addr":"172.21.15.134:6831/729224739","heartbeat_front_addr":"172.21.15.134:6829/729224739","state":["exists","up"]},{"osd":5,"uuid":"944cdd7f-47c0-4b71-b7e0-d093542f4528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3132275652},{"type":"v1","addr":"172.21.15.134:6809","nonce":3132275652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3132275652},{"type":"v1","addr":"172.21.15.134:6811","nonce":3132275652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3132275652},{"type":"v1","addr":"172.21.15.134:6815","nonce":3132275652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3132275652},{"type":"v1","addr":"172.21.15.134:6813","nonce":3132275652}]},"public_addr":"172.21.15.134:6809/3132275652","cluster_addr":"172.21.15.134:6811/3132275652","heartbeat_back_addr":"172.21.15.134:6815/3132275652","heartbeat_front_addr":"172.21.15.134:6813/3132275652","state":["exists","up"]},{"osd":6,"uuid":"cc2446e2-1d21-4135-ac04-6af7c990058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":992702668},{"type":"v1","addr":"172.21.15.134:6801","nonce":992702668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":992702668},{"type":"v1","addr":"172.21.15.134:6803","nonce":992702668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":992702668},{"type":"v1","addr":"172.21.15.134:6807","nonce":992702668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":992702668},{"type":"v1","addr":"172.21.15.134:6805","nonce":992702668}]},"public_addr":"172.21.15.134:6801/992702668","cluster_addr":"172.21.15.134:6803/992702668","heartbeat_back_addr":"172.21.15.134:6807/992702668","heartbeat_front_addr":"172.21.15.134:6805/992702668","state":["exists","up"]},{"osd":7,"uuid":"3df1fc7a-82fc-4257-8629-98027b6e2aed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":3291062284},{"type":"v1","addr":"172.21.15.134:6817","nonce":3291062284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":3291062284},{"type":"v1","addr":"172.21.15.134:6819","nonce":3291062284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":3291062284},{"type":"v1","addr":"172.21.15.134:6823","nonce":3291062284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":3291062284},{"type":"v1","addr":"172.21.15.134:6821","nonce":3291062284}]},"public_addr":"172.21.15.134:6817/3291062284","cluster_addr":"172.21.15.134:6819/3291062284","heartbeat_back_addr":"172.21.15.134:6823/3291062284","heartbeat_front_addr":"172.21.15.134:6821/3291062284","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.403941+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.382563+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.411544+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.330444+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.405430+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.387737+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.408922+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.454450+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:30:37.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.110+0000 7efd32814640 1 -- 172.21.15.106:0/1957007515 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7efd180744b0 msgr2=0x7efd180769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:37.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.110+0000 7efd32814640 1 --2- 172.21.15.106:0/1957007515 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7efd180744b0 0x7efd180769c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7efd20033000 tx=0x7efd20008030 comp rx=0 tx=0).stop 2024-04-25T10:30:37.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.110+0000 7efd32814640 1 -- 172.21.15.106:0/1957007515 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd14005910 msgr2=0x7efd2c1539f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:37.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.110+0000 7efd32814640 1 --2- 172.21.15.106:0/1957007515 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd14005910 0x7efd2c1539f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7efd1c007340 tx=0x7efd1c007800 comp rx=0 tx=0).stop 2024-04-25T10:30:37.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.110+0000 7efd32814640 1 -- 172.21.15.106:0/1957007515 shutdown_connections 2024-04-25T10:30:37.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.110+0000 7efd32814640 1 --2- 172.21.15.106:0/1957007515 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7efd180744b0 0x7efd180769c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:37.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.110+0000 7efd32814640 1 --2- 172.21.15.106:0/1957007515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efd2c14eb20 0x7efd2c0ce320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:37.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.110+0000 7efd32814640 1 --2- 172.21.15.106:0/1957007515 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efd2c148d60 0x7efd2c14a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:37.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.110+0000 7efd32814640 1 --2- 172.21.15.106:0/1957007515 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd14005910 0x7efd2c1539f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:37.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.110+0000 7efd32814640 1 -- 172.21.15.106:0/1957007515 >> 172.21.15.106:0/1957007515 conn(0x7efd2c06c9b0 msgr2=0x7efd2c072870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:37.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.111+0000 7efd32814640 1 -- 172.21.15.106:0/1957007515 shutdown_connections 2024-04-25T10:30:37.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.111+0000 7efd32814640 1 -- 172.21.15.106:0/1957007515 wait complete. 2024-04-25T10:30:37.124 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:30:37.124 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:30:37.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.228+0000 7f4d3b89f640 1 Processor -- start 2024-04-25T10:30:37.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.228+0000 7f4d3b89f640 1 -- start start 2024-04-25T10:30:37.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.229+0000 7f4d3b89f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4d340c9c00 0x7f4d340c9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:37.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.229+0000 7f4d3b89f640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d340c96d0 con 0x7f4d34153590 2024-04-25T10:30:37.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.229+0000 7f4d3b89f640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d34074790 con 0x7f4d34153180 2024-04-25T10:30:37.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.229+0000 7f4d3b89f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f4d340741d0 con 0x7f4d340c9c00 2024-04-25T10:30:37.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.229+0000 7f4d39e3c640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f4d34153590 0x7f4d34159340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:44238/0 (socket says 172.21.15.106:44238) 2024-04-25T10:30:37.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.229+0000 7f4d39e3c640 1 -- 172.21.15.106:0/1464874142 learned_addr learned my addr 172.21.15.106:0/1464874142 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:30:37.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.230+0000 7f4d3963b640 1 --2- 172.21.15.106:0/1464874142 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4d340c9c00 0x7f4d340c9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:37.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.230+0000 7f4d2bfff640 1 -- 172.21.15.106:0/1464874142 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1592387094 0 0) 0x7f4d340c96d0 con 0x7f4d34153590 2024-04-25T10:30:37.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.230+0000 7f4d2bfff640 1 -- 172.21.15.106:0/1464874142 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4d20003440 con 0x7f4d34153590 2024-04-25T10:30:37.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.230+0000 7f4d2bfff640 1 -- 172.21.15.106:0/1464874142 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 531086811 0 0) 0x7f4d20003440 con 0x7f4d34153590 2024-04-25T10:30:37.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.230+0000 7f4d2bfff640 1 -- 172.21.15.106:0/1464874142 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4d340c9c00 msgr2=0x7f4d340c9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:37.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.230+0000 7f4d2bfff640 1 --2- 172.21.15.106:0/1464874142 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4d340c9c00 0x7f4d340c9190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:37.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.230+0000 7f4d2bfff640 1 -- 172.21.15.106:0/1464874142 >> v1:172.21.15.134:6789/0 conn(0x7f4d34153180 legacy=0x7f4d3406ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:37.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.230+0000 7f4d2bfff640 1 -- 172.21.15.106:0/1464874142 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d340744b0 con 0x7f4d34153590 2024-04-25T10:30:37.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.230+0000 7f4d2bfff640 1 -- 172.21.15.106:0/1464874142 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3776459350 0 0) 0x7f4d2c0019f0 con 0x7f4d34153590 2024-04-25T10:30:37.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.230+0000 7f4d2bfff640 1 -- 172.21.15.106:0/1464874142 >> v1:172.21.15.106:6789/0 conn(0x7f4d34153590 legacy=0x7f4d34159340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:37.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.230+0000 7f4d2bfff640 1 --2- 172.21.15.106:0/1464874142 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d20003ad0 0x7f4d20003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:37.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.230+0000 7f4d2bfff640 1 --2- 172.21.15.106:0/1464874142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d200088a0 0x7f4d2000ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:37.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.230+0000 7f4d2bfff640 1 --2- 172.21.15.106:0/1464874142 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4d340c9c00 0x7f4d2000d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:37.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.230+0000 7f4d2bfff640 1 -- 172.21.15.106:0/1464874142 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4d340741d0 con 0x7f4d20003ad0 2024-04-25T10:30:37.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.230+0000 7f4d2bfff640 1 -- 172.21.15.106:0/1464874142 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4d2000dd80 con 0x7f4d200088a0 2024-04-25T10:30:37.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.230+0000 7f4d2bfff640 1 -- 172.21.15.106:0/1464874142 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f4d2000df80 con 0x7f4d340c9c00 2024-04-25T10:30:37.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.230+0000 7f4d3963b640 1 --2- 172.21.15.106:0/1464874142 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d20003ad0 0x7f4d20003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:37.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.230+0000 7f4d38e3a640 1 --2- 172.21.15.106:0/1464874142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d200088a0 0x7f4d2000ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:37.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.230+0000 7f4d39e3c640 1 --2- 172.21.15.106:0/1464874142 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4d340c9c00 0x7f4d2000d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:37.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.230+0000 7f4d3963b640 1 -- 172.21.15.106:0/1464874142 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4d340c9c00 msgr2=0x7f4d2000d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:37.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.230+0000 7f4d3963b640 1 --2- 172.21.15.106:0/1464874142 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4d340c9c00 0x7f4d2000d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:37.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.230+0000 7f4d3963b640 1 -- 172.21.15.106:0/1464874142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d200088a0 msgr2=0x7f4d2000ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:37.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.230+0000 7f4d3963b640 1 --2- 172.21.15.106:0/1464874142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d200088a0 0x7f4d2000ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:37.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.230+0000 7f4d3963b640 1 -- 172.21.15.106:0/1464874142 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d20003440 con 0x7f4d20003ad0 2024-04-25T10:30:37.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.230+0000 7f4d39e3c640 1 --2- 172.21.15.106:0/1464874142 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4d340c9c00 0x7f4d2000d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:37.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.231+0000 7f4d3963b640 1 --2- 172.21.15.106:0/1464874142 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d20003ad0 0x7f4d20003f40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4d30009ac0 tx=0x7f4d20003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=990403c4c9194e72 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:37.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.231+0000 7f4d2bfff640 1 -- 172.21.15.106:0/1464874142 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d30036070 con 0x7f4d20003ad0 2024-04-25T10:30:37.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.231+0000 7f4d2bfff640 1 -- 172.21.15.106:0/1464874142 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d30031400 con 0x7f4d20003ad0 2024-04-25T10:30:37.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.231+0000 7f4d2bfff640 1 -- 172.21.15.106:0/1464874142 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d30009690 con 0x7f4d20003ad0 2024-04-25T10:30:37.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.232+0000 7f4d3b89f640 1 -- 172.21.15.106:0/1464874142 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d20003ad0 msgr2=0x7f4d20003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:37.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.232+0000 7f4d3b89f640 1 --2- 172.21.15.106:0/1464874142 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d20003ad0 0x7f4d20003f40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4d30009ac0 tx=0x7f4d20003730 comp rx=0 tx=0).stop 2024-04-25T10:30:37.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.232+0000 7f4d39e3c640 1 -- 172.21.15.106:0/1464874142 reap_dead start 2024-04-25T10:30:37.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.232+0000 7f4d3b89f640 1 -- 172.21.15.106:0/1464874142 shutdown_connections 2024-04-25T10:30:37.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.232+0000 7f4d3b89f640 1 -- 172.21.15.106:0/1464874142 >> 172.21.15.106:0/1464874142 conn(0x7f4d3406c9b0 msgr2=0x7f4d3406cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:37.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.232+0000 7f4d3b89f640 1 -- 172.21.15.106:0/1464874142 shutdown_connections 2024-04-25T10:30:37.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.232+0000 7f4d3b89f640 1 -- 172.21.15.106:0/1464874142 wait complete. 2024-04-25T10:30:37.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.234+0000 7f4d3b89f640 1 Processor -- start 2024-04-25T10:30:37.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.234+0000 7f4d3b89f640 1 -- start start 2024-04-25T10:30:37.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.235+0000 7f4d3b89f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d20003ad0 0x7f4d34157600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:37.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.235+0000 7f4d3b89f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d341506b0 0x7f4d34154c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:37.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.235+0000 7f4d3b89f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4d34158bb0 0x7f4d34153bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:37.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.235+0000 7f4d3b89f640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4d340c58e0 con 0x7f4d20003ad0 2024-04-25T10:30:37.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.235+0000 7f4d3b89f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4d340c5600 con 0x7f4d341506b0 2024-04-25T10:30:37.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.235+0000 7f4d3b89f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f4d340ca5e0 con 0x7f4d34158bb0 2024-04-25T10:30:37.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.235+0000 7f4d3963b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d20003ad0 0x7f4d34157600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:37.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.235+0000 7f4d3963b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d20003ad0 0x7f4d34157600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47486/0 (socket says 172.21.15.106:47486) 2024-04-25T10:30:37.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.235+0000 7f4d3963b640 1 -- 172.21.15.106:0/468610294 learned_addr learned my addr 172.21.15.106:0/468610294 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:37.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.235+0000 7f4d3963b640 1 -- 172.21.15.106:0/468610294 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4d34158bb0 msgr2=0x7f4d34153bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:37.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.235+0000 7f4d3963b640 1 --2- 172.21.15.106:0/468610294 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4d34158bb0 0x7f4d34153bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:37.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.235+0000 7f4d3963b640 1 -- 172.21.15.106:0/468610294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d341506b0 msgr2=0x7f4d34154c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:37.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.235+0000 7f4d3963b640 1 --2- 172.21.15.106:0/468610294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d341506b0 0x7f4d34154c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:37.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.235+0000 7f4d3963b640 1 -- 172.21.15.106:0/468610294 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d34157b40 con 0x7f4d20003ad0 2024-04-25T10:30:37.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.235+0000 7f4d3963b640 1 --2- 172.21.15.106:0/468610294 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d20003ad0 0x7f4d34157600 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4d30034040 tx=0x7f4d30004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:37.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.235+0000 7f4d29ffb640 1 -- 172.21.15.106:0/468610294 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d30040020 con 0x7f4d20003ad0 2024-04-25T10:30:37.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.235+0000 7f4d29ffb640 1 -- 172.21.15.106:0/468610294 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d30003b40 con 0x7f4d20003ad0 2024-04-25T10:30:37.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.235+0000 7f4d3b89f640 1 -- 172.21.15.106:0/468610294 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d34155b80 con 0x7f4d20003ad0 2024-04-25T10:30:37.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.235+0000 7f4d29ffb640 1 -- 172.21.15.106:0/468610294 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d30036040 con 0x7f4d20003ad0 2024-04-25T10:30:37.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.236+0000 7f4d3b89f640 1 -- 172.21.15.106:0/468610294 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d34156140 con 0x7f4d20003ad0 2024-04-25T10:30:37.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.237+0000 7f4d3b89f640 1 -- 172.21.15.106:0/468610294 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d34156380 con 0x7f4d20003ad0 2024-04-25T10:30:37.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.237+0000 7f4d29ffb640 1 -- 172.21.15.106:0/468610294 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4d30043020 con 0x7f4d20003ad0 2024-04-25T10:30:37.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.238+0000 7f4d29ffb640 1 --2- 172.21.15.106:0/468610294 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f4d180744e0 0x7f4d180769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:37.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.238+0000 7f4d29ffb640 1 -- 172.21.15.106:0/468610294 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4d300b5540 con 0x7f4d20003ad0 2024-04-25T10:30:37.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.238+0000 7f4d38e3a640 1 --2- 172.21.15.106:0/468610294 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f4d180744e0 0x7f4d180769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:37.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.238+0000 7f4d38e3a640 1 --2- 172.21.15.106:0/468610294 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f4d180744e0 0x7f4d180769f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4d340c76a0 tx=0x7f4d24009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:37.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.242+0000 7f4d29ffb640 1 -- 172.21.15.106:0/468610294 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4d34156380 con 0x7f4d20003ad0 2024-04-25T10:30:37.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.412+0000 7f4d3b89f640 1 -- 172.21.15.106:0/468610294 --> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4d34155910 con 0x7f4d180744e0 2024-04-25T10:30:37.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.419+0000 7f4d29ffb640 1 -- 172.21.15.106:0/468610294 <== mgr.4107 v2:172.21.15.106:6832/1744210167 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251834 (secure 0 0 0) 0x7f4d34155910 con 0x7f4d180744e0 2024-04-25T10:30:37.424 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:30:37.426 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-25T10:30:37.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.422+0000 7f4d3b89f640 1 -- 172.21.15.106:0/468610294 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f4d180744e0 msgr2=0x7f4d180769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:37.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.422+0000 7f4d3b89f640 1 --2- 172.21.15.106:0/468610294 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f4d180744e0 0x7f4d180769f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4d340c76a0 tx=0x7f4d24009040 comp rx=0 tx=0).stop 2024-04-25T10:30:37.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.422+0000 7f4d3b89f640 1 -- 172.21.15.106:0/468610294 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d20003ad0 msgr2=0x7f4d34157600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:37.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.422+0000 7f4d3b89f640 1 --2- 172.21.15.106:0/468610294 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d20003ad0 0x7f4d34157600 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4d30034040 tx=0x7f4d30004730 comp rx=0 tx=0).stop 2024-04-25T10:30:37.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.422+0000 7f4d3b89f640 1 -- 172.21.15.106:0/468610294 shutdown_connections 2024-04-25T10:30:37.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.422+0000 7f4d3b89f640 1 --2- 172.21.15.106:0/468610294 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f4d180744e0 0x7f4d180769f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:37.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.422+0000 7f4d3b89f640 1 --2- 172.21.15.106:0/468610294 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4d34158bb0 0x7f4d34153bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:37.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.422+0000 7f4d3b89f640 1 --2- 172.21.15.106:0/468610294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d341506b0 0x7f4d34154c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:37.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.422+0000 7f4d3b89f640 1 --2- 172.21.15.106:0/468610294 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d20003ad0 0x7f4d34157600 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:37.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.422+0000 7f4d3b89f640 1 -- 172.21.15.106:0/468610294 >> 172.21.15.106:0/468610294 conn(0x7f4d3406c9b0 msgr2=0x7f4d340c9190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:37.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.422+0000 7f4d3b89f640 1 -- 172.21.15.106:0/468610294 shutdown_connections 2024-04-25T10:30:37.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.422+0000 7f4d3b89f640 1 -- 172.21.15.106:0/468610294 wait complete. 2024-04-25T10:30:37.438 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-04-25T10:30:36.096588+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":106,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218900,"kb_used_data":2712,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755820,"statfs":{"total":773094113280,"available":772869959680,"internally_reserved":0,"allocated":2777088,"data_stored":1723008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.703718"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795275+0000","last_change":"2024-04-25T10:30:32.440983+0000","last_active":"2024-04-25T10:30:34.795275+0000","last_peered":"2024-04-25T10:30:34.795275+0000","last_clean":"2024-04-25T10:30:34.795275+0000","last_became_active":"2024-04-25T10:30:32.440552+0000","last_became_peered":"2024-04-25T10:30:32.440552+0000","last_unstale":"2024-04-25T10:30:34.795275+0000","last_undegraded":"2024-04-25T10:30:34.795275+0000","last_fullsized":"2024-04-25T10:30:34.795275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:47.963061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.412686+0000","last_change":"2024-04-25T10:30:32.425695+0000","last_active":"2024-04-25T10:30:33.412686+0000","last_peered":"2024-04-25T10:30:33.412686+0000","last_clean":"2024-04-25T10:30:33.412686+0000","last_became_active":"2024-04-25T10:30:32.425368+0000","last_became_peered":"2024-04-25T10:30:32.425368+0000","last_unstale":"2024-04-25T10:30:33.412686+0000","last_undegraded":"2024-04-25T10:30:33.412686+0000","last_fullsized":"2024-04-25T10:30:33.412686+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:39.120522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.196620+0000","last_change":"2024-04-25T10:30:32.430144+0000","last_active":"2024-04-25T10:30:34.196620+0000","last_peered":"2024-04-25T10:30:34.196620+0000","last_clean":"2024-04-25T10:30:34.196620+0000","last_became_active":"2024-04-25T10:30:32.429754+0000","last_became_peered":"2024-04-25T10:30:32.429754+0000","last_unstale":"2024-04-25T10:30:34.196620+0000","last_undegraded":"2024-04-25T10:30:34.196620+0000","last_fullsized":"2024-04-25T10:30:34.196620+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:11.448907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794948+0000","last_change":"2024-04-25T10:30:32.442478+0000","last_active":"2024-04-25T10:30:34.794948+0000","last_peered":"2024-04-25T10:30:34.794948+0000","last_clean":"2024-04-25T10:30:34.794948+0000","last_became_active":"2024-04-25T10:30:32.442182+0000","last_became_peered":"2024-04-25T10:30:32.442182+0000","last_unstale":"2024-04-25T10:30:34.794948+0000","last_undegraded":"2024-04-25T10:30:34.794948+0000","last_fullsized":"2024-04-25T10:30:34.794948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:22.185978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794683+0000","last_change":"2024-04-25T10:30:32.438231+0000","last_active":"2024-04-25T10:30:34.794683+0000","last_peered":"2024-04-25T10:30:34.794683+0000","last_clean":"2024-04-25T10:30:34.794683+0000","last_became_active":"2024-04-25T10:30:32.437656+0000","last_became_peered":"2024-04-25T10:30:32.437656+0000","last_unstale":"2024-04-25T10:30:34.794683+0000","last_undegraded":"2024-04-25T10:30:34.794683+0000","last_fullsized":"2024-04-25T10:30:34.794683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:56.915999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795645+0000","last_change":"2024-04-25T10:30:32.439221+0000","last_active":"2024-04-25T10:30:34.795645+0000","last_peered":"2024-04-25T10:30:34.795645+0000","last_clean":"2024-04-25T10:30:34.795645+0000","last_became_active":"2024-04-25T10:30:32.438923+0000","last_became_peered":"2024-04-25T10:30:32.438923+0000","last_unstale":"2024-04-25T10:30:34.795645+0000","last_undegraded":"2024-04-25T10:30:34.795645+0000","last_fullsized":"2024-04-25T10:30:34.795645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:40.469409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794157+0000","last_change":"2024-04-25T10:30:32.429453+0000","last_active":"2024-04-25T10:30:34.794157+0000","last_peered":"2024-04-25T10:30:34.794157+0000","last_clean":"2024-04-25T10:30:34.794157+0000","last_became_active":"2024-04-25T10:30:32.429131+0000","last_became_peered":"2024-04-25T10:30:32.429131+0000","last_unstale":"2024-04-25T10:30:34.794157+0000","last_undegraded":"2024-04-25T10:30:34.794157+0000","last_fullsized":"2024-04-25T10:30:34.794157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:26.785922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415260+0000","last_change":"2024-04-25T10:30:32.438860+0000","last_active":"2024-04-25T10:30:33.415260+0000","last_peered":"2024-04-25T10:30:33.415260+0000","last_clean":"2024-04-25T10:30:33.415260+0000","last_became_active":"2024-04-25T10:30:32.438491+0000","last_became_peered":"2024-04-25T10:30:32.438491+0000","last_unstale":"2024-04-25T10:30:33.415260+0000","last_undegraded":"2024-04-25T10:30:33.415260+0000","last_fullsized":"2024-04-25T10:30:33.415260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:08.610707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795467+0000","last_change":"2024-04-25T10:30:32.440331+0000","last_active":"2024-04-25T10:30:34.795467+0000","last_peered":"2024-04-25T10:30:34.795467+0000","last_clean":"2024-04-25T10:30:34.795467+0000","last_became_active":"2024-04-25T10:30:32.440043+0000","last_became_peered":"2024-04-25T10:30:32.440043+0000","last_unstale":"2024-04-25T10:30:34.795467+0000","last_undegraded":"2024-04-25T10:30:34.795467+0000","last_fullsized":"2024-04-25T10:30:34.795467+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:17.684685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794636+0000","last_change":"2024-04-25T10:30:32.435388+0000","last_active":"2024-04-25T10:30:34.794636+0000","last_peered":"2024-04-25T10:30:34.794636+0000","last_clean":"2024-04-25T10:30:34.794636+0000","last_became_active":"2024-04-25T10:30:32.434983+0000","last_became_peered":"2024-04-25T10:30:32.434983+0000","last_unstale":"2024-04-25T10:30:34.794636+0000","last_undegraded":"2024-04-25T10:30:34.794636+0000","last_fullsized":"2024-04-25T10:30:34.794636+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:39.256604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.793928+0000","last_change":"2024-04-25T10:30:32.435884+0000","last_active":"2024-04-25T10:30:34.793928+0000","last_peered":"2024-04-25T10:30:34.793928+0000","last_clean":"2024-04-25T10:30:34.793928+0000","last_became_active":"2024-04-25T10:30:32.434645+0000","last_became_peered":"2024-04-25T10:30:32.434645+0000","last_unstale":"2024-04-25T10:30:34.793928+0000","last_undegraded":"2024-04-25T10:30:34.793928+0000","last_fullsized":"2024-04-25T10:30:34.793928+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:07.778437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795720+0000","last_change":"2024-04-25T10:30:32.436403+0000","last_active":"2024-04-25T10:30:34.795720+0000","last_peered":"2024-04-25T10:30:34.795720+0000","last_clean":"2024-04-25T10:30:34.795720+0000","last_became_active":"2024-04-25T10:30:32.435908+0000","last_became_peered":"2024-04-25T10:30:32.435908+0000","last_unstale":"2024-04-25T10:30:34.795720+0000","last_undegraded":"2024-04-25T10:30:34.795720+0000","last_fullsized":"2024-04-25T10:30:34.795720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:03.339316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415872+0000","last_change":"2024-04-25T10:30:32.432110+0000","last_active":"2024-04-25T10:30:33.415872+0000","last_peered":"2024-04-25T10:30:33.415872+0000","last_clean":"2024-04-25T10:30:33.415872+0000","last_became_active":"2024-04-25T10:30:32.431792+0000","last_became_peered":"2024-04-25T10:30:32.431792+0000","last_unstale":"2024-04-25T10:30:33.415872+0000","last_undegraded":"2024-04-25T10:30:33.415872+0000","last_fullsized":"2024-04-25T10:30:33.415872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:34.185989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.196476+0000","last_change":"2024-04-25T10:30:32.433676+0000","last_active":"2024-04-25T10:30:34.196476+0000","last_peered":"2024-04-25T10:30:34.196476+0000","last_clean":"2024-04-25T10:30:34.196476+0000","last_became_active":"2024-04-25T10:30:32.433329+0000","last_became_peered":"2024-04-25T10:30:32.433329+0000","last_unstale":"2024-04-25T10:30:34.196476+0000","last_undegraded":"2024-04-25T10:30:34.196476+0000","last_fullsized":"2024-04-25T10:30:34.196476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:59.942368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.412754+0000","last_change":"2024-04-25T10:30:32.423582+0000","last_active":"2024-04-25T10:30:33.412754+0000","last_peered":"2024-04-25T10:30:33.412754+0000","last_clean":"2024-04-25T10:30:33.412754+0000","last_became_active":"2024-04-25T10:30:32.422991+0000","last_became_peered":"2024-04-25T10:30:32.422991+0000","last_unstale":"2024-04-25T10:30:33.412754+0000","last_undegraded":"2024-04-25T10:30:33.412754+0000","last_fullsized":"2024-04-25T10:30:33.412754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:11:47.132928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794545+0000","last_change":"2024-04-25T10:30:32.440235+0000","last_active":"2024-04-25T10:30:34.794545+0000","last_peered":"2024-04-25T10:30:34.794545+0000","last_clean":"2024-04-25T10:30:34.794545+0000","last_became_active":"2024-04-25T10:30:32.439191+0000","last_became_peered":"2024-04-25T10:30:32.439191+0000","last_unstale":"2024-04-25T10:30:34.794545+0000","last_undegraded":"2024-04-25T10:30:34.794545+0000","last_fullsized":"2024-04-25T10:30:34.794545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:51.423086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795170+0000","last_change":"2024-04-25T10:30:32.442418+0000","last_active":"2024-04-25T10:30:34.795170+0000","last_peered":"2024-04-25T10:30:34.795170+0000","last_clean":"2024-04-25T10:30:34.795170+0000","last_became_active":"2024-04-25T10:30:32.442125+0000","last_became_peered":"2024-04-25T10:30:32.442125+0000","last_unstale":"2024-04-25T10:30:34.795170+0000","last_undegraded":"2024-04-25T10:30:34.795170+0000","last_fullsized":"2024-04-25T10:30:34.795170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:14.213809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415344+0000","last_change":"2024-04-25T10:30:32.434758+0000","last_active":"2024-04-25T10:30:33.415344+0000","last_peered":"2024-04-25T10:30:33.415344+0000","last_clean":"2024-04-25T10:30:33.415344+0000","last_became_active":"2024-04-25T10:30:32.434492+0000","last_became_peered":"2024-04-25T10:30:32.434492+0000","last_unstale":"2024-04-25T10:30:33.415344+0000","last_undegraded":"2024-04-25T10:30:33.415344+0000","last_fullsized":"2024-04-25T10:30:33.415344+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:41.454407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.796066+0000","last_change":"2024-04-25T10:30:32.440399+0000","last_active":"2024-04-25T10:30:34.796066+0000","last_peered":"2024-04-25T10:30:34.796066+0000","last_clean":"2024-04-25T10:30:34.796066+0000","last_became_active":"2024-04-25T10:30:32.438227+0000","last_became_peered":"2024-04-25T10:30:32.438227+0000","last_unstale":"2024-04-25T10:30:34.796066+0000","last_undegraded":"2024-04-25T10:30:34.796066+0000","last_fullsized":"2024-04-25T10:30:34.796066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:49.571671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794149+0000","last_change":"2024-04-25T10:30:32.431609+0000","last_active":"2024-04-25T10:30:34.794149+0000","last_peered":"2024-04-25T10:30:34.794149+0000","last_clean":"2024-04-25T10:30:34.794149+0000","last_became_active":"2024-04-25T10:30:32.431314+0000","last_became_peered":"2024-04-25T10:30:32.431314+0000","last_unstale":"2024-04-25T10:30:34.794149+0000","last_undegraded":"2024-04-25T10:30:34.794149+0000","last_fullsized":"2024-04-25T10:30:34.794149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:33.421989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.196668+0000","last_change":"2024-04-25T10:30:32.432612+0000","last_active":"2024-04-25T10:30:34.196668+0000","last_peered":"2024-04-25T10:30:34.196668+0000","last_clean":"2024-04-25T10:30:34.196668+0000","last_became_active":"2024-04-25T10:30:32.432271+0000","last_became_peered":"2024-04-25T10:30:32.432271+0000","last_unstale":"2024-04-25T10:30:34.196668+0000","last_undegraded":"2024-04-25T10:30:34.196668+0000","last_fullsized":"2024-04-25T10:30:34.196668+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:14.281493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795827+0000","last_change":"2024-04-25T10:30:32.444618+0000","last_active":"2024-04-25T10:30:34.795827+0000","last_peered":"2024-04-25T10:30:34.795827+0000","last_clean":"2024-04-25T10:30:34.795827+0000","last_became_active":"2024-04-25T10:30:32.444273+0000","last_became_peered":"2024-04-25T10:30:32.444273+0000","last_unstale":"2024-04-25T10:30:34.795827+0000","last_undegraded":"2024-04-25T10:30:34.795827+0000","last_fullsized":"2024-04-25T10:30:34.795827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:09.295251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415348+0000","last_change":"2024-04-25T10:30:32.426578+0000","last_active":"2024-04-25T10:30:33.415348+0000","last_peered":"2024-04-25T10:30:33.415348+0000","last_clean":"2024-04-25T10:30:33.415348+0000","last_became_active":"2024-04-25T10:30:32.426223+0000","last_became_peered":"2024-04-25T10:30:32.426223+0000","last_unstale":"2024-04-25T10:30:33.415348+0000","last_undegraded":"2024-04-25T10:30:33.415348+0000","last_fullsized":"2024-04-25T10:30:33.415348+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:53.700585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794721+0000","last_change":"2024-04-25T10:30:32.439767+0000","last_active":"2024-04-25T10:30:34.794721+0000","last_peered":"2024-04-25T10:30:34.794721+0000","last_clean":"2024-04-25T10:30:34.794721+0000","last_became_active":"2024-04-25T10:30:32.438801+0000","last_became_peered":"2024-04-25T10:30:32.438801+0000","last_unstale":"2024-04-25T10:30:34.794721+0000","last_undegraded":"2024-04-25T10:30:34.794721+0000","last_fullsized":"2024-04-25T10:30:34.794721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:58.231512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415544+0000","last_change":"2024-04-25T10:30:32.427424+0000","last_active":"2024-04-25T10:30:33.415544+0000","last_peered":"2024-04-25T10:30:33.415544+0000","last_clean":"2024-04-25T10:30:33.415544+0000","last_became_active":"2024-04-25T10:30:32.427081+0000","last_became_peered":"2024-04-25T10:30:32.427081+0000","last_unstale":"2024-04-25T10:30:33.415544+0000","last_undegraded":"2024-04-25T10:30:33.415544+0000","last_fullsized":"2024-04-25T10:30:33.415544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:47.372451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794727+0000","last_change":"2024-04-25T10:30:32.432472+0000","last_active":"2024-04-25T10:30:34.794727+0000","last_peered":"2024-04-25T10:30:34.794727+0000","last_clean":"2024-04-25T10:30:34.794727+0000","last_became_active":"2024-04-25T10:30:32.432121+0000","last_became_peered":"2024-04-25T10:30:32.432121+0000","last_unstale":"2024-04-25T10:30:34.794727+0000","last_undegraded":"2024-04-25T10:30:34.794727+0000","last_fullsized":"2024-04-25T10:30:34.794727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:34.191339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.412571+0000","last_change":"2024-04-25T10:30:32.425108+0000","last_active":"2024-04-25T10:30:33.412571+0000","last_peered":"2024-04-25T10:30:33.412571+0000","last_clean":"2024-04-25T10:30:33.412571+0000","last_became_active":"2024-04-25T10:30:32.424794+0000","last_became_peered":"2024-04-25T10:30:32.424794+0000","last_unstale":"2024-04-25T10:30:33.412571+0000","last_undegraded":"2024-04-25T10:30:33.412571+0000","last_fullsized":"2024-04-25T10:30:33.412571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:01.282758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415734+0000","last_change":"2024-04-25T10:30:32.426672+0000","last_active":"2024-04-25T10:30:33.415734+0000","last_peered":"2024-04-25T10:30:33.415734+0000","last_clean":"2024-04-25T10:30:33.415734+0000","last_became_active":"2024-04-25T10:30:32.426348+0000","last_became_peered":"2024-04-25T10:30:32.426348+0000","last_unstale":"2024-04-25T10:30:33.415734+0000","last_undegraded":"2024-04-25T10:30:33.415734+0000","last_fullsized":"2024-04-25T10:30:33.415734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:57.645772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.196609+0000","last_change":"2024-04-25T10:30:32.427989+0000","last_active":"2024-04-25T10:30:34.196609+0000","last_peered":"2024-04-25T10:30:34.196609+0000","last_clean":"2024-04-25T10:30:34.196609+0000","last_became_active":"2024-04-25T10:30:32.427676+0000","last_became_peered":"2024-04-25T10:30:32.427676+0000","last_unstale":"2024-04-25T10:30:34.196609+0000","last_undegraded":"2024-04-25T10:30:34.196609+0000","last_fullsized":"2024-04-25T10:30:34.196609+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:15:02.332186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795935+0000","last_change":"2024-04-25T10:30:32.441112+0000","last_active":"2024-04-25T10:30:34.795935+0000","last_peered":"2024-04-25T10:30:34.795935+0000","last_clean":"2024-04-25T10:30:34.795935+0000","last_became_active":"2024-04-25T10:30:32.440815+0000","last_became_peered":"2024-04-25T10:30:32.440815+0000","last_unstale":"2024-04-25T10:30:34.795935+0000","last_undegraded":"2024-04-25T10:30:34.795935+0000","last_fullsized":"2024-04-25T10:30:34.795935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:02.956759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415178+0000","last_change":"2024-04-25T10:30:32.430902+0000","last_active":"2024-04-25T10:30:33.415178+0000","last_peered":"2024-04-25T10:30:33.415178+0000","last_clean":"2024-04-25T10:30:33.415178+0000","last_became_active":"2024-04-25T10:30:32.430615+0000","last_became_peered":"2024-04-25T10:30:32.430615+0000","last_unstale":"2024-04-25T10:30:33.415178+0000","last_undegraded":"2024-04-25T10:30:33.415178+0000","last_fullsized":"2024-04-25T10:30:33.415178+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:12.747595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795247+0000","last_change":"2024-04-25T10:30:32.435379+0000","last_active":"2024-04-25T10:30:34.795247+0000","last_peered":"2024-04-25T10:30:34.795247+0000","last_clean":"2024-04-25T10:30:34.795247+0000","last_became_active":"2024-04-25T10:30:32.435055+0000","last_became_peered":"2024-04-25T10:30:32.435055+0000","last_unstale":"2024-04-25T10:30:34.795247+0000","last_undegraded":"2024-04-25T10:30:34.795247+0000","last_fullsized":"2024-04-25T10:30:34.795247+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:13.623762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794490+0000","last_change":"2024-04-25T10:30:32.422102+0000","last_active":"2024-04-25T10:30:34.794490+0000","last_peered":"2024-04-25T10:30:34.794490+0000","last_clean":"2024-04-25T10:30:34.794490+0000","last_became_active":"2024-04-25T10:30:32.421766+0000","last_became_peered":"2024-04-25T10:30:32.421766+0000","last_unstale":"2024-04-25T10:30:34.794490+0000","last_undegraded":"2024-04-25T10:30:34.794490+0000","last_fullsized":"2024-04-25T10:30:34.794490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:30.300572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415973+0000","last_change":"2024-04-25T10:30:31.418065+0000","last_active":"2024-04-25T10:30:33.415973+0000","last_peered":"2024-04-25T10:30:33.415973+0000","last_clean":"2024-04-25T10:30:33.415973+0000","last_became_active":"2024-04-25T10:30:31.417451+0000","last_became_peered":"2024-04-25T10:30:31.417451+0000","last_unstale":"2024-04-25T10:30:33.415973+0000","last_undegraded":"2024-04-25T10:30:33.415973+0000","last_fullsized":"2024-04-25T10:30:33.415973+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:17.262861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.196777+0000","last_change":"2024-04-25T10:30:32.426913+0000","last_active":"2024-04-25T10:30:34.196777+0000","last_peered":"2024-04-25T10:30:34.196777+0000","last_clean":"2024-04-25T10:30:34.196777+0000","last_became_active":"2024-04-25T10:30:32.426616+0000","last_became_peered":"2024-04-25T10:30:32.426616+0000","last_unstale":"2024-04-25T10:30:34.196777+0000","last_undegraded":"2024-04-25T10:30:34.196777+0000","last_fullsized":"2024-04-25T10:30:34.196777+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:50.614932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795542+0000","last_change":"2024-04-25T10:30:31.416432+0000","last_active":"2024-04-25T10:30:34.795542+0000","last_peered":"2024-04-25T10:30:34.795542+0000","last_clean":"2024-04-25T10:30:34.795542+0000","last_became_active":"2024-04-25T10:30:31.415112+0000","last_became_peered":"2024-04-25T10:30:31.415112+0000","last_unstale":"2024-04-25T10:30:34.795542+0000","last_undegraded":"2024-04-25T10:30:34.795542+0000","last_fullsized":"2024-04-25T10:30:34.795542+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:22.674365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415646+0000","last_change":"2024-04-25T10:30:31.418059+0000","last_active":"2024-04-25T10:30:33.415646+0000","last_peered":"2024-04-25T10:30:33.415646+0000","last_clean":"2024-04-25T10:30:33.415646+0000","last_became_active":"2024-04-25T10:30:31.417716+0000","last_became_peered":"2024-04-25T10:30:31.417716+0000","last_unstale":"2024-04-25T10:30:33.415646+0000","last_undegraded":"2024-04-25T10:30:33.415646+0000","last_fullsized":"2024-04-25T10:30:33.415646+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:57.752218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415428+0000","last_change":"2024-04-25T10:30:32.428078+0000","last_active":"2024-04-25T10:30:33.415428+0000","last_peered":"2024-04-25T10:30:33.415428+0000","last_clean":"2024-04-25T10:30:33.415428+0000","last_became_active":"2024-04-25T10:30:32.427700+0000","last_became_peered":"2024-04-25T10:30:32.427700+0000","last_unstale":"2024-04-25T10:30:33.415428+0000","last_undegraded":"2024-04-25T10:30:33.415428+0000","last_fullsized":"2024-04-25T10:30:33.415428+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:26.931791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795722+0000","last_change":"2024-04-25T10:30:31.415030+0000","last_active":"2024-04-25T10:30:34.795722+0000","last_peered":"2024-04-25T10:30:34.795722+0000","last_clean":"2024-04-25T10:30:34.795722+0000","last_became_active":"2024-04-25T10:30:31.414542+0000","last_became_peered":"2024-04-25T10:30:31.414542+0000","last_unstale":"2024-04-25T10:30:34.795722+0000","last_undegraded":"2024-04-25T10:30:34.795722+0000","last_fullsized":"2024-04-25T10:30:34.795722+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:20.082420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.412471+0000","last_change":"2024-04-25T10:30:32.425207+0000","last_active":"2024-04-25T10:30:33.412471+0000","last_peered":"2024-04-25T10:30:33.412471+0000","last_clean":"2024-04-25T10:30:33.412471+0000","last_became_active":"2024-04-25T10:30:32.424849+0000","last_became_peered":"2024-04-25T10:30:32.424849+0000","last_unstale":"2024-04-25T10:30:33.412471+0000","last_undegraded":"2024-04-25T10:30:33.412471+0000","last_fullsized":"2024-04-25T10:30:33.412471+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:11.533686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794249+0000","last_change":"2024-04-25T10:30:31.407700+0000","last_active":"2024-04-25T10:30:34.794249+0000","last_peered":"2024-04-25T10:30:34.794249+0000","last_clean":"2024-04-25T10:30:34.794249+0000","last_became_active":"2024-04-25T10:30:31.407376+0000","last_became_peered":"2024-04-25T10:30:31.407376+0000","last_unstale":"2024-04-25T10:30:34.794249+0000","last_undegraded":"2024-04-25T10:30:34.794249+0000","last_fullsized":"2024-04-25T10:30:34.794249+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:50.112742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794157+0000","last_change":"2024-04-25T10:30:32.444599+0000","last_active":"2024-04-25T10:30:34.794157+0000","last_peered":"2024-04-25T10:30:34.794157+0000","last_clean":"2024-04-25T10:30:34.794157+0000","last_became_active":"2024-04-25T10:30:32.444250+0000","last_became_peered":"2024-04-25T10:30:32.444250+0000","last_unstale":"2024-04-25T10:30:34.794157+0000","last_undegraded":"2024-04-25T10:30:34.794157+0000","last_fullsized":"2024-04-25T10:30:34.794157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:55.711525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794280+0000","last_change":"2024-04-25T10:30:31.405051+0000","last_active":"2024-04-25T10:30:34.794280+0000","last_peered":"2024-04-25T10:30:34.794280+0000","last_clean":"2024-04-25T10:30:34.794280+0000","last_became_active":"2024-04-25T10:30:31.404669+0000","last_became_peered":"2024-04-25T10:30:31.404669+0000","last_unstale":"2024-04-25T10:30:34.794280+0000","last_undegraded":"2024-04-25T10:30:34.794280+0000","last_fullsized":"2024-04-25T10:30:34.794280+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:45.260847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.196763+0000","last_change":"2024-04-25T10:30:32.435804+0000","last_active":"2024-04-25T10:30:34.196763+0000","last_peered":"2024-04-25T10:30:34.196763+0000","last_clean":"2024-04-25T10:30:34.196763+0000","last_became_active":"2024-04-25T10:30:32.435471+0000","last_became_peered":"2024-04-25T10:30:32.435471+0000","last_unstale":"2024-04-25T10:30:34.196763+0000","last_undegraded":"2024-04-25T10:30:34.196763+0000","last_fullsized":"2024-04-25T10:30:34.196763+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:48.213846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.412616+0000","last_change":"2024-04-25T10:30:31.403560+0000","last_active":"2024-04-25T10:30:33.412616+0000","last_peered":"2024-04-25T10:30:33.412616+0000","last_clean":"2024-04-25T10:30:33.412616+0000","last_became_active":"2024-04-25T10:30:31.403167+0000","last_became_peered":"2024-04-25T10:30:31.403167+0000","last_unstale":"2024-04-25T10:30:33.412616+0000","last_undegraded":"2024-04-25T10:30:33.412616+0000","last_fullsized":"2024-04-25T10:30:33.412616+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:54.592832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795908+0000","last_change":"2024-04-25T10:30:32.424101+0000","last_active":"2024-04-25T10:30:34.795908+0000","last_peered":"2024-04-25T10:30:34.795908+0000","last_clean":"2024-04-25T10:30:34.795908+0000","last_became_active":"2024-04-25T10:30:32.423780+0000","last_became_peered":"2024-04-25T10:30:32.423780+0000","last_unstale":"2024-04-25T10:30:34.795908+0000","last_undegraded":"2024-04-25T10:30:34.795908+0000","last_fullsized":"2024-04-25T10:30:34.795908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:29.175348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794280+0000","last_change":"2024-04-25T10:30:31.418201+0000","last_active":"2024-04-25T10:30:34.794280+0000","last_peered":"2024-04-25T10:30:34.794280+0000","last_clean":"2024-04-25T10:30:34.794280+0000","last_became_active":"2024-04-25T10:30:31.417824+0000","last_became_peered":"2024-04-25T10:30:31.417824+0000","last_unstale":"2024-04-25T10:30:34.794280+0000","last_undegraded":"2024-04-25T10:30:34.794280+0000","last_fullsized":"2024-04-25T10:30:34.794280+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:38.584225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795830+0000","last_change":"2024-04-25T10:30:32.431139+0000","last_active":"2024-04-25T10:30:34.795830+0000","last_peered":"2024-04-25T10:30:34.795830+0000","last_clean":"2024-04-25T10:30:34.795830+0000","last_became_active":"2024-04-25T10:30:32.429622+0000","last_became_peered":"2024-04-25T10:30:32.429622+0000","last_unstale":"2024-04-25T10:30:34.795830+0000","last_undegraded":"2024-04-25T10:30:34.795830+0000","last_fullsized":"2024-04-25T10:30:34.795830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:39.993647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.196800+0000","last_change":"2024-04-25T10:30:31.414775+0000","last_active":"2024-04-25T10:30:34.196800+0000","last_peered":"2024-04-25T10:30:34.196800+0000","last_clean":"2024-04-25T10:30:34.196800+0000","last_became_active":"2024-04-25T10:30:31.414422+0000","last_became_peered":"2024-04-25T10:30:31.414422+0000","last_unstale":"2024-04-25T10:30:34.196800+0000","last_undegraded":"2024-04-25T10:30:34.196800+0000","last_fullsized":"2024-04-25T10:30:34.196800+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:37.803505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.196648+0000","last_change":"2024-04-25T10:30:32.432884+0000","last_active":"2024-04-25T10:30:34.196648+0000","last_peered":"2024-04-25T10:30:34.196648+0000","last_clean":"2024-04-25T10:30:34.196648+0000","last_became_active":"2024-04-25T10:30:32.431743+0000","last_became_peered":"2024-04-25T10:30:32.431743+0000","last_unstale":"2024-04-25T10:30:34.196648+0000","last_undegraded":"2024-04-25T10:30:34.196648+0000","last_fullsized":"2024-04-25T10:30:34.196648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:15:06.737631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795999+0000","last_change":"2024-04-25T10:30:31.419497+0000","last_active":"2024-04-25T10:30:34.795999+0000","last_peered":"2024-04-25T10:30:34.795999+0000","last_clean":"2024-04-25T10:30:34.795999+0000","last_became_active":"2024-04-25T10:30:31.419135+0000","last_became_peered":"2024-04-25T10:30:31.419135+0000","last_unstale":"2024-04-25T10:30:34.795999+0000","last_undegraded":"2024-04-25T10:30:34.795999+0000","last_fullsized":"2024-04-25T10:30:34.795999+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:59.403927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794867+0000","last_change":"2024-04-25T10:30:32.434197+0000","last_active":"2024-04-25T10:30:34.794867+0000","last_peered":"2024-04-25T10:30:34.794867+0000","last_clean":"2024-04-25T10:30:34.794867+0000","last_became_active":"2024-04-25T10:30:32.433623+0000","last_became_peered":"2024-04-25T10:30:32.433623+0000","last_unstale":"2024-04-25T10:30:34.794867+0000","last_undegraded":"2024-04-25T10:30:34.794867+0000","last_fullsized":"2024-04-25T10:30:34.794867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:08.027472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.793813+0000","last_change":"2024-04-25T10:30:31.416901+0000","last_active":"2024-04-25T10:30:34.793813+0000","last_peered":"2024-04-25T10:30:34.793813+0000","last_clean":"2024-04-25T10:30:34.793813+0000","last_became_active":"2024-04-25T10:30:31.415706+0000","last_became_peered":"2024-04-25T10:30:31.415706+0000","last_unstale":"2024-04-25T10:30:34.793813+0000","last_undegraded":"2024-04-25T10:30:34.793813+0000","last_fullsized":"2024-04-25T10:30:34.793813+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:23.243039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795344+0000","last_change":"2024-04-25T10:30:32.435089+0000","last_active":"2024-04-25T10:30:34.795344+0000","last_peered":"2024-04-25T10:30:34.795344+0000","last_clean":"2024-04-25T10:30:34.795344+0000","last_became_active":"2024-04-25T10:30:32.434737+0000","last_became_peered":"2024-04-25T10:30:32.434737+0000","last_unstale":"2024-04-25T10:30:34.795344+0000","last_undegraded":"2024-04-25T10:30:34.795344+0000","last_fullsized":"2024-04-25T10:30:34.795344+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:19.738850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415107+0000","last_change":"2024-04-25T10:30:31.405163+0000","last_active":"2024-04-25T10:30:33.415107+0000","last_peered":"2024-04-25T10:30:33.415107+0000","last_clean":"2024-04-25T10:30:33.415107+0000","last_became_active":"2024-04-25T10:30:31.404775+0000","last_became_peered":"2024-04-25T10:30:31.404775+0000","last_unstale":"2024-04-25T10:30:33.415107+0000","last_undegraded":"2024-04-25T10:30:33.415107+0000","last_fullsized":"2024-04-25T10:30:33.415107+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:10:21.595653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794446+0000","last_change":"2024-04-25T10:30:32.432061+0000","last_active":"2024-04-25T10:30:34.794446+0000","last_peered":"2024-04-25T10:30:34.794446+0000","last_clean":"2024-04-25T10:30:34.794446+0000","last_became_active":"2024-04-25T10:30:32.431651+0000","last_became_peered":"2024-04-25T10:30:32.431651+0000","last_unstale":"2024-04-25T10:30:34.794446+0000","last_undegraded":"2024-04-25T10:30:34.794446+0000","last_fullsized":"2024-04-25T10:30:34.794446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:39.062490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795328+0000","last_change":"2024-04-25T10:30:31.420258+0000","last_active":"2024-04-25T10:30:34.795328+0000","last_peered":"2024-04-25T10:30:34.795328+0000","last_clean":"2024-04-25T10:30:34.795328+0000","last_became_active":"2024-04-25T10:30:31.419902+0000","last_became_peered":"2024-04-25T10:30:31.419902+0000","last_unstale":"2024-04-25T10:30:34.795328+0000","last_undegraded":"2024-04-25T10:30:34.795328+0000","last_fullsized":"2024-04-25T10:30:34.795328+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:43.919867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795443+0000","last_change":"2024-04-25T10:30:32.433568+0000","last_active":"2024-04-25T10:30:34.795443+0000","last_peered":"2024-04-25T10:30:34.795443+0000","last_clean":"2024-04-25T10:30:34.795443+0000","last_became_active":"2024-04-25T10:30:32.432572+0000","last_became_peered":"2024-04-25T10:30:32.432572+0000","last_unstale":"2024-04-25T10:30:34.795443+0000","last_undegraded":"2024-04-25T10:30:34.795443+0000","last_fullsized":"2024-04-25T10:30:34.795443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:31.771919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415439+0000","last_change":"2024-04-25T10:30:31.419308+0000","last_active":"2024-04-25T10:30:33.415439+0000","last_peered":"2024-04-25T10:30:33.415439+0000","last_clean":"2024-04-25T10:30:33.415439+0000","last_became_active":"2024-04-25T10:30:31.418978+0000","last_became_peered":"2024-04-25T10:30:31.418978+0000","last_unstale":"2024-04-25T10:30:33.415439+0000","last_undegraded":"2024-04-25T10:30:33.415439+0000","last_fullsized":"2024-04-25T10:30:33.415439+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:33.784804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795670+0000","last_change":"2024-04-25T10:30:32.436109+0000","last_active":"2024-04-25T10:30:34.795670+0000","last_peered":"2024-04-25T10:30:34.795670+0000","last_clean":"2024-04-25T10:30:34.795670+0000","last_became_active":"2024-04-25T10:30:32.434701+0000","last_became_peered":"2024-04-25T10:30:32.434701+0000","last_unstale":"2024-04-25T10:30:34.795670+0000","last_undegraded":"2024-04-25T10:30:34.795670+0000","last_fullsized":"2024-04-25T10:30:34.795670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:19.471867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.412672+0000","last_change":"2024-04-25T10:30:31.416044+0000","last_active":"2024-04-25T10:30:33.412672+0000","last_peered":"2024-04-25T10:30:33.412672+0000","last_clean":"2024-04-25T10:30:33.412672+0000","last_became_active":"2024-04-25T10:30:31.415707+0000","last_became_peered":"2024-04-25T10:30:31.415707+0000","last_unstale":"2024-04-25T10:30:33.412672+0000","last_undegraded":"2024-04-25T10:30:33.412672+0000","last_fullsized":"2024-04-25T10:30:33.412672+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:31.424135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795321+0000","last_change":"2024-04-25T10:30:32.442971+0000","last_active":"2024-04-25T10:30:34.795321+0000","last_peered":"2024-04-25T10:30:34.795321+0000","last_clean":"2024-04-25T10:30:34.795321+0000","last_became_active":"2024-04-25T10:30:32.442656+0000","last_became_peered":"2024-04-25T10:30:32.442656+0000","last_unstale":"2024-04-25T10:30:34.795321+0000","last_undegraded":"2024-04-25T10:30:34.795321+0000","last_fullsized":"2024-04-25T10:30:34.795321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:29.951142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415622+0000","last_change":"2024-04-25T10:30:31.404793+0000","last_active":"2024-04-25T10:30:33.415622+0000","last_peered":"2024-04-25T10:30:33.415622+0000","last_clean":"2024-04-25T10:30:33.415622+0000","last_became_active":"2024-04-25T10:30:31.404392+0000","last_became_peered":"2024-04-25T10:30:31.404392+0000","last_unstale":"2024-04-25T10:30:33.415622+0000","last_undegraded":"2024-04-25T10:30:33.415622+0000","last_fullsized":"2024-04-25T10:30:33.415622+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:00.965450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795922+0000","last_change":"2024-04-25T10:30:32.441438+0000","last_active":"2024-04-25T10:30:34.795922+0000","last_peered":"2024-04-25T10:30:34.795922+0000","last_clean":"2024-04-25T10:30:34.795922+0000","last_became_active":"2024-04-25T10:30:32.440011+0000","last_became_peered":"2024-04-25T10:30:32.440011+0000","last_unstale":"2024-04-25T10:30:34.795922+0000","last_undegraded":"2024-04-25T10:30:34.795922+0000","last_fullsized":"2024-04-25T10:30:34.795922+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:09.127517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":110,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.793865+0000","last_change":"2024-04-25T10:30:21.429098+0000","last_active":"2024-04-25T10:30:34.793865+0000","last_peered":"2024-04-25T10:30:34.793865+0000","last_clean":"2024-04-25T10:30:34.793865+0000","last_became_active":"2024-04-25T10:30:21.427958+0000","last_became_peered":"2024-04-25T10:30:21.427958+0000","last_unstale":"2024-04-25T10:30:34.793865+0000","last_undegraded":"2024-04-25T10:30:34.793865+0000","last_fullsized":"2024-04-25T10:30:34.793865+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:20.186206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:20.186206+0000","last_clean_scrub_stamp":"2024-04-25T10:30:20.186206+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:46.046488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.793946+0000","last_change":"2024-04-25T10:30:31.404974+0000","last_active":"2024-04-25T10:30:34.793946+0000","last_peered":"2024-04-25T10:30:34.793946+0000","last_clean":"2024-04-25T10:30:34.793946+0000","last_became_active":"2024-04-25T10:30:31.404549+0000","last_became_peered":"2024-04-25T10:30:31.404549+0000","last_unstale":"2024-04-25T10:30:34.793946+0000","last_undegraded":"2024-04-25T10:30:34.793946+0000","last_fullsized":"2024-04-25T10:30:34.793946+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:06.191321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415412+0000","last_change":"2024-04-25T10:30:32.434409+0000","last_active":"2024-04-25T10:30:33.415412+0000","last_peered":"2024-04-25T10:30:33.415412+0000","last_clean":"2024-04-25T10:30:33.415412+0000","last_became_active":"2024-04-25T10:30:32.433830+0000","last_became_peered":"2024-04-25T10:30:32.433830+0000","last_unstale":"2024-04-25T10:30:33.415412+0000","last_undegraded":"2024-04-25T10:30:33.415412+0000","last_fullsized":"2024-04-25T10:30:33.415412+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:57.864627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.793793+0000","last_change":"2024-04-25T10:30:31.416025+0000","last_active":"2024-04-25T10:30:34.793793+0000","last_peered":"2024-04-25T10:30:34.793793+0000","last_clean":"2024-04-25T10:30:34.793793+0000","last_became_active":"2024-04-25T10:30:31.415675+0000","last_became_peered":"2024-04-25T10:30:31.415675+0000","last_unstale":"2024-04-25T10:30:34.793793+0000","last_undegraded":"2024-04-25T10:30:34.793793+0000","last_fullsized":"2024-04-25T10:30:34.793793+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:28:48.185715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.196505+0000","last_change":"2024-04-25T10:30:32.424630+0000","last_active":"2024-04-25T10:30:34.196505+0000","last_peered":"2024-04-25T10:30:34.196505+0000","last_clean":"2024-04-25T10:30:34.196505+0000","last_became_active":"2024-04-25T10:30:32.424303+0000","last_became_peered":"2024-04-25T10:30:32.424303+0000","last_unstale":"2024-04-25T10:30:34.196505+0000","last_undegraded":"2024-04-25T10:30:34.196505+0000","last_fullsized":"2024-04-25T10:30:34.196505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:55.535319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415284+0000","last_change":"2024-04-25T10:30:31.419145+0000","last_active":"2024-04-25T10:30:33.415284+0000","last_peered":"2024-04-25T10:30:33.415284+0000","last_clean":"2024-04-25T10:30:33.415284+0000","last_became_active":"2024-04-25T10:30:31.418817+0000","last_became_peered":"2024-04-25T10:30:31.418817+0000","last_unstale":"2024-04-25T10:30:33.415284+0000","last_undegraded":"2024-04-25T10:30:33.415284+0000","last_fullsized":"2024-04-25T10:30:33.415284+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:01.608205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.412646+0000","last_change":"2024-04-25T10:30:32.419911+0000","last_active":"2024-04-25T10:30:33.412646+0000","last_peered":"2024-04-25T10:30:33.412646+0000","last_clean":"2024-04-25T10:30:33.412646+0000","last_became_active":"2024-04-25T10:30:32.419486+0000","last_became_peered":"2024-04-25T10:30:32.419486+0000","last_unstale":"2024-04-25T10:30:33.412646+0000","last_undegraded":"2024-04-25T10:30:33.412646+0000","last_fullsized":"2024-04-25T10:30:33.412646+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:54.967873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.796060+0000","last_change":"2024-04-25T10:30:31.405646+0000","last_active":"2024-04-25T10:30:34.796060+0000","last_peered":"2024-04-25T10:30:34.796060+0000","last_clean":"2024-04-25T10:30:34.796060+0000","last_became_active":"2024-04-25T10:30:31.405238+0000","last_became_peered":"2024-04-25T10:30:31.405238+0000","last_unstale":"2024-04-25T10:30:34.796060+0000","last_undegraded":"2024-04-25T10:30:34.796060+0000","last_fullsized":"2024-04-25T10:30:34.796060+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:13.159787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795730+0000","last_change":"2024-04-25T10:30:32.426130+0000","last_active":"2024-04-25T10:30:34.795730+0000","last_peered":"2024-04-25T10:30:34.795730+0000","last_clean":"2024-04-25T10:30:34.795730+0000","last_became_active":"2024-04-25T10:30:32.425783+0000","last_became_peered":"2024-04-25T10:30:32.425783+0000","last_unstale":"2024-04-25T10:30:34.795730+0000","last_undegraded":"2024-04-25T10:30:34.795730+0000","last_fullsized":"2024-04-25T10:30:34.795730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:30.844640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.412747+0000","last_change":"2024-04-25T10:30:31.412736+0000","last_active":"2024-04-25T10:30:33.412747+0000","last_peered":"2024-04-25T10:30:33.412747+0000","last_clean":"2024-04-25T10:30:33.412747+0000","last_became_active":"2024-04-25T10:30:31.412413+0000","last_became_peered":"2024-04-25T10:30:31.412413+0000","last_unstale":"2024-04-25T10:30:33.412747+0000","last_undegraded":"2024-04-25T10:30:33.412747+0000","last_fullsized":"2024-04-25T10:30:33.412747+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:19:56.236327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795688+0000","last_change":"2024-04-25T10:30:32.440858+0000","last_active":"2024-04-25T10:30:34.795688+0000","last_peered":"2024-04-25T10:30:34.795688+0000","last_clean":"2024-04-25T10:30:34.795688+0000","last_became_active":"2024-04-25T10:30:32.440519+0000","last_became_peered":"2024-04-25T10:30:32.440519+0000","last_unstale":"2024-04-25T10:30:34.795688+0000","last_undegraded":"2024-04-25T10:30:34.795688+0000","last_fullsized":"2024-04-25T10:30:34.795688+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:21.288748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.796138+0000","last_change":"2024-04-25T10:30:31.417958+0000","last_active":"2024-04-25T10:30:34.796138+0000","last_peered":"2024-04-25T10:30:34.796138+0000","last_clean":"2024-04-25T10:30:34.796138+0000","last_became_active":"2024-04-25T10:30:31.417541+0000","last_became_peered":"2024-04-25T10:30:31.417541+0000","last_unstale":"2024-04-25T10:30:34.796138+0000","last_undegraded":"2024-04-25T10:30:34.796138+0000","last_fullsized":"2024-04-25T10:30:34.796138+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:35.373351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.412553+0000","last_change":"2024-04-25T10:30:32.420451+0000","last_active":"2024-04-25T10:30:33.412553+0000","last_peered":"2024-04-25T10:30:33.412553+0000","last_clean":"2024-04-25T10:30:33.412553+0000","last_became_active":"2024-04-25T10:30:32.420109+0000","last_became_peered":"2024-04-25T10:30:32.420109+0000","last_unstale":"2024-04-25T10:30:33.412553+0000","last_undegraded":"2024-04-25T10:30:33.412553+0000","last_fullsized":"2024-04-25T10:30:33.412553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:22.067828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.412601+0000","last_change":"2024-04-25T10:30:31.406005+0000","last_active":"2024-04-25T10:30:33.412601+0000","last_peered":"2024-04-25T10:30:33.412601+0000","last_clean":"2024-04-25T10:30:33.412601+0000","last_became_active":"2024-04-25T10:30:31.404739+0000","last_became_peered":"2024-04-25T10:30:31.404739+0000","last_unstale":"2024-04-25T10:30:33.412601+0000","last_undegraded":"2024-04-25T10:30:33.412601+0000","last_fullsized":"2024-04-25T10:30:33.412601+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:31.771123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415711+0000","last_change":"2024-04-25T10:30:32.437513+0000","last_active":"2024-04-25T10:30:33.415711+0000","last_peered":"2024-04-25T10:30:33.415711+0000","last_clean":"2024-04-25T10:30:33.415711+0000","last_became_active":"2024-04-25T10:30:32.437171+0000","last_became_peered":"2024-04-25T10:30:32.437171+0000","last_unstale":"2024-04-25T10:30:33.415711+0000","last_undegraded":"2024-04-25T10:30:33.415711+0000","last_fullsized":"2024-04-25T10:30:33.415711+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:50.924843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794361+0000","last_change":"2024-04-25T10:30:31.413140+0000","last_active":"2024-04-25T10:30:34.794361+0000","last_peered":"2024-04-25T10:30:34.794361+0000","last_clean":"2024-04-25T10:30:34.794361+0000","last_became_active":"2024-04-25T10:30:31.412821+0000","last_became_peered":"2024-04-25T10:30:31.412821+0000","last_unstale":"2024-04-25T10:30:34.794361+0000","last_undegraded":"2024-04-25T10:30:34.794361+0000","last_fullsized":"2024-04-25T10:30:34.794361+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:37.177254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.196799+0000","last_change":"2024-04-25T10:30:32.434333+0000","last_active":"2024-04-25T10:30:34.196799+0000","last_peered":"2024-04-25T10:30:34.196799+0000","last_clean":"2024-04-25T10:30:34.196799+0000","last_became_active":"2024-04-25T10:30:32.434000+0000","last_became_peered":"2024-04-25T10:30:32.434000+0000","last_unstale":"2024-04-25T10:30:34.196799+0000","last_undegraded":"2024-04-25T10:30:34.196799+0000","last_fullsized":"2024-04-25T10:30:34.196799+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:54.057457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794573+0000","last_change":"2024-04-25T10:30:31.419475+0000","last_active":"2024-04-25T10:30:34.794573+0000","last_peered":"2024-04-25T10:30:34.794573+0000","last_clean":"2024-04-25T10:30:34.794573+0000","last_became_active":"2024-04-25T10:30:31.417605+0000","last_became_peered":"2024-04-25T10:30:31.417605+0000","last_unstale":"2024-04-25T10:30:34.794573+0000","last_undegraded":"2024-04-25T10:30:34.794573+0000","last_fullsized":"2024-04-25T10:30:34.794573+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:00.948269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794414+0000","last_change":"2024-04-25T10:30:32.419693+0000","last_active":"2024-04-25T10:30:34.794414+0000","last_peered":"2024-04-25T10:30:34.794414+0000","last_clean":"2024-04-25T10:30:34.794414+0000","last_became_active":"2024-04-25T10:30:32.419370+0000","last_became_peered":"2024-04-25T10:30:32.419370+0000","last_unstale":"2024-04-25T10:30:34.794414+0000","last_undegraded":"2024-04-25T10:30:34.794414+0000","last_fullsized":"2024-04-25T10:30:34.794414+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:31.199065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.412581+0000","last_change":"2024-04-25T10:30:31.410767+0000","last_active":"2024-04-25T10:30:33.412581+0000","last_peered":"2024-04-25T10:30:33.412581+0000","last_clean":"2024-04-25T10:30:33.412581+0000","last_became_active":"2024-04-25T10:30:31.410404+0000","last_became_peered":"2024-04-25T10:30:31.410404+0000","last_unstale":"2024-04-25T10:30:33.412581+0000","last_undegraded":"2024-04-25T10:30:33.412581+0000","last_fullsized":"2024-04-25T10:30:33.412581+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:37.431532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415585+0000","last_change":"2024-04-25T10:30:32.433687+0000","last_active":"2024-04-25T10:30:33.415585+0000","last_peered":"2024-04-25T10:30:33.415585+0000","last_clean":"2024-04-25T10:30:33.415585+0000","last_became_active":"2024-04-25T10:30:32.433389+0000","last_became_peered":"2024-04-25T10:30:32.433389+0000","last_unstale":"2024-04-25T10:30:33.415585+0000","last_undegraded":"2024-04-25T10:30:33.415585+0000","last_fullsized":"2024-04-25T10:30:33.415585+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:39.155317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.412506+0000","last_change":"2024-04-25T10:30:31.419118+0000","last_active":"2024-04-25T10:30:33.412506+0000","last_peered":"2024-04-25T10:30:33.412506+0000","last_clean":"2024-04-25T10:30:33.412506+0000","last_became_active":"2024-04-25T10:30:31.418822+0000","last_became_peered":"2024-04-25T10:30:31.418822+0000","last_unstale":"2024-04-25T10:30:33.412506+0000","last_undegraded":"2024-04-25T10:30:33.412506+0000","last_fullsized":"2024-04-25T10:30:33.412506+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:22.755042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415890+0000","last_change":"2024-04-25T10:30:32.427894+0000","last_active":"2024-04-25T10:30:33.415890+0000","last_peered":"2024-04-25T10:30:33.415890+0000","last_clean":"2024-04-25T10:30:33.415890+0000","last_became_active":"2024-04-25T10:30:32.427504+0000","last_became_peered":"2024-04-25T10:30:32.427504+0000","last_unstale":"2024-04-25T10:30:33.415890+0000","last_undegraded":"2024-04-25T10:30:33.415890+0000","last_fullsized":"2024-04-25T10:30:33.415890+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:53.877484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794986+0000","last_change":"2024-04-25T10:30:31.417115+0000","last_active":"2024-04-25T10:30:34.794986+0000","last_peered":"2024-04-25T10:30:34.794986+0000","last_clean":"2024-04-25T10:30:34.794986+0000","last_became_active":"2024-04-25T10:30:31.416766+0000","last_became_peered":"2024-04-25T10:30:31.416766+0000","last_unstale":"2024-04-25T10:30:34.794986+0000","last_undegraded":"2024-04-25T10:30:34.794986+0000","last_fullsized":"2024-04-25T10:30:34.794986+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:06.021270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795407+0000","last_change":"2024-04-25T10:30:32.431674+0000","last_active":"2024-04-25T10:30:34.795407+0000","last_peered":"2024-04-25T10:30:34.795407+0000","last_clean":"2024-04-25T10:30:34.795407+0000","last_became_active":"2024-04-25T10:30:32.431395+0000","last_became_peered":"2024-04-25T10:30:32.431395+0000","last_unstale":"2024-04-25T10:30:34.795407+0000","last_undegraded":"2024-04-25T10:30:34.795407+0000","last_fullsized":"2024-04-25T10:30:34.795407+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:49.726323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795508+0000","last_change":"2024-04-25T10:30:32.437700+0000","last_active":"2024-04-25T10:30:34.795508+0000","last_peered":"2024-04-25T10:30:34.795508+0000","last_clean":"2024-04-25T10:30:34.795508+0000","last_became_active":"2024-04-25T10:30:32.437242+0000","last_became_peered":"2024-04-25T10:30:32.437242+0000","last_unstale":"2024-04-25T10:30:34.795508+0000","last_undegraded":"2024-04-25T10:30:34.795508+0000","last_fullsized":"2024-04-25T10:30:34.795508+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:06.499414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795739+0000","last_change":"2024-04-25T10:30:31.419678+0000","last_active":"2024-04-25T10:30:34.795739+0000","last_peered":"2024-04-25T10:30:34.795739+0000","last_clean":"2024-04-25T10:30:34.795739+0000","last_became_active":"2024-04-25T10:30:31.419269+0000","last_became_peered":"2024-04-25T10:30:31.419269+0000","last_unstale":"2024-04-25T10:30:34.795739+0000","last_undegraded":"2024-04-25T10:30:34.795739+0000","last_fullsized":"2024-04-25T10:30:34.795739+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:33.034260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795138+0000","last_change":"2024-04-25T10:30:32.434410+0000","last_active":"2024-04-25T10:30:34.795138+0000","last_peered":"2024-04-25T10:30:34.795138+0000","last_clean":"2024-04-25T10:30:34.795138+0000","last_became_active":"2024-04-25T10:30:32.434093+0000","last_became_peered":"2024-04-25T10:30:32.434093+0000","last_unstale":"2024-04-25T10:30:34.795138+0000","last_undegraded":"2024-04-25T10:30:34.795138+0000","last_fullsized":"2024-04-25T10:30:34.795138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:46.964704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795981+0000","last_change":"2024-04-25T10:30:31.418360+0000","last_active":"2024-04-25T10:30:34.795981+0000","last_peered":"2024-04-25T10:30:34.795981+0000","last_clean":"2024-04-25T10:30:34.795981+0000","last_became_active":"2024-04-25T10:30:31.418005+0000","last_became_peered":"2024-04-25T10:30:31.418005+0000","last_unstale":"2024-04-25T10:30:34.795981+0000","last_undegraded":"2024-04-25T10:30:34.795981+0000","last_fullsized":"2024-04-25T10:30:34.795981+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:15.556859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.196705+0000","last_change":"2024-04-25T10:30:32.439515+0000","last_active":"2024-04-25T10:30:34.196705+0000","last_peered":"2024-04-25T10:30:34.196705+0000","last_clean":"2024-04-25T10:30:34.196705+0000","last_became_active":"2024-04-25T10:30:32.439101+0000","last_became_peered":"2024-04-25T10:30:32.439101+0000","last_unstale":"2024-04-25T10:30:34.196705+0000","last_undegraded":"2024-04-25T10:30:34.196705+0000","last_fullsized":"2024-04-25T10:30:34.196705+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:33.797903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415821+0000","last_change":"2024-04-25T10:30:31.418070+0000","last_active":"2024-04-25T10:30:33.415821+0000","last_peered":"2024-04-25T10:30:33.415821+0000","last_clean":"2024-04-25T10:30:33.415821+0000","last_became_active":"2024-04-25T10:30:31.417737+0000","last_became_peered":"2024-04-25T10:30:31.417737+0000","last_unstale":"2024-04-25T10:30:33.415821+0000","last_undegraded":"2024-04-25T10:30:33.415821+0000","last_fullsized":"2024-04-25T10:30:33.415821+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:05.664855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415785+0000","last_change":"2024-04-25T10:30:32.435258+0000","last_active":"2024-04-25T10:30:33.415785+0000","last_peered":"2024-04-25T10:30:33.415785+0000","last_clean":"2024-04-25T10:30:33.415785+0000","last_became_active":"2024-04-25T10:30:32.434930+0000","last_became_peered":"2024-04-25T10:30:32.434930+0000","last_unstale":"2024-04-25T10:30:33.415785+0000","last_undegraded":"2024-04-25T10:30:33.415785+0000","last_fullsized":"2024-04-25T10:30:33.415785+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:22.727778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.196803+0000","last_change":"2024-04-25T10:30:31.415611+0000","last_active":"2024-04-25T10:30:34.196803+0000","last_peered":"2024-04-25T10:30:34.196803+0000","last_clean":"2024-04-25T10:30:34.196803+0000","last_became_active":"2024-04-25T10:30:31.415071+0000","last_became_peered":"2024-04-25T10:30:31.415071+0000","last_unstale":"2024-04-25T10:30:34.196803+0000","last_undegraded":"2024-04-25T10:30:34.196803+0000","last_fullsized":"2024-04-25T10:30:34.196803+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:30:54.077864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":786432,"data_stored":655002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":192512,"data_stored":64634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":192512,"data_stored":64634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":192512,"data_stored":64634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27800,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344040,"statfs":{"total":96636764160,"available":96608296960,"internally_reserved":0,"allocated":798720,"data_stored":661302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:30:37.442 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:30:37.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.584+0000 7f37c499d640 1 Processor -- start 2024-04-25T10:30:37.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.584+0000 7f37c499d640 1 -- start start 2024-04-25T10:30:37.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.586+0000 7f37c499d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f37c00c6d00 0x7f37c00b4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:37.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.586+0000 7f37c499d640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37c00ce200 con 0x7f37c006d4d0 2024-04-25T10:30:37.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.586+0000 7f37c499d640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37c015d510 con 0x7f37c0156d70 2024-04-25T10:30:37.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.586+0000 7f37c499d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f37c00c7350 con 0x7f37c00c6d00 2024-04-25T10:30:37.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.586+0000 7f37bffff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f37c006d4d0 0x7f37c0167270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:44246/0 (socket says 172.21.15.106:44246) 2024-04-25T10:30:37.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.586+0000 7f37bffff640 1 -- 172.21.15.106:0/2245714210 learned_addr learned my addr 172.21.15.106:0/2245714210 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:30:37.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.586+0000 7f37bf7fe640 1 --2- 172.21.15.106:0/2245714210 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f37c00c6d00 0x7f37c00b4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:37.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.586+0000 7f37be7fc640 1 -- 172.21.15.106:0/2245714210 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 496779360 0 0) 0x7f37c00ce200 con 0x7f37c006d4d0 2024-04-25T10:30:37.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.586+0000 7f37be7fc640 1 -- 172.21.15.106:0/2245714210 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f37a8003520 con 0x7f37c006d4d0 2024-04-25T10:30:37.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.586+0000 7f37be7fc640 1 -- 172.21.15.106:0/2245714210 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2741331977 0 0) 0x7f37c015d510 con 0x7f37c0156d70 2024-04-25T10:30:37.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.586+0000 7f37be7fc640 1 -- 172.21.15.106:0/2245714210 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f37c00ce200 con 0x7f37c0156d70 2024-04-25T10:30:37.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.586+0000 7f37be7fc640 1 -- 172.21.15.106:0/2245714210 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2236627662 0 0) 0x7f37a8003520 con 0x7f37c006d4d0 2024-04-25T10:30:37.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.586+0000 7f37be7fc640 1 -- 172.21.15.106:0/2245714210 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f37c00c6d00 msgr2=0x7f37c00b4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:37.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.586+0000 7f37be7fc640 1 --2- 172.21.15.106:0/2245714210 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f37c00c6d00 0x7f37c00b4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:37.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.586+0000 7f37be7fc640 1 -- 172.21.15.106:0/2245714210 >> v1:172.21.15.134:6789/0 conn(0x7f37c0156d70 legacy=0x7f37c00cd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:37.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.586+0000 7f37be7fc640 1 -- 172.21.15.106:0/2245714210 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37c00cc910 con 0x7f37c006d4d0 2024-04-25T10:30:37.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.586+0000 7f37be7fc640 1 -- 172.21.15.106:0/2245714210 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3776459350 0 0) 0x7f37b80019f0 con 0x7f37c006d4d0 2024-04-25T10:30:37.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.586+0000 7f37bf7fe640 1 --2- 172.21.15.106:0/2245714210 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f37c00c6d00 0x7f37c00b4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:37.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.586+0000 7f37be7fc640 1 -- 172.21.15.106:0/2245714210 >> v1:172.21.15.106:6789/0 conn(0x7f37c006d4d0 legacy=0x7f37c0167270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:37.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.586+0000 7f37be7fc640 1 --2- 172.21.15.106:0/2245714210 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37a8003b70 0x7f37a8003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:37.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.586+0000 7f37be7fc640 1 --2- 172.21.15.106:0/2245714210 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37a8004800 0x7f37a8004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:37.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.586+0000 7f37be7fc640 1 --2- 172.21.15.106:0/2245714210 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f37c00c6d00 0x7f37a80051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:37.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.586+0000 7f37be7fc640 1 -- 172.21.15.106:0/2245714210 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f37a800baa0 con 0x7f37a8004800 2024-04-25T10:30:37.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.586+0000 7f37be7fc640 1 -- 172.21.15.106:0/2245714210 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f37a800bc50 con 0x7f37a8003b70 2024-04-25T10:30:37.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.586+0000 7f37be7fc640 1 -- 172.21.15.106:0/2245714210 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f37a800be50 con 0x7f37c00c6d00 2024-04-25T10:30:37.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.587+0000 7f37beffd640 1 --2- 172.21.15.106:0/2245714210 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37a8004800 0x7f37a8004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:37.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.587+0000 7f37bf7fe640 1 --2- 172.21.15.106:0/2245714210 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37a8003b70 0x7f37a8003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:37.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.587+0000 7f37bffff640 1 --2- 172.21.15.106:0/2245714210 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f37c00c6d00 0x7f37a80051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:37.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.587+0000 7f37beffd640 1 -- 172.21.15.106:0/2245714210 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f37c00c6d00 msgr2=0x7f37a80051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:37.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.587+0000 7f37beffd640 1 --2- 172.21.15.106:0/2245714210 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f37c00c6d00 0x7f37a80051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:37.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.587+0000 7f37beffd640 1 -- 172.21.15.106:0/2245714210 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37a8003b70 msgr2=0x7f37a8003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:37.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.587+0000 7f37beffd640 1 --2- 172.21.15.106:0/2245714210 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37a8003b70 0x7f37a8003fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:37.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.587+0000 7f37beffd640 1 -- 172.21.15.106:0/2245714210 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37a8003520 con 0x7f37a8004800 2024-04-25T10:30:37.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.588+0000 7f37beffd640 1 --2- 172.21.15.106:0/2245714210 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37a8004800 0x7f37a8004c10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f37c00ce3d0 tx=0x7f37a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb1234b0643d6031 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:37.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.588+0000 7f37be7fc640 1 -- 172.21.15.106:0/2245714210 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b4001500 con 0x7f37a8004800 2024-04-25T10:30:37.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.588+0000 7f37be7fc640 1 -- 172.21.15.106:0/2245714210 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37b40052d0 con 0x7f37a8004800 2024-04-25T10:30:37.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.588+0000 7f37be7fc640 1 -- 172.21.15.106:0/2245714210 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b402ac40 con 0x7f37a8004800 2024-04-25T10:30:37.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.588+0000 7f37c499d640 1 -- 172.21.15.106:0/2245714210 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37a8004800 msgr2=0x7f37a8004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:37.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.588+0000 7f37c499d640 1 --2- 172.21.15.106:0/2245714210 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37a8004800 0x7f37a8004c10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f37c00ce3d0 tx=0x7f37a8005830 comp rx=0 tx=0).stop 2024-04-25T10:30:37.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.588+0000 7f37bffff640 1 -- 172.21.15.106:0/2245714210 reap_dead start 2024-04-25T10:30:37.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.588+0000 7f37c499d640 1 -- 172.21.15.106:0/2245714210 shutdown_connections 2024-04-25T10:30:37.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.588+0000 7f37c499d640 1 -- 172.21.15.106:0/2245714210 >> 172.21.15.106:0/2245714210 conn(0x7f37c0068cd0 msgr2=0x7f37c00d1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:37.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.588+0000 7f37c499d640 1 -- 172.21.15.106:0/2245714210 shutdown_connections 2024-04-25T10:30:37.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.588+0000 7f37c499d640 1 -- 172.21.15.106:0/2245714210 wait complete. 2024-04-25T10:30:37.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.590+0000 7f37c499d640 1 Processor -- start 2024-04-25T10:30:37.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.590+0000 7f37c499d640 1 -- start start 2024-04-25T10:30:37.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.591+0000 7f37c499d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f37a8004800 0x7f37c015a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:37.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.591+0000 7f37c499d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37c014f490 0x7f37c014e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:37.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.591+0000 7f37c499d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37c014f8a0 0x7f37c014e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:37.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.592+0000 7f37c499d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f37c00cdc10 con 0x7f37c014f8a0 2024-04-25T10:30:37.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.592+0000 7f37c499d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f37c015d7f0 con 0x7f37c014f490 2024-04-25T10:30:37.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.592+0000 7f37c499d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f37c00cd820 con 0x7f37a8004800 2024-04-25T10:30:37.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.592+0000 7f37bffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37c014f8a0 0x7f37c014e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:37.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.592+0000 7f37bffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37c014f8a0 0x7f37c014e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47508/0 (socket says 172.21.15.106:47508) 2024-04-25T10:30:37.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.592+0000 7f37bffff640 1 -- 172.21.15.106:0/3589266754 learned_addr learned my addr 172.21.15.106:0/3589266754 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:37.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.592+0000 7f37bf7fe640 1 --2- 172.21.15.106:0/3589266754 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f37a8004800 0x7f37c015a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:37.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.592+0000 7f37beffd640 1 --2- 172.21.15.106:0/3589266754 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37c014f490 0x7f37c014e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:37.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.592+0000 7f37bffff640 1 -- 172.21.15.106:0/3589266754 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f37a8004800 msgr2=0x7f37c015a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:37.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.592+0000 7f37bffff640 1 --2- 172.21.15.106:0/3589266754 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f37a8004800 0x7f37c015a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:37.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.592+0000 7f37bffff640 1 -- 172.21.15.106:0/3589266754 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37c014f490 msgr2=0x7f37c014e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:37.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.592+0000 7f37bffff640 1 --2- 172.21.15.106:0/3589266754 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37c014f490 0x7f37c014e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:37.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.592+0000 7f37bffff640 1 -- 172.21.15.106:0/3589266754 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37c015aaa0 con 0x7f37c014f8a0 2024-04-25T10:30:37.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.592+0000 7f37bffff640 1 --2- 172.21.15.106:0/3589266754 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37c014f8a0 0x7f37c014e7d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f37c008ff40 tx=0x7f37b8007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:37.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.592+0000 7f37a7fff640 1 -- 172.21.15.106:0/3589266754 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b8017020 con 0x7f37c014f8a0 2024-04-25T10:30:37.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.592+0000 7f37a7fff640 1 -- 172.21.15.106:0/3589266754 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37b800aab0 con 0x7f37c014f8a0 2024-04-25T10:30:37.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.592+0000 7f37c499d640 1 -- 172.21.15.106:0/3589266754 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37c0154d40 con 0x7f37c014f8a0 2024-04-25T10:30:37.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.592+0000 7f37a7fff640 1 -- 172.21.15.106:0/3589266754 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b80133f0 con 0x7f37c014f8a0 2024-04-25T10:30:37.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.592+0000 7f37c499d640 1 -- 172.21.15.106:0/3589266754 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37c01552c0 con 0x7f37c014f8a0 2024-04-25T10:30:37.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.593+0000 7f37c499d640 1 -- 172.21.15.106:0/3589266754 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37c0050740 con 0x7f37c014f8a0 2024-04-25T10:30:37.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.594+0000 7f37a7fff640 1 -- 172.21.15.106:0/3589266754 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f37b801a020 con 0x7f37c014f8a0 2024-04-25T10:30:37.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.594+0000 7f37a7fff640 1 --2- 172.21.15.106:0/3589266754 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f37ac0744e0 0x7f37ac0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:37.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.595+0000 7f37bf7fe640 1 --2- 172.21.15.106:0/3589266754 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f37ac0744e0 0x7f37ac0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:37.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.595+0000 7f37a7fff640 1 -- 172.21.15.106:0/3589266754 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f37b8095870 con 0x7f37c014f8a0 2024-04-25T10:30:37.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.595+0000 7f37bf7fe640 1 --2- 172.21.15.106:0/3589266754 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f37ac0744e0 0x7f37ac0769f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f37b0009560 tx=0x7f37b0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:37.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.599+0000 7f37a7fff640 1 -- 172.21.15.106:0/3589266754 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f37b8021020 con 0x7f37c014f8a0 2024-04-25T10:30:37.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.771+0000 7f37c499d640 1 -- 172.21.15.106:0/3589266754 --> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f37c00b4c00 con 0x7f37ac0744e0 2024-04-25T10:30:37.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.777+0000 7f37a7fff640 1 -- 172.21.15.106:0/3589266754 <== mgr.4107 v2:172.21.15.106:6832/1744210167 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251834 (secure 0 0 0) 0x7f37c00b4c00 con 0x7f37ac0744e0 2024-04-25T10:30:37.782 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:30:37.783 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-25T10:30:37.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.780+0000 7f37c499d640 1 -- 172.21.15.106:0/3589266754 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f37ac0744e0 msgr2=0x7f37ac0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:37.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.780+0000 7f37c499d640 1 --2- 172.21.15.106:0/3589266754 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f37ac0744e0 0x7f37ac0769f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f37b0009560 tx=0x7f37b0008040 comp rx=0 tx=0).stop 2024-04-25T10:30:37.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.780+0000 7f37c499d640 1 -- 172.21.15.106:0/3589266754 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37c014f8a0 msgr2=0x7f37c014e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:37.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.780+0000 7f37c499d640 1 --2- 172.21.15.106:0/3589266754 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37c014f8a0 0x7f37c014e7d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f37c008ff40 tx=0x7f37b8007bd0 comp rx=0 tx=0).stop 2024-04-25T10:30:37.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.780+0000 7f37c499d640 1 -- 172.21.15.106:0/3589266754 shutdown_connections 2024-04-25T10:30:37.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.780+0000 7f37c499d640 1 --2- 172.21.15.106:0/3589266754 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f37ac0744e0 0x7f37ac0769f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:37.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.780+0000 7f37c499d640 1 --2- 172.21.15.106:0/3589266754 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37c014f8a0 0x7f37c014e7d0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:37.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.780+0000 7f37c499d640 1 --2- 172.21.15.106:0/3589266754 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37c014f490 0x7f37c014e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:37.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.780+0000 7f37c499d640 1 --2- 172.21.15.106:0/3589266754 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f37a8004800 0x7f37c015a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:37.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.780+0000 7f37c499d640 1 -- 172.21.15.106:0/3589266754 >> 172.21.15.106:0/3589266754 conn(0x7f37c0068cd0 msgr2=0x7f37c00d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:37.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.780+0000 7f37c499d640 1 -- 172.21.15.106:0/3589266754 shutdown_connections 2024-04-25T10:30:37.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.780+0000 7f37c499d640 1 -- 172.21.15.106:0/3589266754 wait complete. 2024-04-25T10:30:37.793 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-04-25T10:30:36.096588+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":106,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218900,"kb_used_data":2712,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755820,"statfs":{"total":773094113280,"available":772869959680,"internally_reserved":0,"allocated":2777088,"data_stored":1723008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.703718"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795275+0000","last_change":"2024-04-25T10:30:32.440983+0000","last_active":"2024-04-25T10:30:34.795275+0000","last_peered":"2024-04-25T10:30:34.795275+0000","last_clean":"2024-04-25T10:30:34.795275+0000","last_became_active":"2024-04-25T10:30:32.440552+0000","last_became_peered":"2024-04-25T10:30:32.440552+0000","last_unstale":"2024-04-25T10:30:34.795275+0000","last_undegraded":"2024-04-25T10:30:34.795275+0000","last_fullsized":"2024-04-25T10:30:34.795275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:47.963061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.412686+0000","last_change":"2024-04-25T10:30:32.425695+0000","last_active":"2024-04-25T10:30:33.412686+0000","last_peered":"2024-04-25T10:30:33.412686+0000","last_clean":"2024-04-25T10:30:33.412686+0000","last_became_active":"2024-04-25T10:30:32.425368+0000","last_became_peered":"2024-04-25T10:30:32.425368+0000","last_unstale":"2024-04-25T10:30:33.412686+0000","last_undegraded":"2024-04-25T10:30:33.412686+0000","last_fullsized":"2024-04-25T10:30:33.412686+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:39.120522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.196620+0000","last_change":"2024-04-25T10:30:32.430144+0000","last_active":"2024-04-25T10:30:34.196620+0000","last_peered":"2024-04-25T10:30:34.196620+0000","last_clean":"2024-04-25T10:30:34.196620+0000","last_became_active":"2024-04-25T10:30:32.429754+0000","last_became_peered":"2024-04-25T10:30:32.429754+0000","last_unstale":"2024-04-25T10:30:34.196620+0000","last_undegraded":"2024-04-25T10:30:34.196620+0000","last_fullsized":"2024-04-25T10:30:34.196620+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:11.448907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794948+0000","last_change":"2024-04-25T10:30:32.442478+0000","last_active":"2024-04-25T10:30:34.794948+0000","last_peered":"2024-04-25T10:30:34.794948+0000","last_clean":"2024-04-25T10:30:34.794948+0000","last_became_active":"2024-04-25T10:30:32.442182+0000","last_became_peered":"2024-04-25T10:30:32.442182+0000","last_unstale":"2024-04-25T10:30:34.794948+0000","last_undegraded":"2024-04-25T10:30:34.794948+0000","last_fullsized":"2024-04-25T10:30:34.794948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:22.185978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794683+0000","last_change":"2024-04-25T10:30:32.438231+0000","last_active":"2024-04-25T10:30:34.794683+0000","last_peered":"2024-04-25T10:30:34.794683+0000","last_clean":"2024-04-25T10:30:34.794683+0000","last_became_active":"2024-04-25T10:30:32.437656+0000","last_became_peered":"2024-04-25T10:30:32.437656+0000","last_unstale":"2024-04-25T10:30:34.794683+0000","last_undegraded":"2024-04-25T10:30:34.794683+0000","last_fullsized":"2024-04-25T10:30:34.794683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:56.915999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795645+0000","last_change":"2024-04-25T10:30:32.439221+0000","last_active":"2024-04-25T10:30:34.795645+0000","last_peered":"2024-04-25T10:30:34.795645+0000","last_clean":"2024-04-25T10:30:34.795645+0000","last_became_active":"2024-04-25T10:30:32.438923+0000","last_became_peered":"2024-04-25T10:30:32.438923+0000","last_unstale":"2024-04-25T10:30:34.795645+0000","last_undegraded":"2024-04-25T10:30:34.795645+0000","last_fullsized":"2024-04-25T10:30:34.795645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:40.469409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794157+0000","last_change":"2024-04-25T10:30:32.429453+0000","last_active":"2024-04-25T10:30:34.794157+0000","last_peered":"2024-04-25T10:30:34.794157+0000","last_clean":"2024-04-25T10:30:34.794157+0000","last_became_active":"2024-04-25T10:30:32.429131+0000","last_became_peered":"2024-04-25T10:30:32.429131+0000","last_unstale":"2024-04-25T10:30:34.794157+0000","last_undegraded":"2024-04-25T10:30:34.794157+0000","last_fullsized":"2024-04-25T10:30:34.794157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:26.785922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415260+0000","last_change":"2024-04-25T10:30:32.438860+0000","last_active":"2024-04-25T10:30:33.415260+0000","last_peered":"2024-04-25T10:30:33.415260+0000","last_clean":"2024-04-25T10:30:33.415260+0000","last_became_active":"2024-04-25T10:30:32.438491+0000","last_became_peered":"2024-04-25T10:30:32.438491+0000","last_unstale":"2024-04-25T10:30:33.415260+0000","last_undegraded":"2024-04-25T10:30:33.415260+0000","last_fullsized":"2024-04-25T10:30:33.415260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:08.610707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795467+0000","last_change":"2024-04-25T10:30:32.440331+0000","last_active":"2024-04-25T10:30:34.795467+0000","last_peered":"2024-04-25T10:30:34.795467+0000","last_clean":"2024-04-25T10:30:34.795467+0000","last_became_active":"2024-04-25T10:30:32.440043+0000","last_became_peered":"2024-04-25T10:30:32.440043+0000","last_unstale":"2024-04-25T10:30:34.795467+0000","last_undegraded":"2024-04-25T10:30:34.795467+0000","last_fullsized":"2024-04-25T10:30:34.795467+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:17.684685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794636+0000","last_change":"2024-04-25T10:30:32.435388+0000","last_active":"2024-04-25T10:30:34.794636+0000","last_peered":"2024-04-25T10:30:34.794636+0000","last_clean":"2024-04-25T10:30:34.794636+0000","last_became_active":"2024-04-25T10:30:32.434983+0000","last_became_peered":"2024-04-25T10:30:32.434983+0000","last_unstale":"2024-04-25T10:30:34.794636+0000","last_undegraded":"2024-04-25T10:30:34.794636+0000","last_fullsized":"2024-04-25T10:30:34.794636+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:39.256604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.793928+0000","last_change":"2024-04-25T10:30:32.435884+0000","last_active":"2024-04-25T10:30:34.793928+0000","last_peered":"2024-04-25T10:30:34.793928+0000","last_clean":"2024-04-25T10:30:34.793928+0000","last_became_active":"2024-04-25T10:30:32.434645+0000","last_became_peered":"2024-04-25T10:30:32.434645+0000","last_unstale":"2024-04-25T10:30:34.793928+0000","last_undegraded":"2024-04-25T10:30:34.793928+0000","last_fullsized":"2024-04-25T10:30:34.793928+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:07.778437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795720+0000","last_change":"2024-04-25T10:30:32.436403+0000","last_active":"2024-04-25T10:30:34.795720+0000","last_peered":"2024-04-25T10:30:34.795720+0000","last_clean":"2024-04-25T10:30:34.795720+0000","last_became_active":"2024-04-25T10:30:32.435908+0000","last_became_peered":"2024-04-25T10:30:32.435908+0000","last_unstale":"2024-04-25T10:30:34.795720+0000","last_undegraded":"2024-04-25T10:30:34.795720+0000","last_fullsized":"2024-04-25T10:30:34.795720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:03.339316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415872+0000","last_change":"2024-04-25T10:30:32.432110+0000","last_active":"2024-04-25T10:30:33.415872+0000","last_peered":"2024-04-25T10:30:33.415872+0000","last_clean":"2024-04-25T10:30:33.415872+0000","last_became_active":"2024-04-25T10:30:32.431792+0000","last_became_peered":"2024-04-25T10:30:32.431792+0000","last_unstale":"2024-04-25T10:30:33.415872+0000","last_undegraded":"2024-04-25T10:30:33.415872+0000","last_fullsized":"2024-04-25T10:30:33.415872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:34.185989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.196476+0000","last_change":"2024-04-25T10:30:32.433676+0000","last_active":"2024-04-25T10:30:34.196476+0000","last_peered":"2024-04-25T10:30:34.196476+0000","last_clean":"2024-04-25T10:30:34.196476+0000","last_became_active":"2024-04-25T10:30:32.433329+0000","last_became_peered":"2024-04-25T10:30:32.433329+0000","last_unstale":"2024-04-25T10:30:34.196476+0000","last_undegraded":"2024-04-25T10:30:34.196476+0000","last_fullsized":"2024-04-25T10:30:34.196476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:59.942368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.412754+0000","last_change":"2024-04-25T10:30:32.423582+0000","last_active":"2024-04-25T10:30:33.412754+0000","last_peered":"2024-04-25T10:30:33.412754+0000","last_clean":"2024-04-25T10:30:33.412754+0000","last_became_active":"2024-04-25T10:30:32.422991+0000","last_became_peered":"2024-04-25T10:30:32.422991+0000","last_unstale":"2024-04-25T10:30:33.412754+0000","last_undegraded":"2024-04-25T10:30:33.412754+0000","last_fullsized":"2024-04-25T10:30:33.412754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:11:47.132928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794545+0000","last_change":"2024-04-25T10:30:32.440235+0000","last_active":"2024-04-25T10:30:34.794545+0000","last_peered":"2024-04-25T10:30:34.794545+0000","last_clean":"2024-04-25T10:30:34.794545+0000","last_became_active":"2024-04-25T10:30:32.439191+0000","last_became_peered":"2024-04-25T10:30:32.439191+0000","last_unstale":"2024-04-25T10:30:34.794545+0000","last_undegraded":"2024-04-25T10:30:34.794545+0000","last_fullsized":"2024-04-25T10:30:34.794545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:51.423086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795170+0000","last_change":"2024-04-25T10:30:32.442418+0000","last_active":"2024-04-25T10:30:34.795170+0000","last_peered":"2024-04-25T10:30:34.795170+0000","last_clean":"2024-04-25T10:30:34.795170+0000","last_became_active":"2024-04-25T10:30:32.442125+0000","last_became_peered":"2024-04-25T10:30:32.442125+0000","last_unstale":"2024-04-25T10:30:34.795170+0000","last_undegraded":"2024-04-25T10:30:34.795170+0000","last_fullsized":"2024-04-25T10:30:34.795170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:14.213809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415344+0000","last_change":"2024-04-25T10:30:32.434758+0000","last_active":"2024-04-25T10:30:33.415344+0000","last_peered":"2024-04-25T10:30:33.415344+0000","last_clean":"2024-04-25T10:30:33.415344+0000","last_became_active":"2024-04-25T10:30:32.434492+0000","last_became_peered":"2024-04-25T10:30:32.434492+0000","last_unstale":"2024-04-25T10:30:33.415344+0000","last_undegraded":"2024-04-25T10:30:33.415344+0000","last_fullsized":"2024-04-25T10:30:33.415344+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:41.454407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.796066+0000","last_change":"2024-04-25T10:30:32.440399+0000","last_active":"2024-04-25T10:30:34.796066+0000","last_peered":"2024-04-25T10:30:34.796066+0000","last_clean":"2024-04-25T10:30:34.796066+0000","last_became_active":"2024-04-25T10:30:32.438227+0000","last_became_peered":"2024-04-25T10:30:32.438227+0000","last_unstale":"2024-04-25T10:30:34.796066+0000","last_undegraded":"2024-04-25T10:30:34.796066+0000","last_fullsized":"2024-04-25T10:30:34.796066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:49.571671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794149+0000","last_change":"2024-04-25T10:30:32.431609+0000","last_active":"2024-04-25T10:30:34.794149+0000","last_peered":"2024-04-25T10:30:34.794149+0000","last_clean":"2024-04-25T10:30:34.794149+0000","last_became_active":"2024-04-25T10:30:32.431314+0000","last_became_peered":"2024-04-25T10:30:32.431314+0000","last_unstale":"2024-04-25T10:30:34.794149+0000","last_undegraded":"2024-04-25T10:30:34.794149+0000","last_fullsized":"2024-04-25T10:30:34.794149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:33.421989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.196668+0000","last_change":"2024-04-25T10:30:32.432612+0000","last_active":"2024-04-25T10:30:34.196668+0000","last_peered":"2024-04-25T10:30:34.196668+0000","last_clean":"2024-04-25T10:30:34.196668+0000","last_became_active":"2024-04-25T10:30:32.432271+0000","last_became_peered":"2024-04-25T10:30:32.432271+0000","last_unstale":"2024-04-25T10:30:34.196668+0000","last_undegraded":"2024-04-25T10:30:34.196668+0000","last_fullsized":"2024-04-25T10:30:34.196668+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:14.281493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795827+0000","last_change":"2024-04-25T10:30:32.444618+0000","last_active":"2024-04-25T10:30:34.795827+0000","last_peered":"2024-04-25T10:30:34.795827+0000","last_clean":"2024-04-25T10:30:34.795827+0000","last_became_active":"2024-04-25T10:30:32.444273+0000","last_became_peered":"2024-04-25T10:30:32.444273+0000","last_unstale":"2024-04-25T10:30:34.795827+0000","last_undegraded":"2024-04-25T10:30:34.795827+0000","last_fullsized":"2024-04-25T10:30:34.795827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:09.295251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415348+0000","last_change":"2024-04-25T10:30:32.426578+0000","last_active":"2024-04-25T10:30:33.415348+0000","last_peered":"2024-04-25T10:30:33.415348+0000","last_clean":"2024-04-25T10:30:33.415348+0000","last_became_active":"2024-04-25T10:30:32.426223+0000","last_became_peered":"2024-04-25T10:30:32.426223+0000","last_unstale":"2024-04-25T10:30:33.415348+0000","last_undegraded":"2024-04-25T10:30:33.415348+0000","last_fullsized":"2024-04-25T10:30:33.415348+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:53.700585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794721+0000","last_change":"2024-04-25T10:30:32.439767+0000","last_active":"2024-04-25T10:30:34.794721+0000","last_peered":"2024-04-25T10:30:34.794721+0000","last_clean":"2024-04-25T10:30:34.794721+0000","last_became_active":"2024-04-25T10:30:32.438801+0000","last_became_peered":"2024-04-25T10:30:32.438801+0000","last_unstale":"2024-04-25T10:30:34.794721+0000","last_undegraded":"2024-04-25T10:30:34.794721+0000","last_fullsized":"2024-04-25T10:30:34.794721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:58.231512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415544+0000","last_change":"2024-04-25T10:30:32.427424+0000","last_active":"2024-04-25T10:30:33.415544+0000","last_peered":"2024-04-25T10:30:33.415544+0000","last_clean":"2024-04-25T10:30:33.415544+0000","last_became_active":"2024-04-25T10:30:32.427081+0000","last_became_peered":"2024-04-25T10:30:32.427081+0000","last_unstale":"2024-04-25T10:30:33.415544+0000","last_undegraded":"2024-04-25T10:30:33.415544+0000","last_fullsized":"2024-04-25T10:30:33.415544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:47.372451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794727+0000","last_change":"2024-04-25T10:30:32.432472+0000","last_active":"2024-04-25T10:30:34.794727+0000","last_peered":"2024-04-25T10:30:34.794727+0000","last_clean":"2024-04-25T10:30:34.794727+0000","last_became_active":"2024-04-25T10:30:32.432121+0000","last_became_peered":"2024-04-25T10:30:32.432121+0000","last_unstale":"2024-04-25T10:30:34.794727+0000","last_undegraded":"2024-04-25T10:30:34.794727+0000","last_fullsized":"2024-04-25T10:30:34.794727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:34.191339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.412571+0000","last_change":"2024-04-25T10:30:32.425108+0000","last_active":"2024-04-25T10:30:33.412571+0000","last_peered":"2024-04-25T10:30:33.412571+0000","last_clean":"2024-04-25T10:30:33.412571+0000","last_became_active":"2024-04-25T10:30:32.424794+0000","last_became_peered":"2024-04-25T10:30:32.424794+0000","last_unstale":"2024-04-25T10:30:33.412571+0000","last_undegraded":"2024-04-25T10:30:33.412571+0000","last_fullsized":"2024-04-25T10:30:33.412571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:01.282758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415734+0000","last_change":"2024-04-25T10:30:32.426672+0000","last_active":"2024-04-25T10:30:33.415734+0000","last_peered":"2024-04-25T10:30:33.415734+0000","last_clean":"2024-04-25T10:30:33.415734+0000","last_became_active":"2024-04-25T10:30:32.426348+0000","last_became_peered":"2024-04-25T10:30:32.426348+0000","last_unstale":"2024-04-25T10:30:33.415734+0000","last_undegraded":"2024-04-25T10:30:33.415734+0000","last_fullsized":"2024-04-25T10:30:33.415734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:57.645772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.196609+0000","last_change":"2024-04-25T10:30:32.427989+0000","last_active":"2024-04-25T10:30:34.196609+0000","last_peered":"2024-04-25T10:30:34.196609+0000","last_clean":"2024-04-25T10:30:34.196609+0000","last_became_active":"2024-04-25T10:30:32.427676+0000","last_became_peered":"2024-04-25T10:30:32.427676+0000","last_unstale":"2024-04-25T10:30:34.196609+0000","last_undegraded":"2024-04-25T10:30:34.196609+0000","last_fullsized":"2024-04-25T10:30:34.196609+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:15:02.332186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795935+0000","last_change":"2024-04-25T10:30:32.441112+0000","last_active":"2024-04-25T10:30:34.795935+0000","last_peered":"2024-04-25T10:30:34.795935+0000","last_clean":"2024-04-25T10:30:34.795935+0000","last_became_active":"2024-04-25T10:30:32.440815+0000","last_became_peered":"2024-04-25T10:30:32.440815+0000","last_unstale":"2024-04-25T10:30:34.795935+0000","last_undegraded":"2024-04-25T10:30:34.795935+0000","last_fullsized":"2024-04-25T10:30:34.795935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:02.956759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415178+0000","last_change":"2024-04-25T10:30:32.430902+0000","last_active":"2024-04-25T10:30:33.415178+0000","last_peered":"2024-04-25T10:30:33.415178+0000","last_clean":"2024-04-25T10:30:33.415178+0000","last_became_active":"2024-04-25T10:30:32.430615+0000","last_became_peered":"2024-04-25T10:30:32.430615+0000","last_unstale":"2024-04-25T10:30:33.415178+0000","last_undegraded":"2024-04-25T10:30:33.415178+0000","last_fullsized":"2024-04-25T10:30:33.415178+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:12.747595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795247+0000","last_change":"2024-04-25T10:30:32.435379+0000","last_active":"2024-04-25T10:30:34.795247+0000","last_peered":"2024-04-25T10:30:34.795247+0000","last_clean":"2024-04-25T10:30:34.795247+0000","last_became_active":"2024-04-25T10:30:32.435055+0000","last_became_peered":"2024-04-25T10:30:32.435055+0000","last_unstale":"2024-04-25T10:30:34.795247+0000","last_undegraded":"2024-04-25T10:30:34.795247+0000","last_fullsized":"2024-04-25T10:30:34.795247+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:13.623762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794490+0000","last_change":"2024-04-25T10:30:32.422102+0000","last_active":"2024-04-25T10:30:34.794490+0000","last_peered":"2024-04-25T10:30:34.794490+0000","last_clean":"2024-04-25T10:30:34.794490+0000","last_became_active":"2024-04-25T10:30:32.421766+0000","last_became_peered":"2024-04-25T10:30:32.421766+0000","last_unstale":"2024-04-25T10:30:34.794490+0000","last_undegraded":"2024-04-25T10:30:34.794490+0000","last_fullsized":"2024-04-25T10:30:34.794490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:30.300572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415973+0000","last_change":"2024-04-25T10:30:31.418065+0000","last_active":"2024-04-25T10:30:33.415973+0000","last_peered":"2024-04-25T10:30:33.415973+0000","last_clean":"2024-04-25T10:30:33.415973+0000","last_became_active":"2024-04-25T10:30:31.417451+0000","last_became_peered":"2024-04-25T10:30:31.417451+0000","last_unstale":"2024-04-25T10:30:33.415973+0000","last_undegraded":"2024-04-25T10:30:33.415973+0000","last_fullsized":"2024-04-25T10:30:33.415973+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:17.262861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.196777+0000","last_change":"2024-04-25T10:30:32.426913+0000","last_active":"2024-04-25T10:30:34.196777+0000","last_peered":"2024-04-25T10:30:34.196777+0000","last_clean":"2024-04-25T10:30:34.196777+0000","last_became_active":"2024-04-25T10:30:32.426616+0000","last_became_peered":"2024-04-25T10:30:32.426616+0000","last_unstale":"2024-04-25T10:30:34.196777+0000","last_undegraded":"2024-04-25T10:30:34.196777+0000","last_fullsized":"2024-04-25T10:30:34.196777+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:50.614932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795542+0000","last_change":"2024-04-25T10:30:31.416432+0000","last_active":"2024-04-25T10:30:34.795542+0000","last_peered":"2024-04-25T10:30:34.795542+0000","last_clean":"2024-04-25T10:30:34.795542+0000","last_became_active":"2024-04-25T10:30:31.415112+0000","last_became_peered":"2024-04-25T10:30:31.415112+0000","last_unstale":"2024-04-25T10:30:34.795542+0000","last_undegraded":"2024-04-25T10:30:34.795542+0000","last_fullsized":"2024-04-25T10:30:34.795542+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:22.674365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415646+0000","last_change":"2024-04-25T10:30:31.418059+0000","last_active":"2024-04-25T10:30:33.415646+0000","last_peered":"2024-04-25T10:30:33.415646+0000","last_clean":"2024-04-25T10:30:33.415646+0000","last_became_active":"2024-04-25T10:30:31.417716+0000","last_became_peered":"2024-04-25T10:30:31.417716+0000","last_unstale":"2024-04-25T10:30:33.415646+0000","last_undegraded":"2024-04-25T10:30:33.415646+0000","last_fullsized":"2024-04-25T10:30:33.415646+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:57.752218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415428+0000","last_change":"2024-04-25T10:30:32.428078+0000","last_active":"2024-04-25T10:30:33.415428+0000","last_peered":"2024-04-25T10:30:33.415428+0000","last_clean":"2024-04-25T10:30:33.415428+0000","last_became_active":"2024-04-25T10:30:32.427700+0000","last_became_peered":"2024-04-25T10:30:32.427700+0000","last_unstale":"2024-04-25T10:30:33.415428+0000","last_undegraded":"2024-04-25T10:30:33.415428+0000","last_fullsized":"2024-04-25T10:30:33.415428+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:26.931791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795722+0000","last_change":"2024-04-25T10:30:31.415030+0000","last_active":"2024-04-25T10:30:34.795722+0000","last_peered":"2024-04-25T10:30:34.795722+0000","last_clean":"2024-04-25T10:30:34.795722+0000","last_became_active":"2024-04-25T10:30:31.414542+0000","last_became_peered":"2024-04-25T10:30:31.414542+0000","last_unstale":"2024-04-25T10:30:34.795722+0000","last_undegraded":"2024-04-25T10:30:34.795722+0000","last_fullsized":"2024-04-25T10:30:34.795722+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:20.082420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.412471+0000","last_change":"2024-04-25T10:30:32.425207+0000","last_active":"2024-04-25T10:30:33.412471+0000","last_peered":"2024-04-25T10:30:33.412471+0000","last_clean":"2024-04-25T10:30:33.412471+0000","last_became_active":"2024-04-25T10:30:32.424849+0000","last_became_peered":"2024-04-25T10:30:32.424849+0000","last_unstale":"2024-04-25T10:30:33.412471+0000","last_undegraded":"2024-04-25T10:30:33.412471+0000","last_fullsized":"2024-04-25T10:30:33.412471+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:11.533686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794249+0000","last_change":"2024-04-25T10:30:31.407700+0000","last_active":"2024-04-25T10:30:34.794249+0000","last_peered":"2024-04-25T10:30:34.794249+0000","last_clean":"2024-04-25T10:30:34.794249+0000","last_became_active":"2024-04-25T10:30:31.407376+0000","last_became_peered":"2024-04-25T10:30:31.407376+0000","last_unstale":"2024-04-25T10:30:34.794249+0000","last_undegraded":"2024-04-25T10:30:34.794249+0000","last_fullsized":"2024-04-25T10:30:34.794249+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:50.112742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794157+0000","last_change":"2024-04-25T10:30:32.444599+0000","last_active":"2024-04-25T10:30:34.794157+0000","last_peered":"2024-04-25T10:30:34.794157+0000","last_clean":"2024-04-25T10:30:34.794157+0000","last_became_active":"2024-04-25T10:30:32.444250+0000","last_became_peered":"2024-04-25T10:30:32.444250+0000","last_unstale":"2024-04-25T10:30:34.794157+0000","last_undegraded":"2024-04-25T10:30:34.794157+0000","last_fullsized":"2024-04-25T10:30:34.794157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:55.711525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794280+0000","last_change":"2024-04-25T10:30:31.405051+0000","last_active":"2024-04-25T10:30:34.794280+0000","last_peered":"2024-04-25T10:30:34.794280+0000","last_clean":"2024-04-25T10:30:34.794280+0000","last_became_active":"2024-04-25T10:30:31.404669+0000","last_became_peered":"2024-04-25T10:30:31.404669+0000","last_unstale":"2024-04-25T10:30:34.794280+0000","last_undegraded":"2024-04-25T10:30:34.794280+0000","last_fullsized":"2024-04-25T10:30:34.794280+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:45.260847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.196763+0000","last_change":"2024-04-25T10:30:32.435804+0000","last_active":"2024-04-25T10:30:34.196763+0000","last_peered":"2024-04-25T10:30:34.196763+0000","last_clean":"2024-04-25T10:30:34.196763+0000","last_became_active":"2024-04-25T10:30:32.435471+0000","last_became_peered":"2024-04-25T10:30:32.435471+0000","last_unstale":"2024-04-25T10:30:34.196763+0000","last_undegraded":"2024-04-25T10:30:34.196763+0000","last_fullsized":"2024-04-25T10:30:34.196763+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:48.213846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.412616+0000","last_change":"2024-04-25T10:30:31.403560+0000","last_active":"2024-04-25T10:30:33.412616+0000","last_peered":"2024-04-25T10:30:33.412616+0000","last_clean":"2024-04-25T10:30:33.412616+0000","last_became_active":"2024-04-25T10:30:31.403167+0000","last_became_peered":"2024-04-25T10:30:31.403167+0000","last_unstale":"2024-04-25T10:30:33.412616+0000","last_undegraded":"2024-04-25T10:30:33.412616+0000","last_fullsized":"2024-04-25T10:30:33.412616+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:54.592832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795908+0000","last_change":"2024-04-25T10:30:32.424101+0000","last_active":"2024-04-25T10:30:34.795908+0000","last_peered":"2024-04-25T10:30:34.795908+0000","last_clean":"2024-04-25T10:30:34.795908+0000","last_became_active":"2024-04-25T10:30:32.423780+0000","last_became_peered":"2024-04-25T10:30:32.423780+0000","last_unstale":"2024-04-25T10:30:34.795908+0000","last_undegraded":"2024-04-25T10:30:34.795908+0000","last_fullsized":"2024-04-25T10:30:34.795908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:29.175348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794280+0000","last_change":"2024-04-25T10:30:31.418201+0000","last_active":"2024-04-25T10:30:34.794280+0000","last_peered":"2024-04-25T10:30:34.794280+0000","last_clean":"2024-04-25T10:30:34.794280+0000","last_became_active":"2024-04-25T10:30:31.417824+0000","last_became_peered":"2024-04-25T10:30:31.417824+0000","last_unstale":"2024-04-25T10:30:34.794280+0000","last_undegraded":"2024-04-25T10:30:34.794280+0000","last_fullsized":"2024-04-25T10:30:34.794280+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:38.584225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795830+0000","last_change":"2024-04-25T10:30:32.431139+0000","last_active":"2024-04-25T10:30:34.795830+0000","last_peered":"2024-04-25T10:30:34.795830+0000","last_clean":"2024-04-25T10:30:34.795830+0000","last_became_active":"2024-04-25T10:30:32.429622+0000","last_became_peered":"2024-04-25T10:30:32.429622+0000","last_unstale":"2024-04-25T10:30:34.795830+0000","last_undegraded":"2024-04-25T10:30:34.795830+0000","last_fullsized":"2024-04-25T10:30:34.795830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:39.993647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.196800+0000","last_change":"2024-04-25T10:30:31.414775+0000","last_active":"2024-04-25T10:30:34.196800+0000","last_peered":"2024-04-25T10:30:34.196800+0000","last_clean":"2024-04-25T10:30:34.196800+0000","last_became_active":"2024-04-25T10:30:31.414422+0000","last_became_peered":"2024-04-25T10:30:31.414422+0000","last_unstale":"2024-04-25T10:30:34.196800+0000","last_undegraded":"2024-04-25T10:30:34.196800+0000","last_fullsized":"2024-04-25T10:30:34.196800+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:37.803505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.196648+0000","last_change":"2024-04-25T10:30:32.432884+0000","last_active":"2024-04-25T10:30:34.196648+0000","last_peered":"2024-04-25T10:30:34.196648+0000","last_clean":"2024-04-25T10:30:34.196648+0000","last_became_active":"2024-04-25T10:30:32.431743+0000","last_became_peered":"2024-04-25T10:30:32.431743+0000","last_unstale":"2024-04-25T10:30:34.196648+0000","last_undegraded":"2024-04-25T10:30:34.196648+0000","last_fullsized":"2024-04-25T10:30:34.196648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:15:06.737631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795999+0000","last_change":"2024-04-25T10:30:31.419497+0000","last_active":"2024-04-25T10:30:34.795999+0000","last_peered":"2024-04-25T10:30:34.795999+0000","last_clean":"2024-04-25T10:30:34.795999+0000","last_became_active":"2024-04-25T10:30:31.419135+0000","last_became_peered":"2024-04-25T10:30:31.419135+0000","last_unstale":"2024-04-25T10:30:34.795999+0000","last_undegraded":"2024-04-25T10:30:34.795999+0000","last_fullsized":"2024-04-25T10:30:34.795999+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:59.403927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794867+0000","last_change":"2024-04-25T10:30:32.434197+0000","last_active":"2024-04-25T10:30:34.794867+0000","last_peered":"2024-04-25T10:30:34.794867+0000","last_clean":"2024-04-25T10:30:34.794867+0000","last_became_active":"2024-04-25T10:30:32.433623+0000","last_became_peered":"2024-04-25T10:30:32.433623+0000","last_unstale":"2024-04-25T10:30:34.794867+0000","last_undegraded":"2024-04-25T10:30:34.794867+0000","last_fullsized":"2024-04-25T10:30:34.794867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:08.027472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.793813+0000","last_change":"2024-04-25T10:30:31.416901+0000","last_active":"2024-04-25T10:30:34.793813+0000","last_peered":"2024-04-25T10:30:34.793813+0000","last_clean":"2024-04-25T10:30:34.793813+0000","last_became_active":"2024-04-25T10:30:31.415706+0000","last_became_peered":"2024-04-25T10:30:31.415706+0000","last_unstale":"2024-04-25T10:30:34.793813+0000","last_undegraded":"2024-04-25T10:30:34.793813+0000","last_fullsized":"2024-04-25T10:30:34.793813+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:23.243039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795344+0000","last_change":"2024-04-25T10:30:32.435089+0000","last_active":"2024-04-25T10:30:34.795344+0000","last_peered":"2024-04-25T10:30:34.795344+0000","last_clean":"2024-04-25T10:30:34.795344+0000","last_became_active":"2024-04-25T10:30:32.434737+0000","last_became_peered":"2024-04-25T10:30:32.434737+0000","last_unstale":"2024-04-25T10:30:34.795344+0000","last_undegraded":"2024-04-25T10:30:34.795344+0000","last_fullsized":"2024-04-25T10:30:34.795344+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:19.738850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415107+0000","last_change":"2024-04-25T10:30:31.405163+0000","last_active":"2024-04-25T10:30:33.415107+0000","last_peered":"2024-04-25T10:30:33.415107+0000","last_clean":"2024-04-25T10:30:33.415107+0000","last_became_active":"2024-04-25T10:30:31.404775+0000","last_became_peered":"2024-04-25T10:30:31.404775+0000","last_unstale":"2024-04-25T10:30:33.415107+0000","last_undegraded":"2024-04-25T10:30:33.415107+0000","last_fullsized":"2024-04-25T10:30:33.415107+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:10:21.595653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794446+0000","last_change":"2024-04-25T10:30:32.432061+0000","last_active":"2024-04-25T10:30:34.794446+0000","last_peered":"2024-04-25T10:30:34.794446+0000","last_clean":"2024-04-25T10:30:34.794446+0000","last_became_active":"2024-04-25T10:30:32.431651+0000","last_became_peered":"2024-04-25T10:30:32.431651+0000","last_unstale":"2024-04-25T10:30:34.794446+0000","last_undegraded":"2024-04-25T10:30:34.794446+0000","last_fullsized":"2024-04-25T10:30:34.794446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:39.062490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795328+0000","last_change":"2024-04-25T10:30:31.420258+0000","last_active":"2024-04-25T10:30:34.795328+0000","last_peered":"2024-04-25T10:30:34.795328+0000","last_clean":"2024-04-25T10:30:34.795328+0000","last_became_active":"2024-04-25T10:30:31.419902+0000","last_became_peered":"2024-04-25T10:30:31.419902+0000","last_unstale":"2024-04-25T10:30:34.795328+0000","last_undegraded":"2024-04-25T10:30:34.795328+0000","last_fullsized":"2024-04-25T10:30:34.795328+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:43.919867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795443+0000","last_change":"2024-04-25T10:30:32.433568+0000","last_active":"2024-04-25T10:30:34.795443+0000","last_peered":"2024-04-25T10:30:34.795443+0000","last_clean":"2024-04-25T10:30:34.795443+0000","last_became_active":"2024-04-25T10:30:32.432572+0000","last_became_peered":"2024-04-25T10:30:32.432572+0000","last_unstale":"2024-04-25T10:30:34.795443+0000","last_undegraded":"2024-04-25T10:30:34.795443+0000","last_fullsized":"2024-04-25T10:30:34.795443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:31.771919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415439+0000","last_change":"2024-04-25T10:30:31.419308+0000","last_active":"2024-04-25T10:30:33.415439+0000","last_peered":"2024-04-25T10:30:33.415439+0000","last_clean":"2024-04-25T10:30:33.415439+0000","last_became_active":"2024-04-25T10:30:31.418978+0000","last_became_peered":"2024-04-25T10:30:31.418978+0000","last_unstale":"2024-04-25T10:30:33.415439+0000","last_undegraded":"2024-04-25T10:30:33.415439+0000","last_fullsized":"2024-04-25T10:30:33.415439+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:33.784804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795670+0000","last_change":"2024-04-25T10:30:32.436109+0000","last_active":"2024-04-25T10:30:34.795670+0000","last_peered":"2024-04-25T10:30:34.795670+0000","last_clean":"2024-04-25T10:30:34.795670+0000","last_became_active":"2024-04-25T10:30:32.434701+0000","last_became_peered":"2024-04-25T10:30:32.434701+0000","last_unstale":"2024-04-25T10:30:34.795670+0000","last_undegraded":"2024-04-25T10:30:34.795670+0000","last_fullsized":"2024-04-25T10:30:34.795670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:19.471867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.412672+0000","last_change":"2024-04-25T10:30:31.416044+0000","last_active":"2024-04-25T10:30:33.412672+0000","last_peered":"2024-04-25T10:30:33.412672+0000","last_clean":"2024-04-25T10:30:33.412672+0000","last_became_active":"2024-04-25T10:30:31.415707+0000","last_became_peered":"2024-04-25T10:30:31.415707+0000","last_unstale":"2024-04-25T10:30:33.412672+0000","last_undegraded":"2024-04-25T10:30:33.412672+0000","last_fullsized":"2024-04-25T10:30:33.412672+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:31.424135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795321+0000","last_change":"2024-04-25T10:30:32.442971+0000","last_active":"2024-04-25T10:30:34.795321+0000","last_peered":"2024-04-25T10:30:34.795321+0000","last_clean":"2024-04-25T10:30:34.795321+0000","last_became_active":"2024-04-25T10:30:32.442656+0000","last_became_peered":"2024-04-25T10:30:32.442656+0000","last_unstale":"2024-04-25T10:30:34.795321+0000","last_undegraded":"2024-04-25T10:30:34.795321+0000","last_fullsized":"2024-04-25T10:30:34.795321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:29.951142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415622+0000","last_change":"2024-04-25T10:30:31.404793+0000","last_active":"2024-04-25T10:30:33.415622+0000","last_peered":"2024-04-25T10:30:33.415622+0000","last_clean":"2024-04-25T10:30:33.415622+0000","last_became_active":"2024-04-25T10:30:31.404392+0000","last_became_peered":"2024-04-25T10:30:31.404392+0000","last_unstale":"2024-04-25T10:30:33.415622+0000","last_undegraded":"2024-04-25T10:30:33.415622+0000","last_fullsized":"2024-04-25T10:30:33.415622+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:00.965450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795922+0000","last_change":"2024-04-25T10:30:32.441438+0000","last_active":"2024-04-25T10:30:34.795922+0000","last_peered":"2024-04-25T10:30:34.795922+0000","last_clean":"2024-04-25T10:30:34.795922+0000","last_became_active":"2024-04-25T10:30:32.440011+0000","last_became_peered":"2024-04-25T10:30:32.440011+0000","last_unstale":"2024-04-25T10:30:34.795922+0000","last_undegraded":"2024-04-25T10:30:34.795922+0000","last_fullsized":"2024-04-25T10:30:34.795922+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:09.127517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":110,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.793865+0000","last_change":"2024-04-25T10:30:21.429098+0000","last_active":"2024-04-25T10:30:34.793865+0000","last_peered":"2024-04-25T10:30:34.793865+0000","last_clean":"2024-04-25T10:30:34.793865+0000","last_became_active":"2024-04-25T10:30:21.427958+0000","last_became_peered":"2024-04-25T10:30:21.427958+0000","last_unstale":"2024-04-25T10:30:34.793865+0000","last_undegraded":"2024-04-25T10:30:34.793865+0000","last_fullsized":"2024-04-25T10:30:34.793865+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:20.186206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:20.186206+0000","last_clean_scrub_stamp":"2024-04-25T10:30:20.186206+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:46.046488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.793946+0000","last_change":"2024-04-25T10:30:31.404974+0000","last_active":"2024-04-25T10:30:34.793946+0000","last_peered":"2024-04-25T10:30:34.793946+0000","last_clean":"2024-04-25T10:30:34.793946+0000","last_became_active":"2024-04-25T10:30:31.404549+0000","last_became_peered":"2024-04-25T10:30:31.404549+0000","last_unstale":"2024-04-25T10:30:34.793946+0000","last_undegraded":"2024-04-25T10:30:34.793946+0000","last_fullsized":"2024-04-25T10:30:34.793946+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:06.191321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415412+0000","last_change":"2024-04-25T10:30:32.434409+0000","last_active":"2024-04-25T10:30:33.415412+0000","last_peered":"2024-04-25T10:30:33.415412+0000","last_clean":"2024-04-25T10:30:33.415412+0000","last_became_active":"2024-04-25T10:30:32.433830+0000","last_became_peered":"2024-04-25T10:30:32.433830+0000","last_unstale":"2024-04-25T10:30:33.415412+0000","last_undegraded":"2024-04-25T10:30:33.415412+0000","last_fullsized":"2024-04-25T10:30:33.415412+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:57.864627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.793793+0000","last_change":"2024-04-25T10:30:31.416025+0000","last_active":"2024-04-25T10:30:34.793793+0000","last_peered":"2024-04-25T10:30:34.793793+0000","last_clean":"2024-04-25T10:30:34.793793+0000","last_became_active":"2024-04-25T10:30:31.415675+0000","last_became_peered":"2024-04-25T10:30:31.415675+0000","last_unstale":"2024-04-25T10:30:34.793793+0000","last_undegraded":"2024-04-25T10:30:34.793793+0000","last_fullsized":"2024-04-25T10:30:34.793793+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:28:48.185715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.196505+0000","last_change":"2024-04-25T10:30:32.424630+0000","last_active":"2024-04-25T10:30:34.196505+0000","last_peered":"2024-04-25T10:30:34.196505+0000","last_clean":"2024-04-25T10:30:34.196505+0000","last_became_active":"2024-04-25T10:30:32.424303+0000","last_became_peered":"2024-04-25T10:30:32.424303+0000","last_unstale":"2024-04-25T10:30:34.196505+0000","last_undegraded":"2024-04-25T10:30:34.196505+0000","last_fullsized":"2024-04-25T10:30:34.196505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:55.535319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415284+0000","last_change":"2024-04-25T10:30:31.419145+0000","last_active":"2024-04-25T10:30:33.415284+0000","last_peered":"2024-04-25T10:30:33.415284+0000","last_clean":"2024-04-25T10:30:33.415284+0000","last_became_active":"2024-04-25T10:30:31.418817+0000","last_became_peered":"2024-04-25T10:30:31.418817+0000","last_unstale":"2024-04-25T10:30:33.415284+0000","last_undegraded":"2024-04-25T10:30:33.415284+0000","last_fullsized":"2024-04-25T10:30:33.415284+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:01.608205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.412646+0000","last_change":"2024-04-25T10:30:32.419911+0000","last_active":"2024-04-25T10:30:33.412646+0000","last_peered":"2024-04-25T10:30:33.412646+0000","last_clean":"2024-04-25T10:30:33.412646+0000","last_became_active":"2024-04-25T10:30:32.419486+0000","last_became_peered":"2024-04-25T10:30:32.419486+0000","last_unstale":"2024-04-25T10:30:33.412646+0000","last_undegraded":"2024-04-25T10:30:33.412646+0000","last_fullsized":"2024-04-25T10:30:33.412646+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:54.967873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.796060+0000","last_change":"2024-04-25T10:30:31.405646+0000","last_active":"2024-04-25T10:30:34.796060+0000","last_peered":"2024-04-25T10:30:34.796060+0000","last_clean":"2024-04-25T10:30:34.796060+0000","last_became_active":"2024-04-25T10:30:31.405238+0000","last_became_peered":"2024-04-25T10:30:31.405238+0000","last_unstale":"2024-04-25T10:30:34.796060+0000","last_undegraded":"2024-04-25T10:30:34.796060+0000","last_fullsized":"2024-04-25T10:30:34.796060+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:13.159787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795730+0000","last_change":"2024-04-25T10:30:32.426130+0000","last_active":"2024-04-25T10:30:34.795730+0000","last_peered":"2024-04-25T10:30:34.795730+0000","last_clean":"2024-04-25T10:30:34.795730+0000","last_became_active":"2024-04-25T10:30:32.425783+0000","last_became_peered":"2024-04-25T10:30:32.425783+0000","last_unstale":"2024-04-25T10:30:34.795730+0000","last_undegraded":"2024-04-25T10:30:34.795730+0000","last_fullsized":"2024-04-25T10:30:34.795730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:30.844640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.412747+0000","last_change":"2024-04-25T10:30:31.412736+0000","last_active":"2024-04-25T10:30:33.412747+0000","last_peered":"2024-04-25T10:30:33.412747+0000","last_clean":"2024-04-25T10:30:33.412747+0000","last_became_active":"2024-04-25T10:30:31.412413+0000","last_became_peered":"2024-04-25T10:30:31.412413+0000","last_unstale":"2024-04-25T10:30:33.412747+0000","last_undegraded":"2024-04-25T10:30:33.412747+0000","last_fullsized":"2024-04-25T10:30:33.412747+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:19:56.236327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795688+0000","last_change":"2024-04-25T10:30:32.440858+0000","last_active":"2024-04-25T10:30:34.795688+0000","last_peered":"2024-04-25T10:30:34.795688+0000","last_clean":"2024-04-25T10:30:34.795688+0000","last_became_active":"2024-04-25T10:30:32.440519+0000","last_became_peered":"2024-04-25T10:30:32.440519+0000","last_unstale":"2024-04-25T10:30:34.795688+0000","last_undegraded":"2024-04-25T10:30:34.795688+0000","last_fullsized":"2024-04-25T10:30:34.795688+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:21.288748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.796138+0000","last_change":"2024-04-25T10:30:31.417958+0000","last_active":"2024-04-25T10:30:34.796138+0000","last_peered":"2024-04-25T10:30:34.796138+0000","last_clean":"2024-04-25T10:30:34.796138+0000","last_became_active":"2024-04-25T10:30:31.417541+0000","last_became_peered":"2024-04-25T10:30:31.417541+0000","last_unstale":"2024-04-25T10:30:34.796138+0000","last_undegraded":"2024-04-25T10:30:34.796138+0000","last_fullsized":"2024-04-25T10:30:34.796138+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:35.373351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.412553+0000","last_change":"2024-04-25T10:30:32.420451+0000","last_active":"2024-04-25T10:30:33.412553+0000","last_peered":"2024-04-25T10:30:33.412553+0000","last_clean":"2024-04-25T10:30:33.412553+0000","last_became_active":"2024-04-25T10:30:32.420109+0000","last_became_peered":"2024-04-25T10:30:32.420109+0000","last_unstale":"2024-04-25T10:30:33.412553+0000","last_undegraded":"2024-04-25T10:30:33.412553+0000","last_fullsized":"2024-04-25T10:30:33.412553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:22.067828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.412601+0000","last_change":"2024-04-25T10:30:31.406005+0000","last_active":"2024-04-25T10:30:33.412601+0000","last_peered":"2024-04-25T10:30:33.412601+0000","last_clean":"2024-04-25T10:30:33.412601+0000","last_became_active":"2024-04-25T10:30:31.404739+0000","last_became_peered":"2024-04-25T10:30:31.404739+0000","last_unstale":"2024-04-25T10:30:33.412601+0000","last_undegraded":"2024-04-25T10:30:33.412601+0000","last_fullsized":"2024-04-25T10:30:33.412601+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:31.771123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415711+0000","last_change":"2024-04-25T10:30:32.437513+0000","last_active":"2024-04-25T10:30:33.415711+0000","last_peered":"2024-04-25T10:30:33.415711+0000","last_clean":"2024-04-25T10:30:33.415711+0000","last_became_active":"2024-04-25T10:30:32.437171+0000","last_became_peered":"2024-04-25T10:30:32.437171+0000","last_unstale":"2024-04-25T10:30:33.415711+0000","last_undegraded":"2024-04-25T10:30:33.415711+0000","last_fullsized":"2024-04-25T10:30:33.415711+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:50.924843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794361+0000","last_change":"2024-04-25T10:30:31.413140+0000","last_active":"2024-04-25T10:30:34.794361+0000","last_peered":"2024-04-25T10:30:34.794361+0000","last_clean":"2024-04-25T10:30:34.794361+0000","last_became_active":"2024-04-25T10:30:31.412821+0000","last_became_peered":"2024-04-25T10:30:31.412821+0000","last_unstale":"2024-04-25T10:30:34.794361+0000","last_undegraded":"2024-04-25T10:30:34.794361+0000","last_fullsized":"2024-04-25T10:30:34.794361+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:37.177254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.196799+0000","last_change":"2024-04-25T10:30:32.434333+0000","last_active":"2024-04-25T10:30:34.196799+0000","last_peered":"2024-04-25T10:30:34.196799+0000","last_clean":"2024-04-25T10:30:34.196799+0000","last_became_active":"2024-04-25T10:30:32.434000+0000","last_became_peered":"2024-04-25T10:30:32.434000+0000","last_unstale":"2024-04-25T10:30:34.196799+0000","last_undegraded":"2024-04-25T10:30:34.196799+0000","last_fullsized":"2024-04-25T10:30:34.196799+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:54.057457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794573+0000","last_change":"2024-04-25T10:30:31.419475+0000","last_active":"2024-04-25T10:30:34.794573+0000","last_peered":"2024-04-25T10:30:34.794573+0000","last_clean":"2024-04-25T10:30:34.794573+0000","last_became_active":"2024-04-25T10:30:31.417605+0000","last_became_peered":"2024-04-25T10:30:31.417605+0000","last_unstale":"2024-04-25T10:30:34.794573+0000","last_undegraded":"2024-04-25T10:30:34.794573+0000","last_fullsized":"2024-04-25T10:30:34.794573+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:00.948269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794414+0000","last_change":"2024-04-25T10:30:32.419693+0000","last_active":"2024-04-25T10:30:34.794414+0000","last_peered":"2024-04-25T10:30:34.794414+0000","last_clean":"2024-04-25T10:30:34.794414+0000","last_became_active":"2024-04-25T10:30:32.419370+0000","last_became_peered":"2024-04-25T10:30:32.419370+0000","last_unstale":"2024-04-25T10:30:34.794414+0000","last_undegraded":"2024-04-25T10:30:34.794414+0000","last_fullsized":"2024-04-25T10:30:34.794414+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:31.199065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.412581+0000","last_change":"2024-04-25T10:30:31.410767+0000","last_active":"2024-04-25T10:30:33.412581+0000","last_peered":"2024-04-25T10:30:33.412581+0000","last_clean":"2024-04-25T10:30:33.412581+0000","last_became_active":"2024-04-25T10:30:31.410404+0000","last_became_peered":"2024-04-25T10:30:31.410404+0000","last_unstale":"2024-04-25T10:30:33.412581+0000","last_undegraded":"2024-04-25T10:30:33.412581+0000","last_fullsized":"2024-04-25T10:30:33.412581+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:37.431532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415585+0000","last_change":"2024-04-25T10:30:32.433687+0000","last_active":"2024-04-25T10:30:33.415585+0000","last_peered":"2024-04-25T10:30:33.415585+0000","last_clean":"2024-04-25T10:30:33.415585+0000","last_became_active":"2024-04-25T10:30:32.433389+0000","last_became_peered":"2024-04-25T10:30:32.433389+0000","last_unstale":"2024-04-25T10:30:33.415585+0000","last_undegraded":"2024-04-25T10:30:33.415585+0000","last_fullsized":"2024-04-25T10:30:33.415585+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:39.155317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.412506+0000","last_change":"2024-04-25T10:30:31.419118+0000","last_active":"2024-04-25T10:30:33.412506+0000","last_peered":"2024-04-25T10:30:33.412506+0000","last_clean":"2024-04-25T10:30:33.412506+0000","last_became_active":"2024-04-25T10:30:31.418822+0000","last_became_peered":"2024-04-25T10:30:31.418822+0000","last_unstale":"2024-04-25T10:30:33.412506+0000","last_undegraded":"2024-04-25T10:30:33.412506+0000","last_fullsized":"2024-04-25T10:30:33.412506+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:22.755042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415890+0000","last_change":"2024-04-25T10:30:32.427894+0000","last_active":"2024-04-25T10:30:33.415890+0000","last_peered":"2024-04-25T10:30:33.415890+0000","last_clean":"2024-04-25T10:30:33.415890+0000","last_became_active":"2024-04-25T10:30:32.427504+0000","last_became_peered":"2024-04-25T10:30:32.427504+0000","last_unstale":"2024-04-25T10:30:33.415890+0000","last_undegraded":"2024-04-25T10:30:33.415890+0000","last_fullsized":"2024-04-25T10:30:33.415890+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:53.877484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794986+0000","last_change":"2024-04-25T10:30:31.417115+0000","last_active":"2024-04-25T10:30:34.794986+0000","last_peered":"2024-04-25T10:30:34.794986+0000","last_clean":"2024-04-25T10:30:34.794986+0000","last_became_active":"2024-04-25T10:30:31.416766+0000","last_became_peered":"2024-04-25T10:30:31.416766+0000","last_unstale":"2024-04-25T10:30:34.794986+0000","last_undegraded":"2024-04-25T10:30:34.794986+0000","last_fullsized":"2024-04-25T10:30:34.794986+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:06.021270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795407+0000","last_change":"2024-04-25T10:30:32.431674+0000","last_active":"2024-04-25T10:30:34.795407+0000","last_peered":"2024-04-25T10:30:34.795407+0000","last_clean":"2024-04-25T10:30:34.795407+0000","last_became_active":"2024-04-25T10:30:32.431395+0000","last_became_peered":"2024-04-25T10:30:32.431395+0000","last_unstale":"2024-04-25T10:30:34.795407+0000","last_undegraded":"2024-04-25T10:30:34.795407+0000","last_fullsized":"2024-04-25T10:30:34.795407+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:49.726323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795508+0000","last_change":"2024-04-25T10:30:32.437700+0000","last_active":"2024-04-25T10:30:34.795508+0000","last_peered":"2024-04-25T10:30:34.795508+0000","last_clean":"2024-04-25T10:30:34.795508+0000","last_became_active":"2024-04-25T10:30:32.437242+0000","last_became_peered":"2024-04-25T10:30:32.437242+0000","last_unstale":"2024-04-25T10:30:34.795508+0000","last_undegraded":"2024-04-25T10:30:34.795508+0000","last_fullsized":"2024-04-25T10:30:34.795508+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:06.499414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795739+0000","last_change":"2024-04-25T10:30:31.419678+0000","last_active":"2024-04-25T10:30:34.795739+0000","last_peered":"2024-04-25T10:30:34.795739+0000","last_clean":"2024-04-25T10:30:34.795739+0000","last_became_active":"2024-04-25T10:30:31.419269+0000","last_became_peered":"2024-04-25T10:30:31.419269+0000","last_unstale":"2024-04-25T10:30:34.795739+0000","last_undegraded":"2024-04-25T10:30:34.795739+0000","last_fullsized":"2024-04-25T10:30:34.795739+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:33.034260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795138+0000","last_change":"2024-04-25T10:30:32.434410+0000","last_active":"2024-04-25T10:30:34.795138+0000","last_peered":"2024-04-25T10:30:34.795138+0000","last_clean":"2024-04-25T10:30:34.795138+0000","last_became_active":"2024-04-25T10:30:32.434093+0000","last_became_peered":"2024-04-25T10:30:32.434093+0000","last_unstale":"2024-04-25T10:30:34.795138+0000","last_undegraded":"2024-04-25T10:30:34.795138+0000","last_fullsized":"2024-04-25T10:30:34.795138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:46.964704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795981+0000","last_change":"2024-04-25T10:30:31.418360+0000","last_active":"2024-04-25T10:30:34.795981+0000","last_peered":"2024-04-25T10:30:34.795981+0000","last_clean":"2024-04-25T10:30:34.795981+0000","last_became_active":"2024-04-25T10:30:31.418005+0000","last_became_peered":"2024-04-25T10:30:31.418005+0000","last_unstale":"2024-04-25T10:30:34.795981+0000","last_undegraded":"2024-04-25T10:30:34.795981+0000","last_fullsized":"2024-04-25T10:30:34.795981+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:15.556859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.196705+0000","last_change":"2024-04-25T10:30:32.439515+0000","last_active":"2024-04-25T10:30:34.196705+0000","last_peered":"2024-04-25T10:30:34.196705+0000","last_clean":"2024-04-25T10:30:34.196705+0000","last_became_active":"2024-04-25T10:30:32.439101+0000","last_became_peered":"2024-04-25T10:30:32.439101+0000","last_unstale":"2024-04-25T10:30:34.196705+0000","last_undegraded":"2024-04-25T10:30:34.196705+0000","last_fullsized":"2024-04-25T10:30:34.196705+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:33.797903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415821+0000","last_change":"2024-04-25T10:30:31.418070+0000","last_active":"2024-04-25T10:30:33.415821+0000","last_peered":"2024-04-25T10:30:33.415821+0000","last_clean":"2024-04-25T10:30:33.415821+0000","last_became_active":"2024-04-25T10:30:31.417737+0000","last_became_peered":"2024-04-25T10:30:31.417737+0000","last_unstale":"2024-04-25T10:30:33.415821+0000","last_undegraded":"2024-04-25T10:30:33.415821+0000","last_fullsized":"2024-04-25T10:30:33.415821+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:05.664855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415785+0000","last_change":"2024-04-25T10:30:32.435258+0000","last_active":"2024-04-25T10:30:33.415785+0000","last_peered":"2024-04-25T10:30:33.415785+0000","last_clean":"2024-04-25T10:30:33.415785+0000","last_became_active":"2024-04-25T10:30:32.434930+0000","last_became_peered":"2024-04-25T10:30:32.434930+0000","last_unstale":"2024-04-25T10:30:33.415785+0000","last_undegraded":"2024-04-25T10:30:33.415785+0000","last_fullsized":"2024-04-25T10:30:33.415785+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:22.727778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.196803+0000","last_change":"2024-04-25T10:30:31.415611+0000","last_active":"2024-04-25T10:30:34.196803+0000","last_peered":"2024-04-25T10:30:34.196803+0000","last_clean":"2024-04-25T10:30:34.196803+0000","last_became_active":"2024-04-25T10:30:31.415071+0000","last_became_peered":"2024-04-25T10:30:31.415071+0000","last_unstale":"2024-04-25T10:30:34.196803+0000","last_undegraded":"2024-04-25T10:30:34.196803+0000","last_fullsized":"2024-04-25T10:30:34.196803+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:30:54.077864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":786432,"data_stored":655002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":192512,"data_stored":64634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":192512,"data_stored":64634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":192512,"data_stored":64634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27800,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344040,"statfs":{"total":96636764160,"available":96608296960,"internally_reserved":0,"allocated":798720,"data_stored":661302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:30:37.798 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:30:37.798 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T10:30:37.798 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-25T10:30:37.798 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-25T10:30:37.798 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-25T10:30:37.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.939+0000 7f7823eaf640 1 Processor -- start 2024-04-25T10:30:37.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.939+0000 7f7823eaf640 1 -- start start 2024-04-25T10:30:37.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.940+0000 7f7823eaf640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f781c0cbff0 0x7f781c0cb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:37.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.940+0000 7f7823eaf640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f781c156ec0 0x7f781c0c61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:37.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.940+0000 7f7823eaf640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f781c0c66e0 0x7f781c0cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:37.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.940+0000 7f7823eaf640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f781c161270 con 0x7f781c0c66e0 2024-04-25T10:30:37.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.940+0000 7f7823eaf640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f781c15dae0 con 0x7f781c156ec0 2024-04-25T10:30:37.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.940+0000 7f7823eaf640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f781c1604b0 con 0x7f781c0cbff0 2024-04-25T10:30:37.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.940+0000 7f782244c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f781c0c66e0 0x7f781c0cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:37.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.941+0000 7f782244c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f781c0c66e0 0x7f781c0cce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47510/0 (socket says 172.21.15.106:47510) 2024-04-25T10:30:37.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.941+0000 7f782244c640 1 -- 172.21.15.106:0/3718464917 learned_addr learned my addr 172.21.15.106:0/3718464917 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:37.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.941+0000 7f782144a640 1 --2- 172.21.15.106:0/3718464917 >> v2:172.21.15.134:3300/0 conn(0x7f781c156ec0 0x7f781c0c61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:37.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.941+0000 7f7821c4b640 1 --2- 172.21.15.106:0/3718464917 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f781c0cbff0 0x7f781c0cb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:37.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.941+0000 7f782244c640 1 -- 172.21.15.106:0/3718464917 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f781c0cbff0 msgr2=0x7f781c0cb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:37.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.941+0000 7f782244c640 1 --2- 172.21.15.106:0/3718464917 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f781c0cbff0 0x7f781c0cb270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:37.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.941+0000 7f782244c640 1 -- 172.21.15.106:0/3718464917 >> v2:172.21.15.134:3300/0 conn(0x7f781c156ec0 msgr2=0x7f781c0c61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:37.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.941+0000 7f782244c640 1 --2- 172.21.15.106:0/3718464917 >> v2:172.21.15.134:3300/0 conn(0x7f781c156ec0 0x7f781c0c61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:37.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.941+0000 7f782244c640 1 -- 172.21.15.106:0/3718464917 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f781c160790 con 0x7f781c0c66e0 2024-04-25T10:30:37.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.942+0000 7f782244c640 1 --2- 172.21.15.106:0/3718464917 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f781c0c66e0 0x7f781c0cce00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f781c0cd8f0 tx=0x7f781402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6025116ec2a7d77 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:37.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.942+0000 7f7820c49640 1 -- 172.21.15.106:0/3718464917 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7814036070 con 0x7f781c0c66e0 2024-04-25T10:30:37.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.942+0000 7f7820c49640 1 -- 172.21.15.106:0/3718464917 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7814031600 con 0x7f781c0c66e0 2024-04-25T10:30:37.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.942+0000 7f7820c49640 1 -- 172.21.15.106:0/3718464917 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78140319d0 con 0x7f781c0c66e0 2024-04-25T10:30:37.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.942+0000 7f7823eaf640 1 -- 172.21.15.106:0/3718464917 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f781c0c66e0 msgr2=0x7f781c0cce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:37.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.942+0000 7f7823eaf640 1 --2- 172.21.15.106:0/3718464917 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f781c0c66e0 0x7f781c0cce00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f781c0cd8f0 tx=0x7f781402a730 comp rx=0 tx=0).stop 2024-04-25T10:30:37.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.942+0000 7f7823eaf640 1 -- 172.21.15.106:0/3718464917 shutdown_connections 2024-04-25T10:30:37.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.942+0000 7f7823eaf640 1 --2- 172.21.15.106:0/3718464917 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f781c0c66e0 0x7f781c0cce00 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:37.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.942+0000 7f7823eaf640 1 --2- 172.21.15.106:0/3718464917 >> v2:172.21.15.134:3300/0 conn(0x7f781c156ec0 0x7f781c0c61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:37.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.942+0000 7f7823eaf640 1 --2- 172.21.15.106:0/3718464917 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f781c0cbff0 0x7f781c0cb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:37.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.942+0000 7f7823eaf640 1 -- 172.21.15.106:0/3718464917 >> 172.21.15.106:0/3718464917 conn(0x7f781c0727d0 msgr2=0x7f781c072c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:37.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.942+0000 7f7823eaf640 1 -- 172.21.15.106:0/3718464917 shutdown_connections 2024-04-25T10:30:37.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.942+0000 7f7823eaf640 1 -- 172.21.15.106:0/3718464917 wait complete. 2024-04-25T10:30:37.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.944+0000 7f7823eaf640 1 Processor -- start 2024-04-25T10:30:37.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.944+0000 7f7823eaf640 1 -- start start 2024-04-25T10:30:37.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.945+0000 7f7823eaf640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f781c0c66e0 0x7f781c1598c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:37.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.945+0000 7f7823eaf640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f781c156ec0 0x7f781c14e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:37.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.945+0000 7f7823eaf640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f781c14edc0 0x7f781c0cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:37.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.945+0000 7f7823eaf640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f781c1572d0 con 0x7f781c14edc0 2024-04-25T10:30:37.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.945+0000 7f7823eaf640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f781c160d50 con 0x7f781c0c66e0 2024-04-25T10:30:37.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.945+0000 7f7823eaf640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f781c15d730 con 0x7f781c156ec0 2024-04-25T10:30:37.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.945+0000 7f782244c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f781c14edc0 0x7f781c0cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:37.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.945+0000 7f782244c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f781c14edc0 0x7f781c0cdf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47524/0 (socket says 172.21.15.106:47524) 2024-04-25T10:30:37.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.945+0000 7f782244c640 1 -- 172.21.15.106:0/1105504088 learned_addr learned my addr 172.21.15.106:0/1105504088 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:37.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.946+0000 7f7821c4b640 1 --2- 172.21.15.106:0/1105504088 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f781c0c66e0 0x7f781c1598c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:37.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.946+0000 7f782144a640 1 --2- 172.21.15.106:0/1105504088 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f781c156ec0 0x7f781c14e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:37.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.946+0000 7f782244c640 1 -- 172.21.15.106:0/1105504088 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f781c156ec0 msgr2=0x7f781c14e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:37.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.946+0000 7f782244c640 1 --2- 172.21.15.106:0/1105504088 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f781c156ec0 0x7f781c14e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:37.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.946+0000 7f782244c640 1 -- 172.21.15.106:0/1105504088 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f781c0c66e0 msgr2=0x7f781c1598c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:37.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.946+0000 7f782244c640 1 --2- 172.21.15.106:0/1105504088 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f781c0c66e0 0x7f781c1598c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:37.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.946+0000 7f782244c640 1 -- 172.21.15.106:0/1105504088 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f781c159e00 con 0x7f781c14edc0 2024-04-25T10:30:37.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.946+0000 7f782244c640 1 --2- 172.21.15.106:0/1105504088 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f781c14edc0 0x7f781c0cdf40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7814009f40 tx=0x7f7814004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:37.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.946+0000 7f78127fc640 1 -- 172.21.15.106:0/1105504088 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7814036050 con 0x7f781c14edc0 2024-04-25T10:30:37.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.946+0000 7f78127fc640 1 -- 172.21.15.106:0/1105504088 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7814042070 con 0x7f781c14edc0 2024-04-25T10:30:37.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.946+0000 7f7823eaf640 1 -- 172.21.15.106:0/1105504088 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f781c1f3510 con 0x7f781c14edc0 2024-04-25T10:30:37.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.946+0000 7f78127fc640 1 -- 172.21.15.106:0/1105504088 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7814009510 con 0x7f781c14edc0 2024-04-25T10:30:37.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.946+0000 7f7823eaf640 1 -- 172.21.15.106:0/1105504088 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f781c1f37b0 con 0x7f781c14edc0 2024-04-25T10:30:37.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.947+0000 7f7823eaf640 1 -- 172.21.15.106:0/1105504088 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f781c159fd0 con 0x7f781c14edc0 2024-04-25T10:30:37.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.948+0000 7f78127fc640 1 -- 172.21.15.106:0/1105504088 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7814007bf0 con 0x7f781c14edc0 2024-04-25T10:30:37.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.948+0000 7f78127fc640 1 --2- 172.21.15.106:0/1105504088 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f78000744b0 0x7f78000769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:37.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.948+0000 7f7821c4b640 1 --2- 172.21.15.106:0/1105504088 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f78000744b0 0x7f78000769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:37.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.948+0000 7f78127fc640 1 -- 172.21.15.106:0/1105504088 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f78140b5e40 con 0x7f781c14edc0 2024-04-25T10:30:37.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.949+0000 7f7821c4b640 1 --2- 172.21.15.106:0/1105504088 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f78000744b0 0x7f78000769c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7818009aa0 tx=0x7f7818009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:37.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:37.953+0000 7f78127fc640 1 -- 172.21.15.106:0/1105504088 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f781407ff80 con 0x7f781c14edc0 2024-04-25T10:30:38.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.175+0000 7f7823eaf640 1 -- 172.21.15.106:0/1105504088 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f781c06d8d0 con 0x7f781c14edc0 2024-04-25T10:30:38.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.178+0000 7f78127fc640 1 -- 172.21.15.106:0/1105504088 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+186750 (secure 0 0 0) 0x7f7814084280 con 0x7f781c14edc0 2024-04-25T10:30:38.183 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:30:38.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.180+0000 7f7823eaf640 1 -- 172.21.15.106:0/1105504088 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f78000744b0 msgr2=0x7f78000769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.180+0000 7f7823eaf640 1 --2- 172.21.15.106:0/1105504088 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f78000744b0 0x7f78000769c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7818009aa0 tx=0x7f7818009190 comp rx=0 tx=0).stop 2024-04-25T10:30:38.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.180+0000 7f7823eaf640 1 -- 172.21.15.106:0/1105504088 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f781c14edc0 msgr2=0x7f781c0cdf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.180+0000 7f7823eaf640 1 --2- 172.21.15.106:0/1105504088 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f781c14edc0 0x7f781c0cdf40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7814009f40 tx=0x7f7814004a40 comp rx=0 tx=0).stop 2024-04-25T10:30:38.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.180+0000 7f7823eaf640 1 -- 172.21.15.106:0/1105504088 shutdown_connections 2024-04-25T10:30:38.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.180+0000 7f7823eaf640 1 --2- 172.21.15.106:0/1105504088 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f78000744b0 0x7f78000769c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.180+0000 7f7823eaf640 1 --2- 172.21.15.106:0/1105504088 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f781c14edc0 0x7f781c0cdf40 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.180+0000 7f7823eaf640 1 --2- 172.21.15.106:0/1105504088 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f781c156ec0 0x7f781c14e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.180+0000 7f7823eaf640 1 --2- 172.21.15.106:0/1105504088 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f781c0c66e0 0x7f781c1598c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.181+0000 7f7823eaf640 1 -- 172.21.15.106:0/1105504088 >> 172.21.15.106:0/1105504088 conn(0x7f781c0727d0 msgr2=0x7f781c0ccca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:38.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.181+0000 7f7823eaf640 1 -- 172.21.15.106:0/1105504088 shutdown_connections 2024-04-25T10:30:38.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.181+0000 7f7823eaf640 1 -- 172.21.15.106:0/1105504088 wait complete. 2024-04-25T10:30:38.194 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":5,"flags":0,"active_gid":4107,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":1744210167},{"type":"v1","addr":"172.21.15.106:6833","nonce":1744210167}]},"active_addr":"172.21.15.106:6833/1744210167","active_change":"2024-04-25T10:30:12.086809+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4100,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.106:0","nonce":2225385427}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.106:0","nonce":1902829372}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.106:0","nonce":3855488972}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.106:0","nonce":3326812853}]}]} 2024-04-25T10:30:38.197 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-25T10:30:38.197 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-25T10:30:38.198 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:30:38.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.303+0000 7f7621870640 1 Processor -- start 2024-04-25T10:30:38.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.303+0000 7f7621870640 1 -- start start 2024-04-25T10:30:38.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.304+0000 7f7621870640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f761c0c7e30 0x7f761c0c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.304+0000 7f7621870640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f761c156ef0 0x7f761c0b4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.304+0000 7f7621870640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f761c0ce630 con 0x7f761c06d000 2024-04-25T10:30:38.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.304+0000 7f7621870640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f761c0c7c60 con 0x7f761c0c7e30 2024-04-25T10:30:38.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.304+0000 7f7621870640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f761c0c7980 con 0x7f761c156ef0 2024-04-25T10:30:38.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.305+0000 7f761affd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f761c0c7e30 0x7f761c0c8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.305+0000 7f761affd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f761c0c7e30 0x7f761c0c8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47536/0 (socket says 172.21.15.106:47536) 2024-04-25T10:30:38.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.305+0000 7f761affd640 1 -- 172.21.15.106:0/1122821046 learned_addr learned my addr 172.21.15.106:0/1122821046 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:38.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.305+0000 7f761a7fc640 1 --2- 172.21.15.106:0/1122821046 >> v2:172.21.15.134:3300/0 conn(0x7f761c156ef0 0x7f761c0b4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.305+0000 7f761affd640 1 -- 172.21.15.106:0/1122821046 >> v2:172.21.15.134:3300/0 conn(0x7f761c156ef0 msgr2=0x7f761c0b4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:38.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.305+0000 7f761affd640 1 --2- 172.21.15.106:0/1122821046 >> v2:172.21.15.134:3300/0 conn(0x7f761c156ef0 0x7f761c0b4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.305+0000 7f761affd640 1 -- 172.21.15.106:0/1122821046 >> v1:172.21.15.134:6789/0 conn(0x7f761c06d000 legacy=0x7f761c15ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:38.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.305+0000 7f761affd640 1 -- 172.21.15.106:0/1122821046 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f761c0c7010 con 0x7f761c0c7e30 2024-04-25T10:30:38.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.306+0000 7f761affd640 1 --2- 172.21.15.106:0/1122821046 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f761c0c7e30 0x7f761c0c8d90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f761c0c7de0 tx=0x7f760c02a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7451df1704d50135 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.306+0000 7f7619ffb640 1 -- 172.21.15.106:0/1122821046 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f760c036070 con 0x7f761c0c7e30 2024-04-25T10:30:38.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.306+0000 7f7619ffb640 1 -- 172.21.15.106:0/1122821046 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f760c031600 con 0x7f761c0c7e30 2024-04-25T10:30:38.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.306+0000 7f7619ffb640 1 -- 172.21.15.106:0/1122821046 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f760c0319d0 con 0x7f761c0c7e30 2024-04-25T10:30:38.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.306+0000 7f7621870640 1 -- 172.21.15.106:0/1122821046 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f761c0c7e30 msgr2=0x7f761c0c8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.306+0000 7f7621870640 1 --2- 172.21.15.106:0/1122821046 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f761c0c7e30 0x7f761c0c8d90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f761c0c7de0 tx=0x7f760c02a640 comp rx=0 tx=0).stop 2024-04-25T10:30:38.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.306+0000 7f7621870640 1 -- 172.21.15.106:0/1122821046 shutdown_connections 2024-04-25T10:30:38.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.306+0000 7f7621870640 1 --2- 172.21.15.106:0/1122821046 >> v2:172.21.15.134:3300/0 conn(0x7f761c156ef0 0x7f761c0b4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.306+0000 7f7621870640 1 --2- 172.21.15.106:0/1122821046 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f761c0c7e30 0x7f761c0c8d90 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.306+0000 7f7621870640 1 -- 172.21.15.106:0/1122821046 >> 172.21.15.106:0/1122821046 conn(0x7f761c068b10 msgr2=0x7f761c0d1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:38.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.306+0000 7f7621870640 1 -- 172.21.15.106:0/1122821046 shutdown_connections 2024-04-25T10:30:38.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.307+0000 7f7621870640 1 -- 172.21.15.106:0/1122821046 wait complete. 2024-04-25T10:30:38.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.308+0000 7f7621870640 1 Processor -- start 2024-04-25T10:30:38.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.308+0000 7f7621870640 1 -- start start 2024-04-25T10:30:38.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.309+0000 7f7621870640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f761c06d000 0x7f761c1510e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.309+0000 7f7621870640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f761c0c7e30 0x7f761c1f4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.309+0000 7f7621870640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f761c156ef0 0x7f761c1e96d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.310+0000 7f7621870640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f761c15d6c0 con 0x7f761c06d000 2024-04-25T10:30:38.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.310+0000 7f7621870640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f761c15d540 con 0x7f761c0c7e30 2024-04-25T10:30:38.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.310+0000 7f7621870640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f761c15c7d0 con 0x7f761c156ef0 2024-04-25T10:30:38.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.310+0000 7f761affd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f761c06d000 0x7f761c1510e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.310+0000 7f761affd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f761c06d000 0x7f761c1510e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47544/0 (socket says 172.21.15.106:47544) 2024-04-25T10:30:38.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.310+0000 7f761affd640 1 -- 172.21.15.106:0/1264994045 learned_addr learned my addr 172.21.15.106:0/1264994045 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:38.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.310+0000 7f761affd640 1 -- 172.21.15.106:0/1264994045 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f761c156ef0 msgr2=0x7f761c1e96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.310+0000 7f761b7fe640 1 --2- 172.21.15.106:0/1264994045 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f761c156ef0 0x7f761c1e96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.310+0000 7f761a7fc640 1 --2- 172.21.15.106:0/1264994045 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f761c0c7e30 0x7f761c1f4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.310+0000 7f761affd640 1 --2- 172.21.15.106:0/1264994045 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f761c156ef0 0x7f761c1e96d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.310+0000 7f761affd640 1 -- 172.21.15.106:0/1264994045 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f761c0c7e30 msgr2=0x7f761c1f4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.310+0000 7f761affd640 1 --2- 172.21.15.106:0/1264994045 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f761c0c7e30 0x7f761c1f4000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.310+0000 7f761affd640 1 -- 172.21.15.106:0/1264994045 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f761c151620 con 0x7f761c06d000 2024-04-25T10:30:38.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.310+0000 7f761affd640 1 --2- 172.21.15.106:0/1264994045 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f761c06d000 0x7f761c1510e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f760c009f10 tx=0x7f760c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.310+0000 7f7613fff640 1 -- 172.21.15.106:0/1264994045 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f760c036050 con 0x7f761c06d000 2024-04-25T10:30:38.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.310+0000 7f7613fff640 1 -- 172.21.15.106:0/1264994045 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f760c042070 con 0x7f761c06d000 2024-04-25T10:30:38.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.310+0000 7f7621870640 1 -- 172.21.15.106:0/1264994045 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f761c1f36c0 con 0x7f761c06d000 2024-04-25T10:30:38.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.310+0000 7f7613fff640 1 -- 172.21.15.106:0/1264994045 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f760c009510 con 0x7f761c06d000 2024-04-25T10:30:38.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.310+0000 7f7621870640 1 -- 172.21.15.106:0/1264994045 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f761c15b1f0 con 0x7f761c06d000 2024-04-25T10:30:38.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.311+0000 7f7621870640 1 -- 172.21.15.106:0/1264994045 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f761c06a3d0 con 0x7f761c06d000 2024-04-25T10:30:38.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.312+0000 7f7613fff640 1 -- 172.21.15.106:0/1264994045 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f760c007c50 con 0x7f761c06d000 2024-04-25T10:30:38.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.312+0000 7f7613fff640 1 --2- 172.21.15.106:0/1264994045 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f75fc0744e0 0x7f75fc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.312+0000 7f761a7fc640 1 --2- 172.21.15.106:0/1264994045 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f75fc0744e0 0x7f75fc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.312+0000 7f7613fff640 1 -- 172.21.15.106:0/1264994045 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f760c0b6070 con 0x7f761c06d000 2024-04-25T10:30:38.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.313+0000 7f761a7fc640 1 --2- 172.21.15.106:0/1264994045 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f75fc0744e0 0x7f75fc0769f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f761c0cdb00 tx=0x7f7604008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.319 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.317+0000 7f7613fff640 1 -- 172.21.15.106:0/1264994045 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f760c02ae20 con 0x7f761c06d000 2024-04-25T10:30:38.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.496+0000 7f7621870640 1 -- 172.21.15.106:0/1264994045 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f761c076430 con 0x7f761c06d000 2024-04-25T10:30:38.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.497+0000 7f7613fff640 1 -- 172.21.15.106:0/1264994045 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16641 (secure 0 0 0) 0x7f760c080920 con 0x7f761c06d000 2024-04-25T10:30:38.500 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:30:38.500 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":20,"fsid":"3ee77985-e8e4-41cf-81e9-7c774cf7891f","created":"2024-04-25T10:30:07.981054+0000","modified":"2024-04-25T10:30:33.409267+0000","last_up_change":"2024-04-25T10:30:19.184325+0000","last_in_change":"2024-04-25T10:30:10.217553+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:30:20.111046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:30:30.314914+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:30:30.715500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a5962cb1-dbbd-47b1-8d15-9bae114bfdf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1240717079},{"type":"v1","addr":"172.21.15.106:6801","nonce":1240717079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1240717079},{"type":"v1","addr":"172.21.15.106:6803","nonce":1240717079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1240717079},{"type":"v1","addr":"172.21.15.106:6807","nonce":1240717079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1240717079},{"type":"v1","addr":"172.21.15.106:6805","nonce":1240717079}]},"public_addr":"172.21.15.106:6801/1240717079","cluster_addr":"172.21.15.106:6803/1240717079","heartbeat_back_addr":"172.21.15.106:6807/1240717079","heartbeat_front_addr":"172.21.15.106:6805/1240717079","state":["exists","up"]},{"osd":1,"uuid":"94436e13-4769-4752-8397-b5e396a6f7ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4089653051},{"type":"v1","addr":"172.21.15.106:6825","nonce":4089653051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4089653051},{"type":"v1","addr":"172.21.15.106:6827","nonce":4089653051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4089653051},{"type":"v1","addr":"172.21.15.106:6831","nonce":4089653051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4089653051},{"type":"v1","addr":"172.21.15.106:6829","nonce":4089653051}]},"public_addr":"172.21.15.106:6825/4089653051","cluster_addr":"172.21.15.106:6827/4089653051","heartbeat_back_addr":"172.21.15.106:6831/4089653051","heartbeat_front_addr":"172.21.15.106:6829/4089653051","state":["exists","up"]},{"osd":2,"uuid":"429aae0b-70c6-4e7b-9043-6a11334c7efc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":1441018239},{"type":"v1","addr":"172.21.15.106:6809","nonce":1441018239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1441018239},{"type":"v1","addr":"172.21.15.106:6811","nonce":1441018239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1441018239},{"type":"v1","addr":"172.21.15.106:6815","nonce":1441018239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":1441018239},{"type":"v1","addr":"172.21.15.106:6813","nonce":1441018239}]},"public_addr":"172.21.15.106:6809/1441018239","cluster_addr":"172.21.15.106:6811/1441018239","heartbeat_back_addr":"172.21.15.106:6815/1441018239","heartbeat_front_addr":"172.21.15.106:6813/1441018239","state":["exists","up"]},{"osd":3,"uuid":"9593a4d1-772f-4667-ab0d-2982f841e342","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":1846968711},{"type":"v1","addr":"172.21.15.106:6817","nonce":1846968711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1846968711},{"type":"v1","addr":"172.21.15.106:6819","nonce":1846968711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":1846968711},{"type":"v1","addr":"172.21.15.106:6823","nonce":1846968711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":1846968711},{"type":"v1","addr":"172.21.15.106:6821","nonce":1846968711}]},"public_addr":"172.21.15.106:6817/1846968711","cluster_addr":"172.21.15.106:6819/1846968711","heartbeat_back_addr":"172.21.15.106:6823/1846968711","heartbeat_front_addr":"172.21.15.106:6821/1846968711","state":["exists","up"]},{"osd":4,"uuid":"1b61f16d-5c60-480a-b449-c745aa3f6b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":729224739},{"type":"v1","addr":"172.21.15.134:6825","nonce":729224739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":729224739},{"type":"v1","addr":"172.21.15.134:6827","nonce":729224739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":729224739},{"type":"v1","addr":"172.21.15.134:6831","nonce":729224739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":729224739},{"type":"v1","addr":"172.21.15.134:6829","nonce":729224739}]},"public_addr":"172.21.15.134:6825/729224739","cluster_addr":"172.21.15.134:6827/729224739","heartbeat_back_addr":"172.21.15.134:6831/729224739","heartbeat_front_addr":"172.21.15.134:6829/729224739","state":["exists","up"]},{"osd":5,"uuid":"944cdd7f-47c0-4b71-b7e0-d093542f4528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3132275652},{"type":"v1","addr":"172.21.15.134:6809","nonce":3132275652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3132275652},{"type":"v1","addr":"172.21.15.134:6811","nonce":3132275652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3132275652},{"type":"v1","addr":"172.21.15.134:6815","nonce":3132275652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3132275652},{"type":"v1","addr":"172.21.15.134:6813","nonce":3132275652}]},"public_addr":"172.21.15.134:6809/3132275652","cluster_addr":"172.21.15.134:6811/3132275652","heartbeat_back_addr":"172.21.15.134:6815/3132275652","heartbeat_front_addr":"172.21.15.134:6813/3132275652","state":["exists","up"]},{"osd":6,"uuid":"cc2446e2-1d21-4135-ac04-6af7c990058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":992702668},{"type":"v1","addr":"172.21.15.134:6801","nonce":992702668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":992702668},{"type":"v1","addr":"172.21.15.134:6803","nonce":992702668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":992702668},{"type":"v1","addr":"172.21.15.134:6807","nonce":992702668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":992702668},{"type":"v1","addr":"172.21.15.134:6805","nonce":992702668}]},"public_addr":"172.21.15.134:6801/992702668","cluster_addr":"172.21.15.134:6803/992702668","heartbeat_back_addr":"172.21.15.134:6807/992702668","heartbeat_front_addr":"172.21.15.134:6805/992702668","state":["exists","up"]},{"osd":7,"uuid":"3df1fc7a-82fc-4257-8629-98027b6e2aed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":3291062284},{"type":"v1","addr":"172.21.15.134:6817","nonce":3291062284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":3291062284},{"type":"v1","addr":"172.21.15.134:6819","nonce":3291062284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":3291062284},{"type":"v1","addr":"172.21.15.134:6823","nonce":3291062284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":3291062284},{"type":"v1","addr":"172.21.15.134:6821","nonce":3291062284}]},"public_addr":"172.21.15.134:6817/3291062284","cluster_addr":"172.21.15.134:6819/3291062284","heartbeat_back_addr":"172.21.15.134:6823/3291062284","heartbeat_front_addr":"172.21.15.134:6821/3291062284","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.403941+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.382563+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.411544+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.330444+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.405430+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.387737+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.408922+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.454450+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:30:38.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.499+0000 7f7621870640 1 -- 172.21.15.106:0/1264994045 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f75fc0744e0 msgr2=0x7f75fc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.499+0000 7f7621870640 1 --2- 172.21.15.106:0/1264994045 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f75fc0744e0 0x7f75fc0769f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f761c0cdb00 tx=0x7f7604008040 comp rx=0 tx=0).stop 2024-04-25T10:30:38.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.499+0000 7f7621870640 1 -- 172.21.15.106:0/1264994045 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f761c06d000 msgr2=0x7f761c1510e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.499+0000 7f7621870640 1 --2- 172.21.15.106:0/1264994045 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f761c06d000 0x7f761c1510e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f760c009f10 tx=0x7f760c004a40 comp rx=0 tx=0).stop 2024-04-25T10:30:38.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.499+0000 7f7621870640 1 -- 172.21.15.106:0/1264994045 shutdown_connections 2024-04-25T10:30:38.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.499+0000 7f7621870640 1 --2- 172.21.15.106:0/1264994045 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f75fc0744e0 0x7f75fc0769f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.499+0000 7f7621870640 1 --2- 172.21.15.106:0/1264994045 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f761c156ef0 0x7f761c1e96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.499+0000 7f7621870640 1 --2- 172.21.15.106:0/1264994045 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f761c0c7e30 0x7f761c1f4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.499+0000 7f7621870640 1 --2- 172.21.15.106:0/1264994045 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f761c06d000 0x7f761c1510e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.499+0000 7f7621870640 1 -- 172.21.15.106:0/1264994045 >> 172.21.15.106:0/1264994045 conn(0x7f761c068b10 msgr2=0x7f761c0cc630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:38.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.499+0000 7f7621870640 1 -- 172.21.15.106:0/1264994045 shutdown_connections 2024-04-25T10:30:38.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.499+0000 7f7621870640 1 -- 172.21.15.106:0/1264994045 wait complete. 2024-04-25T10:30:38.515 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-25T10:30:38.515 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:30:38.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.619+0000 7f5b41127640 1 Processor -- start 2024-04-25T10:30:38.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.619+0000 7f5b41127640 1 -- start start 2024-04-25T10:30:38.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.620+0000 7f5b41127640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f5b3c154f60 0x7f5b3c0cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.620+0000 7f5b41127640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f5b3c0ca8d0 0x7f5b3c0cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.620+0000 7f5b41127640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b3c0cc940 con 0x7f5b3c0cace0 2024-04-25T10:30:38.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.620+0000 7f5b41127640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f5b3c15e190 con 0x7f5b3c154f60 2024-04-25T10:30:38.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.620+0000 7f5b41127640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f5b3c0b3740 con 0x7f5b3c0ca8d0 2024-04-25T10:30:38.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.621+0000 7f5b3ad9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f5b3c154f60 0x7f5b3c0cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.621+0000 7f5b3b59e640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f5b3c0cace0 0x7f5b3c15e620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:44260/0 (socket says 172.21.15.106:44260) 2024-04-25T10:30:38.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.621+0000 7f5b3b59e640 1 -- 172.21.15.106:0/919880122 learned_addr learned my addr 172.21.15.106:0/919880122 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:30:38.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.621+0000 7f5b3a59c640 1 --2- 172.21.15.106:0/919880122 >> v2:172.21.15.134:3300/0 conn(0x7f5b3c0ca8d0 0x7f5b3c0cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.621+0000 7f5b3ad9d640 1 -- 172.21.15.106:0/919880122 >> v2:172.21.15.134:3300/0 conn(0x7f5b3c0ca8d0 msgr2=0x7f5b3c0cbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:38.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.621+0000 7f5b3ad9d640 1 --2- 172.21.15.106:0/919880122 >> v2:172.21.15.134:3300/0 conn(0x7f5b3c0ca8d0 0x7f5b3c0cbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.621+0000 7f5b3ad9d640 1 -- 172.21.15.106:0/919880122 >> v1:172.21.15.106:6789/0 conn(0x7f5b3c0cace0 legacy=0x7f5b3c15e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.621+0000 7f5b3ad9d640 1 -- 172.21.15.106:0/919880122 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b3c0b3a00 con 0x7f5b3c154f60 2024-04-25T10:30:38.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.621+0000 7f5b39d9b640 1 -- 172.21.15.106:0/919880122 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3736667988 0 0) 0x7f5b34001110 con 0x7f5b3c0cace0 2024-04-25T10:30:38.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.622+0000 7f5b3ad9d640 1 --2- 172.21.15.106:0/919880122 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b3c154f60 0x7f5b3c0cb5b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c0cb2b0 tx=0x7f5b2c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ad2e1ba8a7a095a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.622+0000 7f5b39d9b640 1 -- 172.21.15.106:0/919880122 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c007740 con 0x7f5b3c154f60 2024-04-25T10:30:38.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.622+0000 7f5b39d9b640 1 -- 172.21.15.106:0/919880122 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b2c0078e0 con 0x7f5b3c154f60 2024-04-25T10:30:38.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.622+0000 7f5b39d9b640 1 -- 172.21.15.106:0/919880122 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c007cb0 con 0x7f5b3c154f60 2024-04-25T10:30:38.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.622+0000 7f5b41127640 1 -- 172.21.15.106:0/919880122 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b3c154f60 msgr2=0x7f5b3c0cb5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.622+0000 7f5b41127640 1 --2- 172.21.15.106:0/919880122 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b3c154f60 0x7f5b3c0cb5b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c0cb2b0 tx=0x7f5b2c02a710 comp rx=0 tx=0).stop 2024-04-25T10:30:38.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.622+0000 7f5b41127640 1 -- 172.21.15.106:0/919880122 shutdown_connections 2024-04-25T10:30:38.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.622+0000 7f5b41127640 1 --2- 172.21.15.106:0/919880122 >> v2:172.21.15.134:3300/0 conn(0x7f5b3c0ca8d0 0x7f5b3c0cbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.622+0000 7f5b41127640 1 --2- 172.21.15.106:0/919880122 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b3c154f60 0x7f5b3c0cb5b0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.622+0000 7f5b41127640 1 -- 172.21.15.106:0/919880122 >> 172.21.15.106:0/919880122 conn(0x7f5b3c06c5c0 msgr2=0x7f5b3c06c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:38.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.622+0000 7f5b41127640 1 -- 172.21.15.106:0/919880122 shutdown_connections 2024-04-25T10:30:38.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.622+0000 7f5b41127640 1 -- 172.21.15.106:0/919880122 wait complete. 2024-04-25T10:30:38.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.624+0000 7f5b41127640 1 Processor -- start 2024-04-25T10:30:38.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.624+0000 7f5b41127640 1 -- start start 2024-04-25T10:30:38.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.626+0000 7f5b41127640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5b3c0ca8d0 0x7f5b3c1f1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.626+0000 7f5b41127640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b3c0cace0 0x7f5b3c1516e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.626+0000 7f5b41127640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b3c154f60 0x7f5b3c14b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.626+0000 7f5b41127640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3c0b42a0 con 0x7f5b3c0cace0 2024-04-25T10:30:38.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.626+0000 7f5b41127640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3c0b3fc0 con 0x7f5b3c154f60 2024-04-25T10:30:38.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.626+0000 7f5b41127640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f5b3c15ed30 con 0x7f5b3c0ca8d0 2024-04-25T10:30:38.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.626+0000 7f5b3a59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b3c0cace0 0x7f5b3c1516e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.626+0000 7f5b3a59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b3c0cace0 0x7f5b3c1516e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47558/0 (socket says 172.21.15.106:47558) 2024-04-25T10:30:38.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.626+0000 7f5b3a59c640 1 -- 172.21.15.106:0/448848038 learned_addr learned my addr 172.21.15.106:0/448848038 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:38.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.626+0000 7f5b3b59e640 1 --2- 172.21.15.106:0/448848038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b3c154f60 0x7f5b3c14b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.626+0000 7f5b3ad9d640 1 --2- 172.21.15.106:0/448848038 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5b3c0ca8d0 0x7f5b3c1f1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.626+0000 7f5b3a59c640 1 -- 172.21.15.106:0/448848038 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5b3c0ca8d0 msgr2=0x7f5b3c1f1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.626+0000 7f5b3a59c640 1 --2- 172.21.15.106:0/448848038 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5b3c0ca8d0 0x7f5b3c1f1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.626+0000 7f5b3a59c640 1 -- 172.21.15.106:0/448848038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b3c154f60 msgr2=0x7f5b3c14b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.626+0000 7f5b3a59c640 1 --2- 172.21.15.106:0/448848038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b3c154f60 0x7f5b3c14b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.626+0000 7f5b3a59c640 1 -- 172.21.15.106:0/448848038 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b3c1527e0 con 0x7f5b3c0cace0 2024-04-25T10:30:38.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.626+0000 7f5b3a59c640 1 --2- 172.21.15.106:0/448848038 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b3c0cace0 0x7f5b3c1516e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5b3000d5f0 tx=0x7f5b3000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.626+0000 7f5b237fe640 1 -- 172.21.15.106:0/448848038 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b300125f0 con 0x7f5b3c0cace0 2024-04-25T10:30:38.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.626+0000 7f5b237fe640 1 -- 172.21.15.106:0/448848038 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b30019070 con 0x7f5b3c0cace0 2024-04-25T10:30:38.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.626+0000 7f5b41127640 1 -- 172.21.15.106:0/448848038 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b3c152500 con 0x7f5b3c0cace0 2024-04-25T10:30:38.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.626+0000 7f5b237fe640 1 -- 172.21.15.106:0/448848038 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b300129d0 con 0x7f5b3c0cace0 2024-04-25T10:30:38.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.626+0000 7f5b41127640 1 -- 172.21.15.106:0/448848038 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b3c0ca5c0 con 0x7f5b3c0cace0 2024-04-25T10:30:38.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.627+0000 7f5b41127640 1 -- 172.21.15.106:0/448848038 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b3c072c70 con 0x7f5b3c0cace0 2024-04-25T10:30:38.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.628+0000 7f5b237fe640 1 -- 172.21.15.106:0/448848038 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5b30012b70 con 0x7f5b3c0cace0 2024-04-25T10:30:38.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.628+0000 7f5b237fe640 1 --2- 172.21.15.106:0/448848038 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f5b280744b0 0x7f5b280769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.628+0000 7f5b237fe640 1 -- 172.21.15.106:0/448848038 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5b30095530 con 0x7f5b3c0cace0 2024-04-25T10:30:38.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.628+0000 7f5b3ad9d640 1 --2- 172.21.15.106:0/448848038 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f5b280744b0 0x7f5b280769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.629+0000 7f5b3ad9d640 1 --2- 172.21.15.106:0/448848038 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f5b280744b0 0x7f5b280769c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c034000 tx=0x7f5b2c033040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.633+0000 7f5b237fe640 1 -- 172.21.15.106:0/448848038 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5b3c072c70 con 0x7f5b3c0cace0 2024-04-25T10:30:38.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.810+0000 7f5b41127640 1 -- 172.21.15.106:0/448848038 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5b3c06a3d0 con 0x7f5b3c0cace0 2024-04-25T10:30:38.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.811+0000 7f5b237fe640 1 -- 172.21.15.106:0/448848038 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16641 (secure 0 0 0) 0x7f5b3005fde0 con 0x7f5b3c0cace0 2024-04-25T10:30:38.813 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:30:38.814 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":20,"fsid":"3ee77985-e8e4-41cf-81e9-7c774cf7891f","created":"2024-04-25T10:30:07.981054+0000","modified":"2024-04-25T10:30:33.409267+0000","last_up_change":"2024-04-25T10:30:19.184325+0000","last_in_change":"2024-04-25T10:30:10.217553+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:30:20.111046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:30:30.314914+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:30:30.715500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a5962cb1-dbbd-47b1-8d15-9bae114bfdf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1240717079},{"type":"v1","addr":"172.21.15.106:6801","nonce":1240717079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1240717079},{"type":"v1","addr":"172.21.15.106:6803","nonce":1240717079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1240717079},{"type":"v1","addr":"172.21.15.106:6807","nonce":1240717079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1240717079},{"type":"v1","addr":"172.21.15.106:6805","nonce":1240717079}]},"public_addr":"172.21.15.106:6801/1240717079","cluster_addr":"172.21.15.106:6803/1240717079","heartbeat_back_addr":"172.21.15.106:6807/1240717079","heartbeat_front_addr":"172.21.15.106:6805/1240717079","state":["exists","up"]},{"osd":1,"uuid":"94436e13-4769-4752-8397-b5e396a6f7ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4089653051},{"type":"v1","addr":"172.21.15.106:6825","nonce":4089653051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4089653051},{"type":"v1","addr":"172.21.15.106:6827","nonce":4089653051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4089653051},{"type":"v1","addr":"172.21.15.106:6831","nonce":4089653051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4089653051},{"type":"v1","addr":"172.21.15.106:6829","nonce":4089653051}]},"public_addr":"172.21.15.106:6825/4089653051","cluster_addr":"172.21.15.106:6827/4089653051","heartbeat_back_addr":"172.21.15.106:6831/4089653051","heartbeat_front_addr":"172.21.15.106:6829/4089653051","state":["exists","up"]},{"osd":2,"uuid":"429aae0b-70c6-4e7b-9043-6a11334c7efc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":1441018239},{"type":"v1","addr":"172.21.15.106:6809","nonce":1441018239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1441018239},{"type":"v1","addr":"172.21.15.106:6811","nonce":1441018239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1441018239},{"type":"v1","addr":"172.21.15.106:6815","nonce":1441018239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":1441018239},{"type":"v1","addr":"172.21.15.106:6813","nonce":1441018239}]},"public_addr":"172.21.15.106:6809/1441018239","cluster_addr":"172.21.15.106:6811/1441018239","heartbeat_back_addr":"172.21.15.106:6815/1441018239","heartbeat_front_addr":"172.21.15.106:6813/1441018239","state":["exists","up"]},{"osd":3,"uuid":"9593a4d1-772f-4667-ab0d-2982f841e342","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":1846968711},{"type":"v1","addr":"172.21.15.106:6817","nonce":1846968711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1846968711},{"type":"v1","addr":"172.21.15.106:6819","nonce":1846968711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":1846968711},{"type":"v1","addr":"172.21.15.106:6823","nonce":1846968711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":1846968711},{"type":"v1","addr":"172.21.15.106:6821","nonce":1846968711}]},"public_addr":"172.21.15.106:6817/1846968711","cluster_addr":"172.21.15.106:6819/1846968711","heartbeat_back_addr":"172.21.15.106:6823/1846968711","heartbeat_front_addr":"172.21.15.106:6821/1846968711","state":["exists","up"]},{"osd":4,"uuid":"1b61f16d-5c60-480a-b449-c745aa3f6b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":729224739},{"type":"v1","addr":"172.21.15.134:6825","nonce":729224739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":729224739},{"type":"v1","addr":"172.21.15.134:6827","nonce":729224739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":729224739},{"type":"v1","addr":"172.21.15.134:6831","nonce":729224739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":729224739},{"type":"v1","addr":"172.21.15.134:6829","nonce":729224739}]},"public_addr":"172.21.15.134:6825/729224739","cluster_addr":"172.21.15.134:6827/729224739","heartbeat_back_addr":"172.21.15.134:6831/729224739","heartbeat_front_addr":"172.21.15.134:6829/729224739","state":["exists","up"]},{"osd":5,"uuid":"944cdd7f-47c0-4b71-b7e0-d093542f4528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3132275652},{"type":"v1","addr":"172.21.15.134:6809","nonce":3132275652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3132275652},{"type":"v1","addr":"172.21.15.134:6811","nonce":3132275652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3132275652},{"type":"v1","addr":"172.21.15.134:6815","nonce":3132275652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3132275652},{"type":"v1","addr":"172.21.15.134:6813","nonce":3132275652}]},"public_addr":"172.21.15.134:6809/3132275652","cluster_addr":"172.21.15.134:6811/3132275652","heartbeat_back_addr":"172.21.15.134:6815/3132275652","heartbeat_front_addr":"172.21.15.134:6813/3132275652","state":["exists","up"]},{"osd":6,"uuid":"cc2446e2-1d21-4135-ac04-6af7c990058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":992702668},{"type":"v1","addr":"172.21.15.134:6801","nonce":992702668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":992702668},{"type":"v1","addr":"172.21.15.134:6803","nonce":992702668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":992702668},{"type":"v1","addr":"172.21.15.134:6807","nonce":992702668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":992702668},{"type":"v1","addr":"172.21.15.134:6805","nonce":992702668}]},"public_addr":"172.21.15.134:6801/992702668","cluster_addr":"172.21.15.134:6803/992702668","heartbeat_back_addr":"172.21.15.134:6807/992702668","heartbeat_front_addr":"172.21.15.134:6805/992702668","state":["exists","up"]},{"osd":7,"uuid":"3df1fc7a-82fc-4257-8629-98027b6e2aed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":3291062284},{"type":"v1","addr":"172.21.15.134:6817","nonce":3291062284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":3291062284},{"type":"v1","addr":"172.21.15.134:6819","nonce":3291062284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":3291062284},{"type":"v1","addr":"172.21.15.134:6823","nonce":3291062284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":3291062284},{"type":"v1","addr":"172.21.15.134:6821","nonce":3291062284}]},"public_addr":"172.21.15.134:6817/3291062284","cluster_addr":"172.21.15.134:6819/3291062284","heartbeat_back_addr":"172.21.15.134:6823/3291062284","heartbeat_front_addr":"172.21.15.134:6821/3291062284","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.403941+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.382563+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.411544+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.330444+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.405430+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.387737+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.408922+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.454450+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:30:38.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.813+0000 7f5b41127640 1 -- 172.21.15.106:0/448848038 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f5b280744b0 msgr2=0x7f5b280769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.813+0000 7f5b41127640 1 --2- 172.21.15.106:0/448848038 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f5b280744b0 0x7f5b280769c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c034000 tx=0x7f5b2c033040 comp rx=0 tx=0).stop 2024-04-25T10:30:38.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.813+0000 7f5b41127640 1 -- 172.21.15.106:0/448848038 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b3c0cace0 msgr2=0x7f5b3c1516e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.813+0000 7f5b41127640 1 --2- 172.21.15.106:0/448848038 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b3c0cace0 0x7f5b3c1516e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5b3000d5f0 tx=0x7f5b3000dab0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.813+0000 7f5b41127640 1 -- 172.21.15.106:0/448848038 shutdown_connections 2024-04-25T10:30:38.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.813+0000 7f5b41127640 1 --2- 172.21.15.106:0/448848038 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f5b280744b0 0x7f5b280769c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.813+0000 7f5b41127640 1 --2- 172.21.15.106:0/448848038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b3c154f60 0x7f5b3c14b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.813+0000 7f5b41127640 1 --2- 172.21.15.106:0/448848038 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b3c0cace0 0x7f5b3c1516e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.813+0000 7f5b41127640 1 --2- 172.21.15.106:0/448848038 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5b3c0ca8d0 0x7f5b3c1f1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.813+0000 7f5b41127640 1 -- 172.21.15.106:0/448848038 >> 172.21.15.106:0/448848038 conn(0x7f5b3c06c5c0 msgr2=0x7f5b3c15abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:38.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.813+0000 7f5b41127640 1 -- 172.21.15.106:0/448848038 shutdown_connections 2024-04-25T10:30:38.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:38.813+0000 7f5b41127640 1 -- 172.21.15.106:0/448848038 wait complete. 2024-04-25T10:30:38.827 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-25T10:30:38.828 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-25T10:30:38.828 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-25T10:30:38.828 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-25T10:30:38.829 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-25T10:30:38.829 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-25T10:30:38.830 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-25T10:30:38.830 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-25T10:30:39.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.038+0000 7f03e0db2640 1 Processor -- start 2024-04-25T10:30:39.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.038+0000 7f03e0db2640 1 -- start start 2024-04-25T10:30:39.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.039+0000 7f71cfd06640 1 Processor -- start 2024-04-25T10:30:39.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.040+0000 7f71cfd06640 1 -- start start 2024-04-25T10:30:39.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.040+0000 7f587ee30640 1 Processor -- start 2024-04-25T10:30:39.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.040+0000 7f587ee30640 1 -- start start 2024-04-25T10:30:39.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.040+0000 7f352226f640 1 Processor -- start 2024-04-25T10:30:39.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.041+0000 7f352226f640 1 -- start start 2024-04-25T10:30:39.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.041+0000 7fd9ab506640 1 Processor -- start 2024-04-25T10:30:39.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.041+0000 7fb8c9d49640 1 Processor -- start 2024-04-25T10:30:39.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.042+0000 7f03e0db2640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f03dc0c5280 0x7f03dc06ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.042+0000 7f03e0db2640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03dc073c00 con 0x7f03dc0737f0 2024-04-25T10:30:39.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.042+0000 7f71cfd06640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f71c8073870 0x7f71c815b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.041+0000 7fb8c9d49640 1 -- start start 2024-04-25T10:30:39.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.042+0000 7f03e0db2640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03dc073510 con 0x7f03dc073e60 2024-04-25T10:30:39.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.042+0000 7f03e0db2640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f03dc0c4ff0 con 0x7f03dc0c5280 2024-04-25T10:30:39.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.041+0000 7f2f829c5640 1 Processor -- start 2024-04-25T10:30:39.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.041+0000 7f2f829c5640 1 -- start start 2024-04-25T10:30:39.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.042+0000 7f03dad9d640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f03dc073e60 0x7f03dc163660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.106:41164/0 (socket says 172.21.15.106:41164) 2024-04-25T10:30:39.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.042+0000 7f03dad9d640 1 -- 172.21.15.106:0/2484733763 learned_addr learned my addr 172.21.15.106:0/2484733763 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:30:39.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.043+0000 7f03d9d9b640 1 --2- 172.21.15.106:0/2484733763 >> v2:172.21.15.106:3300/0 conn(0x7f03dc0c5280 0x7f03dc06ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.043+0000 7f03d959a640 1 -- 172.21.15.106:0/2484733763 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1706473852 0 0) 0x7f03dc073510 con 0x7f03dc073e60 2024-04-25T10:30:39.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.043+0000 7f03d959a640 1 -- 172.21.15.106:0/2484733763 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f03c4003500 con 0x7f03dc073e60 2024-04-25T10:30:39.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.043+0000 7f71cdaa2640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f71c8073870 0x7f71c815b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.043+0000 7f03d959a640 1 -- 172.21.15.106:0/2484733763 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1655488177 0 0) 0x7f03dc073c00 con 0x7f03dc0737f0 2024-04-25T10:30:39.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.043+0000 7f03d959a640 1 -- 172.21.15.106:0/2484733763 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f03dc073510 con 0x7f03dc0737f0 2024-04-25T10:30:39.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.043+0000 7f587ee30640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f58700d6570 0x7f58700334e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.043+0000 7f587ee30640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f58700dcc50 0x7f5870033a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.043+0000 7f587ee30640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5870047390 con 0x7f5870006420 2024-04-25T10:30:39.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.043+0000 7f587ee30640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f5870047060 con 0x7f58700dcc50 2024-04-25T10:30:39.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.043+0000 7f587ee30640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f5870046b30 con 0x7f58700d6570 2024-04-25T10:30:39.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.043+0000 7f71cdaa2640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f71c8073870 0x7f71c815b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.106:48822/0 (socket says 172.21.15.106:48822) 2024-04-25T10:30:39.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.043+0000 7f71cfd06640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f71c80c7ec0 0x7f71c80c82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.043+0000 7f03d959a640 1 -- 172.21.15.106:0/2484733763 <== mon.1 v1:172.21.15.134:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1200367032 0 0) 0x7f03c4003500 con 0x7f03dc073e60 2024-04-25T10:30:39.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.043+0000 7f03d959a640 1 -- 172.21.15.106:0/2484733763 >> v2:172.21.15.106:3300/0 conn(0x7f03dc0c5280 msgr2=0x7f03dc06ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:39.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.043+0000 7f03d959a640 1 --2- 172.21.15.106:0/2484733763 >> v2:172.21.15.106:3300/0 conn(0x7f03dc0c5280 0x7f03dc06ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.043+0000 7f03d959a640 1 -- 172.21.15.106:0/2484733763 >> v1:172.21.15.106:6789/0 conn(0x7f03dc0737f0 legacy=0x7f03dc152ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.043+0000 7f03d959a640 1 -- 172.21.15.106:0/2484733763 --> v1:172.21.15.134:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03dc072a10 con 0x7f03dc073e60 2024-04-25T10:30:39.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.041+0000 7f0136251640 1 Processor -- start 2024-04-25T10:30:39.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.041+0000 7f0136251640 1 -- start start 2024-04-25T10:30:39.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.043+0000 7f0136251640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f01300c7e30 0x7f01300c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.043+0000 7f0136251640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f0130156ef0 0x7f01300b4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.043+0000 7f0136251640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f013006d000 0x7f013006d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.043+0000 7f587d62d640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f5870006420 0x7f58700dc540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.106:41168/0 (socket says 172.21.15.106:41168) 2024-04-25T10:30:39.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.043+0000 7f71cd2a1640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f71c8165140 0x7f71c80cf030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:44268/0 (socket says 172.21.15.106:44268) 2024-04-25T10:30:39.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.043+0000 7f71cfd06640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71c8073f90 con 0x7f71c8165140 2024-04-25T10:30:39.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.043+0000 7f71cfd06640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f71c8074830 con 0x7f71c80c7ec0 2024-04-25T10:30:39.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.043+0000 7f71cfd06640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f71c80c7a30 con 0x7f71c8073870 2024-04-25T10:30:39.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.043+0000 7f587d62d640 1 -- 172.21.15.106:0/2728513896 learned_addr learned my addr 172.21.15.106:0/2728513896 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:30:39.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.043+0000 7f587e62f640 1 --2- 172.21.15.106:0/2728513896 >> v2:172.21.15.134:3300/0 conn(0x7f58700dcc50 0x7f5870033a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.043+0000 7f03d959a640 1 -- 172.21.15.106:0/2484733763 <== mon.1 v1:172.21.15.134:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3776459350 0 0) 0x7f03d40019f0 con 0x7f03dc073e60 2024-04-25T10:30:39.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.043+0000 7f03d959a640 1 -- 172.21.15.106:0/2484733763 >> v1:172.21.15.134:6789/0 conn(0x7f03dc073e60 legacy=0x7f03dc163660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.043+0000 7f03d959a640 1 --2- 172.21.15.106:0/2484733763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03c4003cd0 0x7f03c40040e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.043+0000 7f03d959a640 1 --2- 172.21.15.106:0/2484733763 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f03c4004730 0x7f03c400b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.043+0000 7f03d959a640 1 --2- 172.21.15.106:0/2484733763 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f03c400b5a0 0x7f03c400da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.043+0000 7f03d959a640 1 -- 172.21.15.106:0/2484733763 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f03c400e290 con 0x7f03c400b5a0 2024-04-25T10:30:39.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.044+0000 7f71cdaa2640 1 -- 172.21.15.106:0/2835032780 learned_addr learned my addr 172.21.15.106:0/2835032780 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:39.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.043+0000 7f03d959a640 1 -- 172.21.15.106:0/2484733763 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f03c400e440 con 0x7f03c4003cd0 2024-04-25T10:30:39.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.043+0000 7f03d959a640 1 -- 172.21.15.106:0/2484733763 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f03c400e640 con 0x7f03c4004730 2024-04-25T10:30:39.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.044+0000 7f03dad9d640 1 --2- 172.21.15.106:0/2484733763 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f03c400b5a0 0x7f03c400da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.044+0000 7f03da59c640 1 --2- 172.21.15.106:0/2484733763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03c4003cd0 0x7f03c40040e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.043+0000 7f587e62f640 1 -- 172.21.15.106:0/2728513896 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f58700d6570 msgr2=0x7f58700334e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:30:39.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.043+0000 7f587de2e640 1 --2- 172.21.15.106:0/2728513896 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f58700d6570 0x7f58700334e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.043+0000 7f587e62f640 1 --2- 172.21.15.106:0/2728513896 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f58700d6570 0x7f58700334e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.043+0000 7f587e62f640 1 -- 172.21.15.106:0/2728513896 >> v1:172.21.15.134:6789/0 conn(0x7f5870006420 legacy=0x7f58700dc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.044+0000 7f587e62f640 1 -- 172.21.15.106:0/2728513896 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5870046d80 con 0x7f58700dcc50 2024-04-25T10:30:39.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.044+0000 7fb8c9d49640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fb8bc009400 0x7fb8bc0d6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.044+0000 7fb8c9d49640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb8bc0df5d0 0x7fb8bc0df9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.044+0000 7fb8c9d49640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8bc046500 con 0x7fb8bc04da10 2024-04-25T10:30:39.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.044+0000 7fb8c9d49640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fb8bc0467c0 con 0x7fb8bc0df5d0 2024-04-25T10:30:39.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.044+0000 7fb8c9d49640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fb8bc04ce50 con 0x7fb8bc009400 2024-04-25T10:30:39.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.041+0000 7fd9ab506640 1 -- start start 2024-04-25T10:30:39.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.043+0000 7fd9ab506640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fd9a415b160 0x7fd9a40c7060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.043+0000 7fd9ab506640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fd9a40c75a0 0x7fd9a416ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.043+0000 7fd9ab506640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9a4161b80 con 0x7fd9a406e9f0 2024-04-25T10:30:39.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.043+0000 7fd9ab506640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fd9a4161e50 con 0x7fd9a40c75a0 2024-04-25T10:30:39.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.043+0000 7fd9ab506640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fd9a40c8460 con 0x7fd9a415b160 2024-04-25T10:30:39.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.043+0000 7fd9a92a2640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fd9a406e9f0 0x7fd9a406d560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.106:41172/0 (socket says 172.21.15.106:41172) 2024-04-25T10:30:39.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.044+0000 7fd9a92a2640 1 -- 172.21.15.106:0/2317744323 learned_addr learned my addr 172.21.15.106:0/2317744323 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:30:39.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.044+0000 7fd9a9aa3640 1 --2- 172.21.15.106:0/2317744323 >> v2:172.21.15.106:3300/0 conn(0x7fd9a40c75a0 0x7fd9a416ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.044+0000 7fd9a9aa3640 1 -- 172.21.15.106:0/2317744323 >> v2:172.21.15.134:3300/0 conn(0x7fd9a415b160 msgr2=0x7fd9a40c7060 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T10:30:39.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.044+0000 7fd9a9aa3640 1 --2- 172.21.15.106:0/2317744323 >> v2:172.21.15.134:3300/0 conn(0x7fd9a415b160 0x7fd9a40c7060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.044+0000 7fd9a9aa3640 1 -- 172.21.15.106:0/2317744323 >> v1:172.21.15.134:6789/0 conn(0x7fd9a406e9f0 legacy=0x7fd9a406d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:39.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.044+0000 7fd9a9aa3640 1 -- 172.21.15.106:0/2317744323 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9a40c8740 con 0x7fd9a40c75a0 2024-04-25T10:30:39.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.044+0000 7fb8c8d47640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fb8bc04da10 0x7fb8bc008cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:44302/0 (socket says 172.21.15.106:44302) 2024-04-25T10:30:39.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.044+0000 7fb8c8d47640 1 -- 172.21.15.106:0/117722550 learned_addr learned my addr 172.21.15.106:0/117722550 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:30:39.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.044+0000 7f352226f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f351404d1c0 0x7f35140478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.044+0000 7f352226f640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35140dc2a0 con 0x7f3514045be0 2024-04-25T10:30:39.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.044+0000 7f352226f640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35140dc580 con 0x7f3514046540 2024-04-25T10:30:39.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.044+0000 7f352226f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f35140339b0 con 0x7f351404d1c0 2024-04-25T10:30:39.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.043+0000 7f0136251640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f01300c7c60 con 0x7f0130156ef0 2024-04-25T10:30:39.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.043+0000 7f0136251640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f01300ce630 con 0x7f01300c7e30 2024-04-25T10:30:39.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.043+0000 7f0136251640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f01300c7980 con 0x7f013006d000 2024-04-25T10:30:39.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.043+0000 7f012ffff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f013006d000 0x7f013006d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.043+0000 7f012ffff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f013006d000 0x7f013006d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.106:48836/0 (socket says 172.21.15.106:48836) 2024-04-25T10:30:39.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.043+0000 7f012ffff640 1 -- 172.21.15.106:0/1795307054 learned_addr learned my addr 172.21.15.106:0/1795307054 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:39.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.044+0000 7f71cdaa2640 1 -- 172.21.15.106:0/2835032780 >> v2:172.21.15.134:3300/0 conn(0x7f71c80c7ec0 msgr2=0x7f71c80c82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:39.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.044+0000 7f71cdaa2640 1 --2- 172.21.15.106:0/2835032780 >> v2:172.21.15.134:3300/0 conn(0x7f71c80c7ec0 0x7f71c80c82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.044+0000 7f71cdaa2640 1 -- 172.21.15.106:0/2835032780 >> v1:172.21.15.106:6789/0 conn(0x7f71c8165140 legacy=0x7f71c80cf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:39.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.044+0000 7f71cdaa2640 1 -- 172.21.15.106:0/2835032780 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71c8074270 con 0x7f71c8073870 2024-04-25T10:30:39.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.044+0000 7f03da59c640 1 -- 172.21.15.106:0/2484733763 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f03c4004730 msgr2=0x7f03c400b010 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T10:30:39.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.044+0000 7f03da59c640 1 --2- 172.21.15.106:0/2484733763 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f03c4004730 0x7f03c400b010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.044+0000 7f03da59c640 1 -- 172.21.15.106:0/2484733763 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f03c400b5a0 msgr2=0x7f03c400da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:39.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.044+0000 7f03da59c640 1 --2- 172.21.15.106:0/2484733763 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f03c400b5a0 0x7f03c400da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.044+0000 7f03da59c640 1 -- 172.21.15.106:0/2484733763 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03c4003500 con 0x7f03c4003cd0 2024-04-25T10:30:39.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.044+0000 7f3520a6c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f351404d1c0 0x7f35140478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.044+0000 7f3520a6c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f351404d1c0 0x7f35140478c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.106:48858/0 (socket says 172.21.15.106:48858) 2024-04-25T10:30:39.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.044+0000 7f3520a6c640 1 -- 172.21.15.106:0/2456126162 learned_addr learned my addr 172.21.15.106:0/2456126162 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:39.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.044+0000 7f352126d640 1 --1- 172.21.15.106:0/2456126162 >> v1:172.21.15.106:6789/0 conn(0x7f3514045be0 0x7f351404c410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:44288/0 (socket says 172.21.15.106:44288) 2024-04-25T10:30:39.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.045+0000 7fb8c9548640 1 --2- 172.21.15.106:0/117722550 >> v2:172.21.15.106:3300/0 conn(0x7fb8bc0df5d0 0x7fb8bc0df9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.044+0000 7f03dad9d640 1 --2- 172.21.15.106:0/2484733763 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f03c400b5a0 0x7f03c400da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:39.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.044+0000 7f03dad9d640 1 -- 172.21.15.106:0/2484733763 reap_dead start 2024-04-25T10:30:39.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.045+0000 7f012effd640 1 --2- 172.21.15.106:0/1795307054 >> v2:172.21.15.106:3300/0 conn(0x7f0130156ef0 0x7f01300b4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.044+0000 7f2f829c5640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f2f74008320 0x7f2f740ddac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.044+0000 7f2f829c5640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2f74007c20 0x7f2f740e4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.045+0000 7f012f7fe640 1 --2- 172.21.15.106:0/1795307054 >> v2:172.21.15.134:3300/0 conn(0x7f01300c7e30 0x7f01300c8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.044+0000 7f2f829c5640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f74046450 con 0x7f2f74046ef0 2024-04-25T10:30:39.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.044+0000 7f2f829c5640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f2f74045be0 con 0x7f2f74007c20 2024-04-25T10:30:39.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.044+0000 7f2f829c5640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f2f74046a60 con 0x7f2f74008320 2024-04-25T10:30:39.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.044+0000 7f2f80f62640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f2f74046ef0 0x7f2f74006c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:44274/0 (socket says 172.21.15.106:44274) 2024-04-25T10:30:39.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.044+0000 7f2f80f62640 1 -- 172.21.15.106:0/651674215 learned_addr learned my addr 172.21.15.106:0/651674215 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:30:39.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.044+0000 7f2f7b7fe640 1 --2- 172.21.15.106:0/651674215 >> v2:172.21.15.106:3300/0 conn(0x7f2f74007c20 0x7f2f740e4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.044+0000 7f2f7bfff640 1 --2- 172.21.15.106:0/651674215 >> v2:172.21.15.134:3300/0 conn(0x7f2f74008320 0x7f2f740ddac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.045+0000 7f2f7b7fe640 1 -- 172.21.15.106:0/651674215 >> v2:172.21.15.134:3300/0 conn(0x7f2f74008320 msgr2=0x7f2f740ddac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:39.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.045+0000 7f2f7b7fe640 1 --2- 172.21.15.106:0/651674215 >> v2:172.21.15.134:3300/0 conn(0x7f2f74008320 0x7f2f740ddac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.045+0000 7f2f7b7fe640 1 -- 172.21.15.106:0/651674215 >> v1:172.21.15.106:6789/0 conn(0x7f2f74046ef0 legacy=0x7f2f74006c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:39.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.045+0000 7f2f7b7fe640 1 -- 172.21.15.106:0/651674215 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f74045e30 con 0x7f2f74007c20 2024-04-25T10:30:39.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.045+0000 7f587e62f640 1 --2- 172.21.15.106:0/2728513896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58700dcc50 0x7f5870033a20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f58740097a0 tx=0x7f5874029fe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4cce43113ff26b4b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.045+0000 7fb8c3fff640 1 --2- 172.21.15.106:0/117722550 >> v2:172.21.15.134:3300/0 conn(0x7fb8bc009400 0x7fb8bc0d6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.045+0000 7f012ffff640 1 -- 172.21.15.106:0/1795307054 >> v2:172.21.15.134:3300/0 conn(0x7f01300c7e30 msgr2=0x7f01300c8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:39.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.045+0000 7f012ffff640 1 --2- 172.21.15.106:0/1795307054 >> v2:172.21.15.134:3300/0 conn(0x7f01300c7e30 0x7f01300c8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.045+0000 7f012ffff640 1 -- 172.21.15.106:0/1795307054 >> v2:172.21.15.106:3300/0 conn(0x7f0130156ef0 msgr2=0x7f01300b4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:39.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.045+0000 7f012ffff640 1 --2- 172.21.15.106:0/1795307054 >> v2:172.21.15.106:3300/0 conn(0x7f0130156ef0 0x7f01300b4ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.045+0000 7f012ffff640 1 -- 172.21.15.106:0/1795307054 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01300c7010 con 0x7f013006d000 2024-04-25T10:30:39.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.045+0000 7fb8c9548640 1 -- 172.21.15.106:0/117722550 >> v2:172.21.15.134:3300/0 conn(0x7fb8bc009400 msgr2=0x7fb8bc0d6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:39.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.045+0000 7fb8c9548640 1 --2- 172.21.15.106:0/117722550 >> v2:172.21.15.134:3300/0 conn(0x7fb8bc009400 0x7fb8bc0d6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.045+0000 7fb8c9548640 1 -- 172.21.15.106:0/117722550 >> v1:172.21.15.106:6789/0 conn(0x7fb8bc04da10 legacy=0x7fb8bc008cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.045+0000 7fb8c9548640 1 -- 172.21.15.106:0/117722550 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8bc04d130 con 0x7fb8bc0df5d0 2024-04-25T10:30:39.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.045+0000 7fd9a9aa3640 1 --2- 172.21.15.106:0/2317744323 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd9a40c75a0 0x7fd9a416ba80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd9a40d2e80 tx=0x7fd99c02a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21d051b8e009aed0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.046+0000 7f587ce2c640 1 -- 172.21.15.106:0/2728513896 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5874037070 con 0x7f58700dcc50 2024-04-25T10:30:39.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.046+0000 7f03da59c640 1 --2- 172.21.15.106:0/2484733763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03c4003cd0 0x7f03c40040e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f03d0001840 tx=0x7f03c4004d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3cc5732c9b0413a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.046+0000 7fd99bfff640 1 -- 172.21.15.106:0/2317744323 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99c007740 con 0x7fd9a40c75a0 2024-04-25T10:30:39.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.046+0000 7fd99bfff640 1 -- 172.21.15.106:0/2317744323 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd99c0078e0 con 0x7fd9a40c75a0 2024-04-25T10:30:39.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.046+0000 7f587ce2c640 1 -- 172.21.15.106:0/2728513896 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f587402ac70 con 0x7f58700dcc50 2024-04-25T10:30:39.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.046+0000 7f03d959a640 1 -- 172.21.15.106:0/2484733763 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d0037070 con 0x7f03c4003cd0 2024-04-25T10:30:39.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.046+0000 7f03d959a640 1 -- 172.21.15.106:0/2484733763 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03d002abc0 con 0x7f03c4003cd0 2024-04-25T10:30:39.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.045+0000 7f3513fff640 1 -- 172.21.15.106:0/2456126162 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2592112598 0 0) 0x7f35140dc580 con 0x7f3514046540 2024-04-25T10:30:39.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.045+0000 7f3513fff640 1 -- 172.21.15.106:0/2456126162 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3508003520 con 0x7f3514046540 2024-04-25T10:30:39.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.045+0000 7f3520a6c640 1 -- 172.21.15.106:0/2456126162 >> v1:172.21.15.134:6789/0 conn(0x7f3514046540 legacy=0x7f3514006420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.045+0000 7f3520a6c640 1 -- 172.21.15.106:0/2456126162 >> v1:172.21.15.106:6789/0 conn(0x7f3514045be0 legacy=0x7f351404c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:39.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.045+0000 7f3520a6c640 1 -- 172.21.15.106:0/2456126162 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3514046de0 con 0x7f351404d1c0 2024-04-25T10:30:39.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.046+0000 7f03d959a640 1 -- 172.21.15.106:0/2484733763 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d0001410 con 0x7f03c4003cd0 2024-04-25T10:30:39.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.046+0000 7fd99bfff640 1 -- 172.21.15.106:0/2317744323 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99c007cb0 con 0x7fd9a40c75a0 2024-04-25T10:30:39.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.046+0000 7f71cdaa2640 1 --2- 172.21.15.106:0/2835032780 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f71c8073870 0x7f71c815b340 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f71c4009870 tx=0x7f71c402a230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50a7e92a3cdb3d78 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.046+0000 7f012ffff640 1 --2- 172.21.15.106:0/1795307054 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f013006d000 0x7f013006d410 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f013008fcb0 tx=0x7f012402a140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=854d454e92cf8878 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.046+0000 7f587ce2c640 1 -- 172.21.15.106:0/2728513896 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5874032660 con 0x7f58700dcc50 2024-04-25T10:30:39.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.046+0000 7f2f7b7fe640 1 --2- 172.21.15.106:0/651674215 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f74007c20 0x7f2f740e4200 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2f7404b280 tx=0x7f2f7002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1af59ce17a722ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.046+0000 7f3520a6c640 1 --2- 172.21.15.106:0/2456126162 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f351404d1c0 0x7f35140478c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f351c06bbc0 tx=0x7f351c0a7f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecc558aa6d6ea5ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.046+0000 7fb8c9548640 1 --2- 172.21.15.106:0/117722550 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8bc0df5d0 0x7fb8bc0df9e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc00e530 tx=0x7fb8b002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ed8a9f7299ce477 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.046+0000 7f71ccaa0640 1 -- 172.21.15.106:0/2835032780 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c4037070 con 0x7f71c8073870 2024-04-25T10:30:39.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.046+0000 7f71ccaa0640 1 -- 172.21.15.106:0/2835032780 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71c4032440 con 0x7f71c8073870 2024-04-25T10:30:39.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.046+0000 7f71ccaa0640 1 -- 172.21.15.106:0/2835032780 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c4032810 con 0x7f71c8073870 2024-04-25T10:30:39.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.046+0000 7f3513fff640 1 -- 172.21.15.106:0/2456126162 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f351c0b5070 con 0x7f351404d1c0 2024-04-25T10:30:39.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.047+0000 7f3513fff640 1 -- 172.21.15.106:0/2456126162 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f351c0a8b50 con 0x7f351404d1c0 2024-04-25T10:30:39.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.047+0000 7f3513fff640 1 -- 172.21.15.106:0/2456126162 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f351c0b0440 con 0x7f351404d1c0 2024-04-25T10:30:39.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.047+0000 7f012e7fc640 1 -- 172.21.15.106:0/1795307054 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0124037070 con 0x7f013006d000 2024-04-25T10:30:39.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.047+0000 7f012e7fc640 1 -- 172.21.15.106:0/1795307054 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0124032440 con 0x7f013006d000 2024-04-25T10:30:39.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.047+0000 7f012e7fc640 1 -- 172.21.15.106:0/1795307054 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0124032810 con 0x7f013006d000 2024-04-25T10:30:39.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.047+0000 7fb8c37fe640 1 -- 172.21.15.106:0/117722550 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b0037070 con 0x7fb8bc0df5d0 2024-04-25T10:30:39.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.046+0000 7f2f7affd640 1 -- 172.21.15.106:0/651674215 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f70037070 con 0x7f2f74007c20 2024-04-25T10:30:39.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.047+0000 7fb8c37fe640 1 -- 172.21.15.106:0/117722550 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8b0032440 con 0x7fb8bc0df5d0 2024-04-25T10:30:39.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.047+0000 7f587ee30640 1 -- 172.21.15.106:0/2728513896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58700dcc50 msgr2=0x7f5870033a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.047+0000 7f587ee30640 1 --2- 172.21.15.106:0/2728513896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58700dcc50 0x7f5870033a20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f58740097a0 tx=0x7f5874029fe0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.047+0000 7fd9ab506640 1 -- 172.21.15.106:0/2317744323 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd9a40c75a0 msgr2=0x7fd9a416ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.047+0000 7fd9ab506640 1 --2- 172.21.15.106:0/2317744323 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd9a40c75a0 0x7fd9a416ba80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd9a40d2e80 tx=0x7fd99c02a5b0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.047+0000 7fd9ab506640 1 -- 172.21.15.106:0/2317744323 shutdown_connections 2024-04-25T10:30:39.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.047+0000 7fd9ab506640 1 --2- 172.21.15.106:0/2317744323 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd9a40c75a0 0x7fd9a416ba80 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.047+0000 7fd9ab506640 1 --2- 172.21.15.106:0/2317744323 >> v2:172.21.15.134:3300/0 conn(0x7fd9a415b160 0x7fd9a40c7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.047+0000 7fd9ab506640 1 -- 172.21.15.106:0/2317744323 >> 172.21.15.106:0/2317744323 conn(0x7fd9a4068ed0 msgr2=0x7fd9a40d20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:39.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.047+0000 7f2f7affd640 1 -- 172.21.15.106:0/651674215 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f7002ae50 con 0x7f2f74007c20 2024-04-25T10:30:39.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.047+0000 7fd9ab506640 1 -- 172.21.15.106:0/2317744323 shutdown_connections 2024-04-25T10:30:39.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.047+0000 7fd9ab506640 1 -- 172.21.15.106:0/2317744323 wait complete. 2024-04-25T10:30:39.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.047+0000 7f03e0db2640 1 -- 172.21.15.106:0/2484733763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03c4003cd0 msgr2=0x7f03c40040e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.047+0000 7f03e0db2640 1 --2- 172.21.15.106:0/2484733763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03c4003cd0 0x7f03c40040e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f03d0001840 tx=0x7f03c4004d50 comp rx=0 tx=0).stop 2024-04-25T10:30:39.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.048+0000 7f71cfd06640 1 -- 172.21.15.106:0/2835032780 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f71c8073870 msgr2=0x7f71c815b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.048+0000 7fb8c37fe640 1 -- 172.21.15.106:0/117722550 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b0032810 con 0x7fb8bc0df5d0 2024-04-25T10:30:39.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.048+0000 7f587ee30640 1 -- 172.21.15.106:0/2728513896 shutdown_connections 2024-04-25T10:30:39.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.048+0000 7f587ee30640 1 --2- 172.21.15.106:0/2728513896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58700dcc50 0x7f5870033a20 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.048+0000 7f587ee30640 1 --2- 172.21.15.106:0/2728513896 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f58700d6570 0x7f58700334e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.048+0000 7f587ee30640 1 -- 172.21.15.106:0/2728513896 >> 172.21.15.106:0/2728513896 conn(0x7f58700503b0 msgr2=0x7f58700507c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:39.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.048+0000 7f2f7affd640 1 -- 172.21.15.106:0/651674215 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f70032660 con 0x7f2f74007c20 2024-04-25T10:30:39.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.048+0000 7f0136251640 1 -- 172.21.15.106:0/1795307054 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f013006d000 msgr2=0x7f013006d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.048+0000 7f0136251640 1 --2- 172.21.15.106:0/1795307054 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f013006d000 0x7f013006d410 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f013008fcb0 tx=0x7f012402a140 comp rx=0 tx=0).stop 2024-04-25T10:30:39.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.048+0000 7f587ee30640 1 -- 172.21.15.106:0/2728513896 shutdown_connections 2024-04-25T10:30:39.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.048+0000 7f587ee30640 1 -- 172.21.15.106:0/2728513896 wait complete. 2024-04-25T10:30:39.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.048+0000 7f03e0db2640 1 -- 172.21.15.106:0/2484733763 shutdown_connections 2024-04-25T10:30:39.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.048+0000 7f03e0db2640 1 --2- 172.21.15.106:0/2484733763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03c4003cd0 0x7f03c40040e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.048+0000 7f71cfd06640 1 --2- 172.21.15.106:0/2835032780 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f71c8073870 0x7f71c815b340 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f71c4009870 tx=0x7f71c402a230 comp rx=0 tx=0).stop 2024-04-25T10:30:39.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.048+0000 7f03e0db2640 1 -- 172.21.15.106:0/2484733763 >> 172.21.15.106:0/2484733763 conn(0x7f03dc06c9b0 msgr2=0x7f03dc06cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:39.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.048+0000 7fb8c9d49640 1 -- 172.21.15.106:0/117722550 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8bc0df5d0 msgr2=0x7fb8bc0df9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.048+0000 7fb8c9d49640 1 --2- 172.21.15.106:0/117722550 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8bc0df5d0 0x7fb8bc0df9e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc00e530 tx=0x7fb8b002a230 comp rx=0 tx=0).stop 2024-04-25T10:30:39.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.048+0000 7f71cfd06640 1 -- 172.21.15.106:0/2835032780 shutdown_connections 2024-04-25T10:30:39.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.048+0000 7f71cfd06640 1 --2- 172.21.15.106:0/2835032780 >> v2:172.21.15.134:3300/0 conn(0x7f71c80c7ec0 0x7f71c80c82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.048+0000 7f71cfd06640 1 --2- 172.21.15.106:0/2835032780 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f71c8073870 0x7f71c815b340 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.048+0000 7f71cfd06640 1 -- 172.21.15.106:0/2835032780 >> 172.21.15.106:0/2835032780 conn(0x7f71c806d8b0 msgr2=0x7f71c806dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:39.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.048+0000 7f03e0db2640 1 -- 172.21.15.106:0/2484733763 shutdown_connections 2024-04-25T10:30:39.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.048+0000 7f2f829c5640 1 -- 172.21.15.106:0/651674215 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f74007c20 msgr2=0x7f2f740e4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.048+0000 7f2f829c5640 1 --2- 172.21.15.106:0/651674215 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f74007c20 0x7f2f740e4200 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2f7404b280 tx=0x7f2f7002a140 comp rx=0 tx=0).stop 2024-04-25T10:30:39.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.048+0000 7f03e0db2640 1 -- 172.21.15.106:0/2484733763 wait complete. 2024-04-25T10:30:39.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.048+0000 7f352226f640 1 -- 172.21.15.106:0/2456126162 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f351404d1c0 msgr2=0x7f35140478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.048+0000 7f352226f640 1 --2- 172.21.15.106:0/2456126162 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f351404d1c0 0x7f35140478c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f351c06bbc0 tx=0x7f351c0a7f90 comp rx=0 tx=0).stop 2024-04-25T10:30:39.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.048+0000 7f2f829c5640 1 -- 172.21.15.106:0/651674215 shutdown_connections 2024-04-25T10:30:39.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.048+0000 7f2f829c5640 1 --2- 172.21.15.106:0/651674215 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f74007c20 0x7f2f740e4200 secure :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2f7404b280 tx=0x7f2f7002a140 comp rx=0 tx=0).stop 2024-04-25T10:30:39.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.048+0000 7f2f829c5640 1 --2- 172.21.15.106:0/651674215 >> v2:172.21.15.134:3300/0 conn(0x7f2f74008320 0x7f2f740ddac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.048+0000 7f2f829c5640 1 -- 172.21.15.106:0/651674215 >> 172.21.15.106:0/651674215 conn(0x7f2f74050c40 msgr2=0x7f2f74051050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:39.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.048+0000 7f0136251640 1 -- 172.21.15.106:0/1795307054 shutdown_connections 2024-04-25T10:30:39.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.048+0000 7f0136251640 1 --2- 172.21.15.106:0/1795307054 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f013006d000 0x7f013006d410 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.048+0000 7f0136251640 1 --2- 172.21.15.106:0/1795307054 >> v2:172.21.15.106:3300/0 conn(0x7f0130156ef0 0x7f01300b4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.048+0000 7f0136251640 1 --2- 172.21.15.106:0/1795307054 >> v2:172.21.15.134:3300/0 conn(0x7f01300c7e30 0x7f01300c8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.048+0000 7f0136251640 1 -- 172.21.15.106:0/1795307054 >> 172.21.15.106:0/1795307054 conn(0x7f0130068b10 msgr2=0x7f01300d1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:39.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.048+0000 7fb8c9d49640 1 -- 172.21.15.106:0/117722550 shutdown_connections 2024-04-25T10:30:39.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.048+0000 7fb8c9d49640 1 --2- 172.21.15.106:0/117722550 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8bc0df5d0 0x7fb8bc0df9e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.048+0000 7fb8c9d49640 1 --2- 172.21.15.106:0/117722550 >> v2:172.21.15.134:3300/0 conn(0x7fb8bc009400 0x7fb8bc0d6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.048+0000 7fb8c9d49640 1 -- 172.21.15.106:0/117722550 >> 172.21.15.106:0/117722550 conn(0x7fb8bc0514c0 msgr2=0x7fb8bc0518d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:39.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.048+0000 7f352226f640 1 -- 172.21.15.106:0/2456126162 shutdown_connections 2024-04-25T10:30:39.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.048+0000 7f352226f640 1 --2- 172.21.15.106:0/2456126162 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f351404d1c0 0x7f35140478c0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.048+0000 7f352226f640 1 -- 172.21.15.106:0/2456126162 >> 172.21.15.106:0/2456126162 conn(0x7f3514050c40 msgr2=0x7f3514051050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:39.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.048+0000 7fb8c9d49640 1 -- 172.21.15.106:0/117722550 shutdown_connections 2024-04-25T10:30:39.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.048+0000 7f71cfd06640 1 -- 172.21.15.106:0/2835032780 shutdown_connections 2024-04-25T10:30:39.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.048+0000 7f352226f640 1 -- 172.21.15.106:0/2456126162 shutdown_connections 2024-04-25T10:30:39.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.048+0000 7fb8c9d49640 1 -- 172.21.15.106:0/117722550 wait complete. 2024-04-25T10:30:39.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.048+0000 7f0136251640 1 -- 172.21.15.106:0/1795307054 shutdown_connections 2024-04-25T10:30:39.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.048+0000 7f0136251640 1 -- 172.21.15.106:0/1795307054 wait complete. 2024-04-25T10:30:39.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.048+0000 7f71cfd06640 1 -- 172.21.15.106:0/2835032780 wait complete. 2024-04-25T10:30:39.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.051+0000 7f71cfd06640 1 Processor -- start 2024-04-25T10:30:39.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.048+0000 7f352226f640 1 -- 172.21.15.106:0/2456126162 wait complete. 2024-04-25T10:30:39.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.051+0000 7f352226f640 1 Processor -- start 2024-04-25T10:30:39.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.051+0000 7f03e0db2640 1 Processor -- start 2024-04-25T10:30:39.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.051+0000 7fb8c9d49640 1 Processor -- start 2024-04-25T10:30:39.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.051+0000 7fb8c9d49640 1 -- start start 2024-04-25T10:30:39.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.051+0000 7f71cfd06640 1 -- start start 2024-04-25T10:30:39.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.052+0000 7f03e0db2640 1 -- start start 2024-04-25T10:30:39.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.050+0000 7fd9ab506640 1 Processor -- start 2024-04-25T10:30:39.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.050+0000 7fd9ab506640 1 -- start start 2024-04-25T10:30:39.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.052+0000 7fd9ab506640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd9a406e9f0 0x7fd9a4200530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.052+0000 7fd9ab506640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd9a40c75a0 0x7fd9a41629f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.052+0000 7fd9ab506640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9a415b160 0x7fd9a4162f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.052+0000 7fd9ab506640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a40c79b0 con 0x7fd9a406e9f0 2024-04-25T10:30:39.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.052+0000 7fd9ab506640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a415b570 con 0x7fd9a415b160 2024-04-25T10:30:39.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.052+0000 7fd9ab506640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fd9a40cde30 con 0x7fd9a40c75a0 2024-04-25T10:30:39.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.051+0000 7f587ee30640 1 Processor -- start 2024-04-25T10:30:39.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.051+0000 7f587ee30640 1 -- start start 2024-04-25T10:30:39.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.053+0000 7f587ee30640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5870006420 0x7f58700dae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.053+0000 7f587ee30640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f58700d6570 0x7f5870185bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.048+0000 7f2f829c5640 1 -- 172.21.15.106:0/651674215 shutdown_connections 2024-04-25T10:30:39.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.048+0000 7f2f829c5640 1 -- 172.21.15.106:0/651674215 wait complete. 2024-04-25T10:30:39.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.051+0000 7f2f829c5640 1 Processor -- start 2024-04-25T10:30:39.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.051+0000 7f2f829c5640 1 -- start start 2024-04-25T10:30:39.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.051+0000 7f352226f640 1 -- start start 2024-04-25T10:30:39.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.053+0000 7f587ee30640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58700dcc50 0x7f58700ebb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.053+0000 7f352226f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3514045be0 0x7f35140e8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.054+0000 7f71cfd06640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71c8073870 0x7f71c815f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.053+0000 7f2f829c5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f74008320 0x7f2f740e1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.054+0000 7f2f829c5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f74046ef0 0x7f2f74193b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.054+0000 7f2f829c5640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2f740e15f0 0x7f2f74005db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.054+0000 7f2f829c5640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2f7404b5c0 con 0x7f2f74046ef0 2024-04-25T10:30:39.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.054+0000 7f2f829c5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2f740466f0 con 0x7f2f74008320 2024-04-25T10:30:39.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.054+0000 7f2f829c5640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f2f740073a0 con 0x7f2f740e15f0 2024-04-25T10:30:39.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.054+0000 7f71cfd06640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f71c80c7ec0 0x7f71c8152540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.053+0000 7f352226f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3514046540 0x7f3514048200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.054+0000 7f352126d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3514045be0 0x7f35140e8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.054+0000 7f352126d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3514045be0 0x7f35140e8670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.106:47868/0 (socket says 172.21.15.106:47868) 2024-04-25T10:30:39.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.054+0000 7f3520a6c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3514046540 0x7f3514048200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.054+0000 7f3520a6c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3514046540 0x7f3514048200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.106:48884/0 (socket says 172.21.15.106:48884) 2024-04-25T10:30:39.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.054+0000 7f71cfd06640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f71c8165140 0x7f71c8153340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.054+0000 7f03e0db2640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f03c4003cd0 0x7f03dc149dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.054+0000 7f2f80f62640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2f740e15f0 0x7f2f74005db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.054+0000 7f2f80f62640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2f740e15f0 0x7f2f74005db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.106:48900/0 (socket says 172.21.15.106:48900) 2024-04-25T10:30:39.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.054+0000 7f2f80f62640 1 -- 172.21.15.106:0/4132734336 learned_addr learned my addr 172.21.15.106:0/4132734336 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:39.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.054+0000 7f352226f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f351404d1c0 0x7f35140bab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.054+0000 7f352226f640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3514047430 con 0x7f351404d1c0 2024-04-25T10:30:39.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.054+0000 7f352226f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3514046b90 con 0x7f3514045be0 2024-04-25T10:30:39.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.054+0000 7f352226f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f3514047e00 con 0x7f3514046540 2024-04-25T10:30:39.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.054+0000 7f352126d640 1 -- 172.21.15.106:0/4176464430 learned_addr learned my addr 172.21.15.106:0/4176464430 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:39.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.053+0000 7f587ee30640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5870045350 con 0x7f58700dcc50 2024-04-25T10:30:39.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.053+0000 7f587ee30640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5870036340 con 0x7f5870006420 2024-04-25T10:30:39.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.053+0000 7f587ee30640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f58700454d0 con 0x7f58700d6570 2024-04-25T10:30:39.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.053+0000 7f587d62d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f58700d6570 0x7f5870185bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.053+0000 7f587d62d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f58700d6570 0x7f5870185bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.106:48874/0 (socket says 172.21.15.106:48874) 2024-04-25T10:30:39.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.053+0000 7f587d62d640 1 -- 172.21.15.106:0/4127959366 learned_addr learned my addr 172.21.15.106:0/4127959366 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:39.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.054+0000 7f71cfd06640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f71c80c8810 con 0x7f71c80c7ec0 2024-04-25T10:30:39.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.054+0000 7f71cfd06640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f71c815b880 con 0x7f71c8073870 2024-04-25T10:30:39.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.054+0000 7f71cfd06640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f71c80c9820 con 0x7f71c8165140 2024-04-25T10:30:39.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.053+0000 7f587de2e640 1 --2- 172.21.15.106:0/4127959366 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5870006420 0x7f58700dae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.054+0000 7f587d62d640 1 -- 172.21.15.106:0/4127959366 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5870006420 msgr2=0x7f58700dae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.054+0000 7f587d62d640 1 --2- 172.21.15.106:0/4127959366 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5870006420 0x7f58700dae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.054+0000 7f587d62d640 1 -- 172.21.15.106:0/4127959366 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58700dcc50 msgr2=0x7f58700ebb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.054+0000 7f587d62d640 1 --2- 172.21.15.106:0/4127959366 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58700dcc50 0x7f58700ebb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.054+0000 7f587d62d640 1 -- 172.21.15.106:0/4127959366 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58700db3c0 con 0x7f58700d6570 2024-04-25T10:30:39.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.054+0000 7f03e0db2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f03dc14a810 0x7f03dc0cd610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.054+0000 7f2f7bfff640 1 --2- 172.21.15.106:0/4132734336 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f74008320 0x7f2f740e1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.054+0000 7fb8c9d49640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb8bc04da10 0x7fb8bc17de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.054+0000 7f2f7b7fe640 1 --2- 172.21.15.106:0/4132734336 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f74046ef0 0x7f2f74193b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.054+0000 7f587d62d640 1 --2- 172.21.15.106:0/4127959366 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f58700d6570 0x7f5870185bc0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5878076d10 tx=0x7f58780783f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.054+0000 7f03e0db2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03dc1440d0 0x7f03dc1444e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.054+0000 7f03e0db2640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f03dc0c8150 con 0x7f03dc14a810 2024-04-25T10:30:39.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.054+0000 7f03e0db2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f03dc0c7c30 con 0x7f03dc1440d0 2024-04-25T10:30:39.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.054+0000 7f03e0db2640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f03dc0c5690 con 0x7f03c4003cd0 2024-04-25T10:30:39.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.051+0000 7f0136251640 1 Processor -- start 2024-04-25T10:30:39.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.051+0000 7f0136251640 1 -- start start 2024-04-25T10:30:39.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.054+0000 7f71cd2a1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f71c80c7ec0 0x7f71c8152540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.054+0000 7f03d9d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f03dc14a810 0x7f03dc0cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.054+0000 7f03da59c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f03c4003cd0 0x7f03dc149dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.053+0000 7f0136251640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f013006d000 0x7f01301f7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.053+0000 7f0136251640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f01300c7e30 0x7f01300d1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.053+0000 7f0136251640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0130156ef0 0x7f0130151f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.054+0000 7f012ffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0130156ef0 0x7f0130151f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.054+0000 7f012ffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0130156ef0 0x7f0130151f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47622/0 (socket says 172.21.15.106:47622) 2024-04-25T10:30:39.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.054+0000 7f012ffff640 1 -- 172.21.15.106:0/3575619058 learned_addr learned my addr 172.21.15.106:0/3575619058 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:39.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.054+0000 7f0136251640 1 -- 172.21.15.106:0/3575619058 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f013015cfb0 con 0x7f0130156ef0 2024-04-25T10:30:39.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.054+0000 7f0136251640 1 -- 172.21.15.106:0/3575619058 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f013015ce30 con 0x7f013006d000 2024-04-25T10:30:39.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.054+0000 7f0136251640 1 -- 172.21.15.106:0/3575619058 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f013015d130 con 0x7f01300c7e30 2024-04-25T10:30:39.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.054+0000 7f2f80f62640 1 -- 172.21.15.106:0/4132734336 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f74008320 msgr2=0x7f2f740e1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.054+0000 7f2f80f62640 1 --2- 172.21.15.106:0/4132734336 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f74008320 0x7f2f740e1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.054+0000 7f2f80f62640 1 -- 172.21.15.106:0/4132734336 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f74046ef0 msgr2=0x7f2f74193b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.054+0000 7fd9a92a2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd9a406e9f0 0x7fd9a4200530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.054+0000 7fd9a92a2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd9a406e9f0 0x7fd9a4200530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47654/0 (socket says 172.21.15.106:47654) 2024-04-25T10:30:39.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.054+0000 7fd9a92a2640 1 -- 172.21.15.106:0/3575079349 learned_addr learned my addr 172.21.15.106:0/3575079349 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:39.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.054+0000 7f3521a6e640 1 --2- 172.21.15.106:0/4176464430 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f351404d1c0 0x7f35140bab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.054+0000 7f03da59c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f03c4003cd0 0x7f03dc149dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.106:48914/0 (socket says 172.21.15.106:48914) 2024-04-25T10:30:39.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.054+0000 7f03da59c640 1 -- 172.21.15.106:0/2132973599 learned_addr learned my addr 172.21.15.106:0/2132973599 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:39.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.054+0000 7f03dad9d640 1 --2- 172.21.15.106:0/2132973599 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03dc1440d0 0x7f03dc1444e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.054+0000 7f71cd2a1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f71c80c7ec0 0x7f71c8152540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47664/0 (socket says 172.21.15.106:47664) 2024-04-25T10:30:39.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.054+0000 7f71cd2a1640 1 -- 172.21.15.106:0/1929653186 learned_addr learned my addr 172.21.15.106:0/1929653186 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:39.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.054+0000 7f03d9d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f03dc14a810 0x7f03dc0cd610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47698/0 (socket says 172.21.15.106:47698) 2024-04-25T10:30:39.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.054+0000 7f012effd640 1 --2- 172.21.15.106:0/3575619058 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f01300c7e30 0x7f01300d1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.054+0000 7f012f7fe640 1 --2- 172.21.15.106:0/3575619058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f013006d000 0x7f01301f7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.054+0000 7f586e7fc640 1 -- 172.21.15.106:0/4127959366 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f587808ec40 con 0x7f58700d6570 2024-04-25T10:30:39.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.054+0000 7f2f80f62640 1 --2- 172.21.15.106:0/4132734336 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f74046ef0 0x7f2f74193b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.054+0000 7f587ee30640 1 -- 172.21.15.106:0/4127959366 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5870186780 con 0x7f58700d6570 2024-04-25T10:30:39.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.055+0000 7f586e7fc640 1 -- 172.21.15.106:0/4127959366 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f587808ede0 con 0x7f58700d6570 2024-04-25T10:30:39.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.054+0000 7f71ce2a3640 1 --2- 172.21.15.106:0/1929653186 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f71c8165140 0x7f71c8153340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.055+0000 7f71cdaa2640 1 --2- 172.21.15.106:0/1929653186 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71c8073870 0x7f71c815f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.055+0000 7fb8c9d49640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb8bc0df5d0 0x7fb8bc0da9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.055+0000 7f587ee30640 1 -- 172.21.15.106:0/4127959366 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f58700dbea0 con 0x7f58700d6570 2024-04-25T10:30:39.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.055+0000 7fb8c9d49640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8bc0daf10 0x7fb8bc17a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.055+0000 7f586e7fc640 1 -- 172.21.15.106:0/4127959366 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f587806c7c0 con 0x7f58700d6570 2024-04-25T10:30:39.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.055+0000 7f03d9d9b640 1 -- 172.21.15.106:0/2132973599 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f03c4003cd0 msgr2=0x7f03dc149dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.055+0000 7f03d9d9b640 1 --2- 172.21.15.106:0/2132973599 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f03c4003cd0 0x7f03dc149dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.055+0000 7f03d9d9b640 1 -- 172.21.15.106:0/2132973599 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03dc1440d0 msgr2=0x7f03dc1444e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.055+0000 7f03d9d9b640 1 --2- 172.21.15.106:0/2132973599 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03dc1440d0 0x7f03dc1444e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.055+0000 7f03d9d9b640 1 -- 172.21.15.106:0/2132973599 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03dc1f2a30 con 0x7f03dc14a810 2024-04-25T10:30:39.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.054+0000 7f2f80f62640 1 -- 172.21.15.106:0/4132734336 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f741940a0 con 0x7f2f740e15f0 2024-04-25T10:30:39.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.055+0000 7f2f7bfff640 1 --2- 172.21.15.106:0/4132734336 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f74008320 0x7f2f740e1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:39.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.055+0000 7f2f7b7fe640 1 --2- 172.21.15.106:0/4132734336 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f74046ef0 0x7f2f74193b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:39.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.055+0000 7f3521a6e640 1 -- 172.21.15.106:0/4176464430 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3514046540 msgr2=0x7f3514048200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.055+0000 7f3521a6e640 1 --2- 172.21.15.106:0/4176464430 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3514046540 0x7f3514048200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.055+0000 7f3521a6e640 1 -- 172.21.15.106:0/4176464430 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3514045be0 msgr2=0x7f35140e8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.055+0000 7f3521a6e640 1 --2- 172.21.15.106:0/4176464430 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3514045be0 0x7f35140e8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.055+0000 7f3521a6e640 1 -- 172.21.15.106:0/4176464430 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35140c6b80 con 0x7f351404d1c0 2024-04-25T10:30:39.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.055+0000 7f71cd2a1640 1 -- 172.21.15.106:0/1929653186 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f71c8165140 msgr2=0x7f71c8153340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.055+0000 7f71cd2a1640 1 --2- 172.21.15.106:0/1929653186 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f71c8165140 0x7f71c8153340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.055+0000 7f71cd2a1640 1 -- 172.21.15.106:0/1929653186 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71c8073870 msgr2=0x7f71c815f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.055+0000 7f71cd2a1640 1 --2- 172.21.15.106:0/1929653186 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71c8073870 0x7f71c815f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.055+0000 7f71cd2a1640 1 -- 172.21.15.106:0/1929653186 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71c8153880 con 0x7f71c80c7ec0 2024-04-25T10:30:39.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.055+0000 7fd9a9aa3640 1 --2- 172.21.15.106:0/3575079349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9a415b160 0x7fd9a4162f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.055+0000 7fd9a8aa1640 1 --2- 172.21.15.106:0/3575079349 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd9a40c75a0 0x7fd9a41629f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.055+0000 7f012f7fe640 1 -- 172.21.15.106:0/3575619058 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f01300c7e30 msgr2=0x7f01300d1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.055+0000 7f012f7fe640 1 --2- 172.21.15.106:0/3575619058 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f01300c7e30 0x7f01300d1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.055+0000 7f012f7fe640 1 -- 172.21.15.106:0/3575619058 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0130156ef0 msgr2=0x7f0130151f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.055+0000 7f012f7fe640 1 --2- 172.21.15.106:0/3575619058 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0130156ef0 0x7f0130151f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.055+0000 7f012f7fe640 1 -- 172.21.15.106:0/3575619058 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01300d00a0 con 0x7f013006d000 2024-04-25T10:30:39.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.055+0000 7fd9a92a2640 1 -- 172.21.15.106:0/3575079349 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd9a40c75a0 msgr2=0x7fd9a41629f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.055+0000 7fd9a92a2640 1 --2- 172.21.15.106:0/3575079349 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd9a40c75a0 0x7fd9a41629f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.055+0000 7fd9a92a2640 1 -- 172.21.15.106:0/3575079349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9a415b160 msgr2=0x7fd9a4162f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.055+0000 7fd9a92a2640 1 --2- 172.21.15.106:0/3575079349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9a415b160 0x7fd9a4162f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.055+0000 7fd9a92a2640 1 -- 172.21.15.106:0/3575079349 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9a415e310 con 0x7fd9a406e9f0 2024-04-25T10:30:39.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.055+0000 7f3521a6e640 1 --2- 172.21.15.106:0/4176464430 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f351404d1c0 0x7f35140bab80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f350c00c410 tx=0x7f350c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.055+0000 7f2f80f62640 1 --2- 172.21.15.106:0/4132734336 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2f740e15f0 0x7f2f74005db0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c06bf60 tx=0x7f2f7c0783f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.055+0000 7f03d9d9b640 1 --2- 172.21.15.106:0/2132973599 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f03dc14a810 0x7f03dc0cd610 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f03cc009b50 tx=0x7f03cc00b510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.055+0000 7f012f7fe640 1 --2- 172.21.15.106:0/3575619058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f013006d000 0x7f01301f7d90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f012000e660 tx=0x7f012000eb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.056+0000 7fd9a92a2640 1 --2- 172.21.15.106:0/3575079349 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd9a406e9f0 0x7fd9a4200530 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd99400b460 tx=0x7fd99400b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.056+0000 7f3511ffb640 1 -- 172.21.15.106:0/4176464430 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f350c017070 con 0x7f351404d1c0 2024-04-25T10:30:39.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.056+0000 7f3511ffb640 1 -- 172.21.15.106:0/4176464430 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f350c0035f0 con 0x7f351404d1c0 2024-04-25T10:30:39.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.056+0000 7f3511ffb640 1 -- 172.21.15.106:0/4176464430 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f350c0039e0 con 0x7f351404d1c0 2024-04-25T10:30:39.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.056+0000 7f352226f640 1 -- 172.21.15.106:0/4176464430 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35140ea870 con 0x7f351404d1c0 2024-04-25T10:30:39.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.056+0000 7fb8c3fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb8bc0df5d0 0x7fb8bc0da9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.056+0000 7fb8c8d47640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb8bc04da10 0x7fb8bc17de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.056+0000 7fb8c3fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb8bc0df5d0 0x7fb8bc0da9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.106:48948/0 (socket says 172.21.15.106:48948) 2024-04-25T10:30:39.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.056+0000 7fb8c8d47640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb8bc04da10 0x7fb8bc17de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.106:47902/0 (socket says 172.21.15.106:47902) 2024-04-25T10:30:39.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.056+0000 7f587ee30640 1 -- 172.21.15.106:0/4127959366 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f58700daa60 con 0x7f58700d6570 2024-04-25T10:30:39.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.056+0000 7fd999ffb640 1 -- 172.21.15.106:0/3575079349 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd994015020 con 0x7fd9a406e9f0 2024-04-25T10:30:39.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.056+0000 7fb8c3fff640 1 -- 172.21.15.106:0/4186972120 learned_addr learned my addr 172.21.15.106:0/4186972120 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:39.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.056+0000 7fb8c9d49640 1 -- 172.21.15.106:0/4186972120 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb8bc034260 con 0x7fb8bc0daf10 2024-04-25T10:30:39.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.056+0000 7fb8c9d49640 1 -- 172.21.15.106:0/4186972120 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb8bc0340e0 con 0x7fb8bc04da10 2024-04-25T10:30:39.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.056+0000 7fb8c9d49640 1 -- 172.21.15.106:0/4186972120 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb8bc04d430 con 0x7fb8bc0df5d0 2024-04-25T10:30:39.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.057+0000 7fb8c3fff640 1 -- 172.21.15.106:0/4186972120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb8bc04da10 msgr2=0x7fb8bc17de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.057+0000 7fb8c3fff640 1 --2- 172.21.15.106:0/4186972120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb8bc04da10 0x7fb8bc17de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.057+0000 7fb8c3fff640 1 -- 172.21.15.106:0/4186972120 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8bc0daf10 msgr2=0x7fb8bc17a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.057+0000 7fb8c3fff640 1 --2- 172.21.15.106:0/4186972120 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8bc0daf10 0x7fb8bc17a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.057+0000 7fb8c3fff640 1 -- 172.21.15.106:0/4186972120 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8bc17e350 con 0x7fb8bc0df5d0 2024-04-25T10:30:39.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.057+0000 7fb8c3fff640 1 --2- 172.21.15.106:0/4186972120 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb8bc0df5d0 0x7fb8bc0da9d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb8c40519b0 tx=0x7fb8c40783f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.056+0000 7f0117fff640 1 -- 172.21.15.106:0/3575619058 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0120017070 con 0x7f013006d000 2024-04-25T10:30:39.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.056+0000 7f0117fff640 1 -- 172.21.15.106:0/3575619058 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f012000c730 con 0x7f013006d000 2024-04-25T10:30:39.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.056+0000 7f0117fff640 1 -- 172.21.15.106:0/3575619058 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012000cae0 con 0x7f013006d000 2024-04-25T10:30:39.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.056+0000 7f0136251640 1 -- 172.21.15.106:0/3575619058 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01300d0380 con 0x7f013006d000 2024-04-25T10:30:39.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.056+0000 7f0136251640 1 -- 172.21.15.106:0/3575619058 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01300d0940 con 0x7f013006d000 2024-04-25T10:30:39.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.057+0000 7f0136251640 1 -- 172.21.15.106:0/3575619058 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0130154ab0 con 0x7f013006d000 2024-04-25T10:30:39.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.056+0000 7f71cd2a1640 1 --2- 172.21.15.106:0/1929653186 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f71c80c7ec0 0x7f71c8152540 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f71bc00b440 tx=0x7f71bc00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.056+0000 7f71ba7fc640 1 -- 172.21.15.106:0/1929653186 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71bc015020 con 0x7f71c80c7ec0 2024-04-25T10:30:39.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.056+0000 7f71ba7fc640 1 -- 172.21.15.106:0/1929653186 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71bc011890 con 0x7f71c80c7ec0 2024-04-25T10:30:39.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.056+0000 7f71ba7fc640 1 -- 172.21.15.106:0/1929653186 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71bc011c80 con 0x7f71c80c7ec0 2024-04-25T10:30:39.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.057+0000 7f03caffd640 1 -- 172.21.15.106:0/2132973599 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03cc01e070 con 0x7f03dc14a810 2024-04-25T10:30:39.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.058+0000 7f03caffd640 1 -- 172.21.15.106:0/2132973599 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03cc00bb40 con 0x7f03dc14a810 2024-04-25T10:30:39.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.056+0000 7fd9ab506640 1 -- 172.21.15.106:0/3575079349 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9a4163c70 con 0x7fd9a406e9f0 2024-04-25T10:30:39.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.056+0000 7fd9ab506640 1 -- 172.21.15.106:0/3575079349 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9a41636b0 con 0x7fd9a406e9f0 2024-04-25T10:30:39.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.056+0000 7f352226f640 1 -- 172.21.15.106:0/4176464430 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35140e8ce0 con 0x7f351404d1c0 2024-04-25T10:30:39.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.056+0000 7f2f78ff9640 1 -- 172.21.15.106:0/4132734336 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f7c07dab0 con 0x7f2f740e15f0 2024-04-25T10:30:39.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.056+0000 7f2f78ff9640 1 -- 172.21.15.106:0/4132734336 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f7c07dc50 con 0x7f2f740e15f0 2024-04-25T10:30:39.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.057+0000 7f2f78ff9640 1 -- 172.21.15.106:0/4132734336 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f7c06c7e0 con 0x7f2f740e15f0 2024-04-25T10:30:39.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.057+0000 7f2f829c5640 1 -- 172.21.15.106:0/4132734336 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f740d5f10 con 0x7f2f740e15f0 2024-04-25T10:30:39.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.057+0000 7f2f829c5640 1 -- 172.21.15.106:0/4132734336 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f740d19d0 con 0x7f2f740e15f0 2024-04-25T10:30:39.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.057+0000 7fd999ffb640 1 -- 172.21.15.106:0/3575079349 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9940118b0 con 0x7fd9a406e9f0 2024-04-25T10:30:39.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.057+0000 7fd999ffb640 1 -- 172.21.15.106:0/3575079349 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd994011ca0 con 0x7fd9a406e9f0 2024-04-25T10:30:39.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.057+0000 7f71cfd06640 1 -- 172.21.15.106:0/1929653186 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71c81616b0 con 0x7f71c80c7ec0 2024-04-25T10:30:39.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.058+0000 7f71cfd06640 1 -- 172.21.15.106:0/1929653186 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71c8161c70 con 0x7f71c80c7ec0 2024-04-25T10:30:39.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.058+0000 7f03caffd640 1 -- 172.21.15.106:0/2132973599 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03cc019410 con 0x7f03dc14a810 2024-04-25T10:30:39.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.058+0000 7f03e0db2640 1 -- 172.21.15.106:0/2132973599 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03dc0cdb50 con 0x7f03dc14a810 2024-04-25T10:30:39.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.058+0000 7f03e0db2640 1 -- 172.21.15.106:0/2132973599 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03dc1f2750 con 0x7f03dc14a810 2024-04-25T10:30:39.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.059+0000 7f586e7fc640 1 -- 172.21.15.106:0/4127959366 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f587806c960 con 0x7f58700d6570 2024-04-25T10:30:39.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.059+0000 7f2f829c5640 1 -- 172.21.15.106:0/4132734336 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2f740059c0 con 0x7f2f740e15f0 2024-04-25T10:30:39.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.059+0000 7f3511ffb640 1 -- 172.21.15.106:0/4176464430 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f350c01a070 con 0x7f351404d1c0 2024-04-25T10:30:39.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.059+0000 7fb8c17fa640 1 -- 172.21.15.106:0/4186972120 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c406db50 con 0x7fb8bc0df5d0 2024-04-25T10:30:39.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.059+0000 7fb8c17fa640 1 -- 172.21.15.106:0/4186972120 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8c406dcf0 con 0x7fb8bc0df5d0 2024-04-25T10:30:39.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.059+0000 7f71abfff640 1 -- 172.21.15.106:0/1929653186 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f71c806a3d0 con 0x7f71c80c7ec0 2024-04-25T10:30:39.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.059+0000 7fb8c9d49640 1 -- 172.21.15.106:0/4186972120 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8bc0de2d0 con 0x7fb8bc0df5d0 2024-04-25T10:30:39.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.059+0000 7f03c8ff9640 1 -- 172.21.15.106:0/2132973599 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f03b4001030 con 0x7f03dc14a810 2024-04-25T10:30:39.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.059+0000 7fb8c17fa640 1 -- 172.21.15.106:0/4186972120 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c408e8b0 con 0x7fb8bc0df5d0 2024-04-25T10:30:39.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.059+0000 7fd9ab506640 1 -- 172.21.15.106:0/3575079349 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd984001030 con 0x7fd9a406e9f0 2024-04-25T10:30:39.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.060+0000 7fb8c9d49640 1 -- 172.21.15.106:0/4186972120 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8bc0da240 con 0x7fb8bc0df5d0 2024-04-25T10:30:39.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.060+0000 7f3511ffb640 1 --2- 172.21.15.106:0/4176464430 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f35000745b0 0x7f3500076ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.060+0000 7f3511ffb640 1 -- 172.21.15.106:0/4176464430 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f350c0943b0 con 0x7f351404d1c0 2024-04-25T10:30:39.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.059+0000 7f0117fff640 1 -- 172.21.15.106:0/3575619058 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f012001a070 con 0x7f013006d000 2024-04-25T10:30:39.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.059+0000 7f0117fff640 1 --2- 172.21.15.106:0/3575619058 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f011c074580 0x7f011c076a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.059+0000 7f0117fff640 1 -- 172.21.15.106:0/3575619058 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f012003ce00 con 0x7f013006d000 2024-04-25T10:30:39.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.061+0000 7fb8c9d49640 1 -- 172.21.15.106:0/4186972120 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb8bc04bb30 con 0x7fb8bc0df5d0 2024-04-25T10:30:39.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.061+0000 7f352126d640 1 --2- 172.21.15.106:0/4176464430 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f35000745b0 0x7f3500076ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.061+0000 7f0117fff640 1 --2- 172.21.15.106:0/3575619058 >> [v2:172.21.15.106:6824/4089653051,v1:172.21.15.106:6825/4089653051] conn(0x7f011c07c850 0x7f011c07eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.061+0000 7f0117fff640 1 -- 172.21.15.106:0/3575619058 --> [v2:172.21.15.106:6824/4089653051,v1:172.21.15.106:6825/4089653051] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0120023e70 con 0x7f011c07c850 2024-04-25T10:30:39.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.061+0000 7f0117fff640 1 -- 172.21.15.106:0/3575619058 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f0120096840 con 0x7f013006d000 2024-04-25T10:30:39.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.061+0000 7f352226f640 1 --2- 172.21.15.106:0/4176464430 >> [v2:172.21.15.134:6800/992702668,v1:172.21.15.134:6801/992702668] conn(0x7f3514007d70 0x7f35140031d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.061+0000 7f352226f640 1 -- 172.21.15.106:0/4176464430 --> [v2:172.21.15.134:6800/992702668,v1:172.21.15.134:6801/992702668] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3514007ba0 con 0x7f3514007d70 2024-04-25T10:30:39.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.061+0000 7f586e7fc640 1 --2- 172.21.15.106:0/4127959366 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f585c0745b0 0x7f585c076ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.061+0000 7f352126d640 1 --2- 172.21.15.106:0/4176464430 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f35000745b0 0x7f3500076ac0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f35180099b0 tx=0x7f3518009110 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.062+0000 7f3520a6c640 1 --2- 172.21.15.106:0/4176464430 >> [v2:172.21.15.134:6800/992702668,v1:172.21.15.134:6801/992702668] conn(0x7f3514007d70 0x7f35140031d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.062+0000 7f587de2e640 1 --2- 172.21.15.106:0/4127959366 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f585c0745b0 0x7f585c076ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.062+0000 7f586e7fc640 1 -- 172.21.15.106:0/4127959366 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f58780b8e00 con 0x7f58700d6570 2024-04-25T10:30:39.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.062+0000 7fb8c17fa640 1 -- 172.21.15.106:0/4186972120 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb8c408ea50 con 0x7fb8bc0df5d0 2024-04-25T10:30:39.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.061+0000 7f012effd640 1 --2- 172.21.15.106:0/3575619058 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f011c074580 0x7f011c076a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.062+0000 7f586e7fc640 1 --2- 172.21.15.106:0/4127959366 >> [v2:172.21.15.106:6808/1441018239,v1:172.21.15.106:6809/1441018239] conn(0x7f585c07c880 0x7f585c07ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.062+0000 7f586e7fc640 1 -- 172.21.15.106:0/4127959366 --> [v2:172.21.15.106:6808/1441018239,v1:172.21.15.106:6809/1441018239] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f585c07f3a0 con 0x7f585c07c880 2024-04-25T10:30:39.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.062+0000 7f586e7fc640 1 -- 172.21.15.106:0/4127959366 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f5878112ab0 con 0x7f58700d6570 2024-04-25T10:30:39.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.062+0000 7f012ffff640 1 --2- 172.21.15.106:0/3575619058 >> [v2:172.21.15.106:6824/4089653051,v1:172.21.15.106:6825/4089653051] conn(0x7f011c07c850 0x7f011c07eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.060+0000 7f2f78ff9640 1 -- 172.21.15.106:0/4132734336 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2f7c06c980 con 0x7f2f740e15f0 2024-04-25T10:30:39.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.061+0000 7f2f78ff9640 1 --2- 172.21.15.106:0/4132734336 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f2f60074580 0x7f2f60076a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.062+0000 7f2f78ff9640 1 -- 172.21.15.106:0/4132734336 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2f7c0a7e00 con 0x7f2f740e15f0 2024-04-25T10:30:39.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.062+0000 7f2f78ff9640 1 --2- 172.21.15.106:0/4132734336 >> [v2:172.21.15.134:6816/3291062284,v1:172.21.15.134:6817/3291062284] conn(0x7f2f6007c850 0x7f2f6007eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.062+0000 7f2f78ff9640 1 -- 172.21.15.106:0/4132734336 --> [v2:172.21.15.134:6816/3291062284,v1:172.21.15.134:6817/3291062284] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2f6007f370 con 0x7f2f6007c850 2024-04-25T10:30:39.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.062+0000 7f2f78ff9640 1 -- 172.21.15.106:0/4132734336 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f2f7c1019d0 con 0x7f2f740e15f0 2024-04-25T10:30:39.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.062+0000 7f2f7bfff640 1 --2- 172.21.15.106:0/4132734336 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f2f60074580 0x7f2f60076a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.062+0000 7f2f7b7fe640 1 --2- 172.21.15.106:0/4132734336 >> [v2:172.21.15.134:6816/3291062284,v1:172.21.15.134:6817/3291062284] conn(0x7f2f6007c850 0x7f2f6007eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.062+0000 7fd999ffb640 1 -- 172.21.15.106:0/3575079349 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd994018020 con 0x7fd9a406e9f0 2024-04-25T10:30:39.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.062+0000 7fd999ffb640 1 --2- 172.21.15.106:0/3575079349 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fd9880745b0 0x7fd988076ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.062+0000 7f03caffd640 1 -- 172.21.15.106:0/2132973599 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f03cc021070 con 0x7f03dc14a810 2024-04-25T10:30:39.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.062+0000 7f71ba7fc640 1 -- 172.21.15.106:0/1929653186 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f71bc018020 con 0x7f71c80c7ec0 2024-04-25T10:30:39.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.062+0000 7fd999ffb640 1 -- 172.21.15.106:0/3575079349 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd994095a00 con 0x7fd9a406e9f0 2024-04-25T10:30:39.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.063+0000 7f2f7b7fe640 1 --2- 172.21.15.106:0/4132734336 >> [v2:172.21.15.134:6816/3291062284,v1:172.21.15.134:6817/3291062284] conn(0x7f2f6007c850 0x7f2f6007eca0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.063+0000 7fd9a8aa1640 1 --2- 172.21.15.106:0/3575079349 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fd9880745b0 0x7fd988076ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.063+0000 7fd999ffb640 1 --2- 172.21.15.106:0/3575079349 >> [v2:172.21.15.134:6808/3132275652,v1:172.21.15.134:6809/3132275652] conn(0x7fd98807c820 0x7fd98807ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.063+0000 7f012ffff640 1 --2- 172.21.15.106:0/3575619058 >> [v2:172.21.15.106:6824/4089653051,v1:172.21.15.106:6825/4089653051] conn(0x7f011c07c850 0x7f011c07eca0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.063+0000 7fd999ffb640 1 -- 172.21.15.106:0/3575079349 --> [v2:172.21.15.134:6808/3132275652,v1:172.21.15.134:6809/3132275652] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd98807f340 con 0x7fd98807c820 2024-04-25T10:30:39.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.063+0000 7fd999ffb640 1 -- 172.21.15.106:0/3575079349 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fd994095df0 con 0x7fd9a406e9f0 2024-04-25T10:30:39.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.062+0000 7fb8c17fa640 1 --2- 172.21.15.106:0/4186972120 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fb8a8074500 0x7fb8a8076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.063+0000 7fb8c17fa640 1 -- 172.21.15.106:0/4186972120 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb8c4111f10 con 0x7fb8bc0df5d0 2024-04-25T10:30:39.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.063+0000 7fb8c17fa640 1 --2- 172.21.15.106:0/4186972120 >> [v2:172.21.15.106:6800/1240717079,v1:172.21.15.106:6801/1240717079] conn(0x7fb8a807c7d0 0x7fb8a807ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.063+0000 7fb8c17fa640 1 -- 172.21.15.106:0/4186972120 --> [v2:172.21.15.106:6800/1240717079,v1:172.21.15.106:6801/1240717079] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb8c4071e50 con 0x7fb8a807c7d0 2024-04-25T10:30:39.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.063+0000 7fb8c17fa640 1 -- 172.21.15.106:0/4186972120 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fb8c4114530 con 0x7fb8bc0df5d0 2024-04-25T10:30:39.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.062+0000 7f587e62f640 1 --2- 172.21.15.106:0/4127959366 >> [v2:172.21.15.106:6808/1441018239,v1:172.21.15.106:6809/1441018239] conn(0x7f585c07c880 0x7f585c07ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.063+0000 7fb8c9548640 1 --2- 172.21.15.106:0/4186972120 >> [v2:172.21.15.106:6800/1240717079,v1:172.21.15.106:6801/1240717079] conn(0x7fb8a807c7d0 0x7fb8a807ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.063+0000 7fd9a9aa3640 1 --2- 172.21.15.106:0/3575079349 >> [v2:172.21.15.134:6808/3132275652,v1:172.21.15.134:6809/3132275652] conn(0x7fd98807c820 0x7fd98807ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.063+0000 7f3520a6c640 1 --2- 172.21.15.106:0/4176464430 >> [v2:172.21.15.134:6800/992702668,v1:172.21.15.134:6801/992702668] conn(0x7f3514007d70 0x7f35140031d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.063+0000 7f03caffd640 1 --2- 172.21.15.106:0/2132973599 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f03bc074580 0x7f03bc076a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.063+0000 7fb8c8d47640 1 --2- 172.21.15.106:0/4186972120 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fb8a8074500 0x7fb8a8076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.063+0000 7f71ba7fc640 1 --2- 172.21.15.106:0/1929653186 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f71ac0745b0 0x7f71ac076ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.063+0000 7f71ba7fc640 1 -- 172.21.15.106:0/1929653186 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f71bc0958d0 con 0x7f71c80c7ec0 2024-04-25T10:30:39.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.063+0000 7f03caffd640 1 -- 172.21.15.106:0/2132973599 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f03cc098c20 con 0x7f03dc14a810 2024-04-25T10:30:39.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.063+0000 7f71ba7fc640 1 --2- 172.21.15.106:0/1929653186 >> [v2:172.21.15.106:6816/1846968711,v1:172.21.15.106:6817/1846968711] conn(0x7f71ac07c880 0x7f71ac07ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.063+0000 7f71ba7fc640 1 -- 172.21.15.106:0/1929653186 --> [v2:172.21.15.106:6816/1846968711,v1:172.21.15.106:6817/1846968711] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f71ac07f3a0 con 0x7f71ac07c880 2024-04-25T10:30:39.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.063+0000 7f71ba7fc640 1 -- 172.21.15.106:0/1929653186 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f71bc095cc0 con 0x7f71c80c7ec0 2024-04-25T10:30:39.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.063+0000 7f03caffd640 1 --2- 172.21.15.106:0/2132973599 >> [v2:172.21.15.134:6824/729224739,v1:172.21.15.134:6825/729224739] conn(0x7f03bc07c850 0x7f03bc07eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.063+0000 7f03caffd640 1 -- 172.21.15.106:0/2132973599 --> [v2:172.21.15.134:6824/729224739,v1:172.21.15.134:6825/729224739] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f03cc024e50 con 0x7f03bc07c850 2024-04-25T10:30:39.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.063+0000 7f03caffd640 1 -- 172.21.15.106:0/2132973599 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f03cc099010 con 0x7f03dc14a810 2024-04-25T10:30:39.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.064+0000 7fd9a9aa3640 1 --2- 172.21.15.106:0/3575079349 >> [v2:172.21.15.134:6808/3132275652,v1:172.21.15.134:6809/3132275652] conn(0x7fd98807c820 0x7fd98807ec70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.064+0000 7f03dad9d640 1 --2- 172.21.15.106:0/2132973599 >> [v2:172.21.15.134:6824/729224739,v1:172.21.15.134:6825/729224739] conn(0x7f03bc07c850 0x7f03bc07eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.064+0000 7fb8c9548640 1 --2- 172.21.15.106:0/4186972120 >> [v2:172.21.15.106:6800/1240717079,v1:172.21.15.106:6801/1240717079] conn(0x7fb8a807c7d0 0x7fb8a807ec20 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.064+0000 7f71ce2a3640 1 --2- 172.21.15.106:0/1929653186 >> [v2:172.21.15.106:6816/1846968711,v1:172.21.15.106:6817/1846968711] conn(0x7f71ac07c880 0x7f71ac07ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.064+0000 7fb8c8d47640 1 --2- 172.21.15.106:0/4186972120 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fb8a8074500 0x7fb8a8076a10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8005880 tx=0x7fb8b8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.064+0000 7f0117fff640 1 -- 172.21.15.106:0/3575619058 <== osd.1 v2:172.21.15.106:6824/4089653051 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0120023e70 con 0x7f011c07c850 2024-04-25T10:30:39.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.064+0000 7f03da59c640 1 --2- 172.21.15.106:0/2132973599 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f03bc074580 0x7f03bc076a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.064+0000 7f2f7bfff640 1 --2- 172.21.15.106:0/4132734336 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f2f60074580 0x7f2f60076a90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2f68009a00 tx=0x7f2f68008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.064+0000 7f71cdaa2640 1 --2- 172.21.15.106:0/1929653186 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f71ac0745b0 0x7f71ac076ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.064+0000 7f2f78ff9640 1 -- 172.21.15.106:0/4132734336 <== osd.7 v2:172.21.15.134:6816/3291062284 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2f6007f370 con 0x7f2f6007c850 2024-04-25T10:30:39.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.064+0000 7f03dad9d640 1 --2- 172.21.15.106:0/2132973599 >> [v2:172.21.15.134:6824/729224739,v1:172.21.15.134:6825/729224739] conn(0x7f03bc07c850 0x7f03bc07eca0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.064+0000 7f012effd640 1 --2- 172.21.15.106:0/3575619058 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f011c074580 0x7f011c076a90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0130156930 tx=0x7f0128007700 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.064+0000 7f3511ffb640 1 -- 172.21.15.106:0/4176464430 <== osd.6 v2:172.21.15.134:6800/992702668 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3514007ba0 con 0x7f3514007d70 2024-04-25T10:30:39.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.064+0000 7f71ce2a3640 1 --2- 172.21.15.106:0/1929653186 >> [v2:172.21.15.106:6816/1846968711,v1:172.21.15.106:6817/1846968711] conn(0x7f71ac07c880 0x7f71ac07ecd0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.065+0000 7fd999ffb640 1 -- 172.21.15.106:0/3575079349 <== osd.5 v2:172.21.15.134:6808/3132275652 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd98807f340 con 0x7fd98807c820 2024-04-25T10:30:39.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.065+0000 7fb8c17fa640 1 -- 172.21.15.106:0/4186972120 <== osd.0 v2:172.21.15.106:6800/1240717079 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb8c4071e50 con 0x7fb8a807c7d0 2024-04-25T10:30:39.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.065+0000 7f03caffd640 1 -- 172.21.15.106:0/2132973599 <== osd.4 v2:172.21.15.134:6824/729224739 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f03cc024e50 con 0x7f03bc07c850 2024-04-25T10:30:39.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.065+0000 7f71cdaa2640 1 --2- 172.21.15.106:0/1929653186 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f71ac0745b0 0x7f71ac076ac0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f71c4034000 tx=0x7f71c402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.065+0000 7f71ba7fc640 1 -- 172.21.15.106:0/1929653186 <== osd.3 v2:172.21.15.106:6816/1846968711 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f71ac07f3a0 con 0x7f71ac07c880 2024-04-25T10:30:39.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.064+0000 7f587e62f640 1 --2- 172.21.15.106:0/4127959366 >> [v2:172.21.15.106:6808/1441018239,v1:172.21.15.106:6809/1441018239] conn(0x7f585c07c880 0x7f585c07ecd0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.064+0000 7f587de2e640 1 --2- 172.21.15.106:0/4127959366 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f585c0745b0 0x7f585c076ac0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5868006fd0 tx=0x7f5868008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.066+0000 7f586e7fc640 1 -- 172.21.15.106:0/4127959366 <== osd.2 v2:172.21.15.106:6808/1441018239 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f585c07f3a0 con 0x7f585c07c880 2024-04-25T10:30:39.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.066+0000 7fd9a8aa1640 1 --2- 172.21.15.106:0/3575079349 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fd9880745b0 0x7fd988076ac0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd9a0006fd0 tx=0x7fd9a0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.066+0000 7f03da59c640 1 --2- 172.21.15.106:0/2132973599 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f03bc074580 0x7f03bc076a90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f03d0001840 tx=0x7f03d002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.096+0000 7f2f829c5640 1 -- 172.21.15.106:0/4132734336 --> [v2:172.21.15.134:6816/3291062284,v1:172.21.15.134:6817/3291062284] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2f740d4680 con 0x7f2f6007c850 2024-04-25T10:30:39.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.096+0000 7f352226f640 1 -- 172.21.15.106:0/4176464430 --> [v2:172.21.15.134:6800/992702668,v1:172.21.15.134:6801/992702668] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f35140e8f40 con 0x7f3514007d70 2024-04-25T10:30:39.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.096+0000 7f2f78ff9640 1 -- 172.21.15.106:0/4132734336 <== osd.7 v2:172.21.15.134:6816/3291062284 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2f740d4680 con 0x7f2f6007c850 2024-04-25T10:30:39.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.097+0000 7f3511ffb640 1 -- 172.21.15.106:0/4176464430 <== osd.6 v2:172.21.15.134:6800/992702668 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f35140e8f40 con 0x7f3514007d70 2024-04-25T10:30:39.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.097+0000 7f0136251640 1 -- 172.21.15.106:0/3575619058 --> [v2:172.21.15.106:6824/4089653051,v1:172.21.15.106:6825/4089653051] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0130154ab0 con 0x7f011c07c850 2024-04-25T10:30:39.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.097+0000 7f2f829c5640 1 -- 172.21.15.106:0/4132734336 >> [v2:172.21.15.134:6816/3291062284,v1:172.21.15.134:6817/3291062284] conn(0x7f2f6007c850 msgr2=0x7f2f6007eca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.097+0000 7f2f829c5640 1 --2- 172.21.15.106:0/4132734336 >> [v2:172.21.15.134:6816/3291062284,v1:172.21.15.134:6817/3291062284] conn(0x7f2f6007c850 0x7f2f6007eca0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.097+0000 7fb8c9d49640 1 -- 172.21.15.106:0/4186972120 --> [v2:172.21.15.106:6800/1240717079,v1:172.21.15.106:6801/1240717079] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb8bc0d42c0 con 0x7fb8a807c7d0 2024-04-25T10:30:39.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.097+0000 7f2f829c5640 1 -- 172.21.15.106:0/4132734336 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f2f60074580 msgr2=0x7f2f60076a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.097+0000 7f0117fff640 1 -- 172.21.15.106:0/3575619058 <== osd.1 v2:172.21.15.106:6824/4089653051 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0130154ab0 con 0x7f011c07c850 2024-04-25T10:30:39.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.097+0000 7f2f829c5640 1 --2- 172.21.15.106:0/4132734336 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f2f60074580 0x7f2f60076a90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2f68009a00 tx=0x7f2f68008040 comp rx=0 tx=0).stop 2024-04-25T10:30:39.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.097+0000 7f352226f640 1 -- 172.21.15.106:0/4176464430 >> [v2:172.21.15.134:6800/992702668,v1:172.21.15.134:6801/992702668] conn(0x7f3514007d70 msgr2=0x7f35140031d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.097+0000 7f352226f640 1 --2- 172.21.15.106:0/4176464430 >> [v2:172.21.15.134:6800/992702668,v1:172.21.15.134:6801/992702668] conn(0x7f3514007d70 0x7f35140031d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.097+0000 7f2f829c5640 1 -- 172.21.15.106:0/4132734336 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2f740e15f0 msgr2=0x7f2f74005db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.097+0000 7f2f829c5640 1 --2- 172.21.15.106:0/4132734336 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2f740e15f0 0x7f2f74005db0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c06bf60 tx=0x7f2f7c0783f0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.097+0000 7f352226f640 1 -- 172.21.15.106:0/4176464430 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f35000745b0 msgr2=0x7f3500076ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.097+0000 7f352226f640 1 --2- 172.21.15.106:0/4176464430 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f35000745b0 0x7f3500076ac0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f35180099b0 tx=0x7f3518009110 comp rx=0 tx=0).stop 2024-04-25T10:30:39.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.097+0000 7f352226f640 1 -- 172.21.15.106:0/4176464430 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f351404d1c0 msgr2=0x7f35140bab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.097+0000 7f352226f640 1 --2- 172.21.15.106:0/4176464430 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f351404d1c0 0x7f35140bab80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f350c00c410 tx=0x7f350c00c8d0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.097+0000 7f3521a6e640 1 -- 172.21.15.106:0/4176464430 reap_dead start 2024-04-25T10:30:39.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.098+0000 7f352226f640 1 -- 172.21.15.106:0/4176464430 shutdown_connections 2024-04-25T10:30:39.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.098+0000 7f352226f640 1 -- 172.21.15.106:0/4176464430 >> 172.21.15.106:0/4176464430 conn(0x7f3514050c40 msgr2=0x7f35140becc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:39.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.098+0000 7f2f80f62640 1 -- 172.21.15.106:0/4132734336 reap_dead start 2024-04-25T10:30:39.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.098+0000 7f2f829c5640 1 -- 172.21.15.106:0/4132734336 shutdown_connections 2024-04-25T10:30:39.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.098+0000 7f2f829c5640 1 -- 172.21.15.106:0/4132734336 >> 172.21.15.106:0/4132734336 conn(0x7f2f74050c40 msgr2=0x7f2f740364e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:39.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.098+0000 7f0115ffb640 1 -- 172.21.15.106:0/3575619058 >> [v2:172.21.15.106:6824/4089653051,v1:172.21.15.106:6825/4089653051] conn(0x7f011c07c850 msgr2=0x7f011c07eca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.098+0000 7f03c8ff9640 1 -- 172.21.15.106:0/2132973599 --> [v2:172.21.15.134:6824/729224739,v1:172.21.15.134:6825/729224739] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f03b4002dd0 con 0x7f03bc07c850 2024-04-25T10:30:39.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.098+0000 7f0115ffb640 1 --2- 172.21.15.106:0/3575619058 >> [v2:172.21.15.106:6824/4089653051,v1:172.21.15.106:6825/4089653051] conn(0x7f011c07c850 0x7f011c07eca0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.098+0000 7f0115ffb640 1 -- 172.21.15.106:0/3575619058 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f011c074580 msgr2=0x7f011c076a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.098+0000 7f0115ffb640 1 --2- 172.21.15.106:0/3575619058 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f011c074580 0x7f011c076a90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0130156930 tx=0x7f0128007700 comp rx=0 tx=0).stop 2024-04-25T10:30:39.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.098+0000 7f0115ffb640 1 -- 172.21.15.106:0/3575619058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f013006d000 msgr2=0x7f01301f7d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.098+0000 7f0115ffb640 1 --2- 172.21.15.106:0/3575619058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f013006d000 0x7f01301f7d90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f012000e660 tx=0x7f012000eb20 comp rx=0 tx=0).stop 2024-04-25T10:30:39.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.098+0000 7f012ffff640 1 -- 172.21.15.106:0/3575619058 reap_dead start 2024-04-25T10:30:39.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.098+0000 7f2f829c5640 1 -- 172.21.15.106:0/4132734336 shutdown_connections 2024-04-25T10:30:39.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.098+0000 7fb8c17fa640 1 -- 172.21.15.106:0/4186972120 <== osd.0 v2:172.21.15.106:6800/1240717079 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb8bc0d42c0 con 0x7fb8a807c7d0 2024-04-25T10:30:39.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.098+0000 7f2f829c5640 1 -- 172.21.15.106:0/4132734336 wait complete. 2024-04-25T10:30:39.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.098+0000 7f03caffd640 1 -- 172.21.15.106:0/2132973599 <== osd.4 v2:172.21.15.134:6824/729224739 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f03b4002dd0 con 0x7f03bc07c850 2024-04-25T10:30:39.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.098+0000 7fd9ab506640 1 -- 172.21.15.106:0/3575079349 --> [v2:172.21.15.134:6808/3132275652,v1:172.21.15.134:6809/3132275652] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd984002dd0 con 0x7fd98807c820 2024-04-25T10:30:39.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.098+0000 7f352226f640 1 -- 172.21.15.106:0/4176464430 shutdown_connections 2024-04-25T10:30:39.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.098+0000 7f352226f640 1 -- 172.21.15.106:0/4176464430 wait complete. 2024-04-25T10:30:39.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.098+0000 7fb8c9d49640 1 -- 172.21.15.106:0/4186972120 >> [v2:172.21.15.106:6800/1240717079,v1:172.21.15.106:6801/1240717079] conn(0x7fb8a807c7d0 msgr2=0x7fb8a807ec20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.098+0000 7fb8c9d49640 1 --2- 172.21.15.106:0/4186972120 >> [v2:172.21.15.106:6800/1240717079,v1:172.21.15.106:6801/1240717079] conn(0x7fb8a807c7d0 0x7fb8a807ec20 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.099+0000 7fd999ffb640 1 -- 172.21.15.106:0/3575079349 <== osd.5 v2:172.21.15.134:6808/3132275652 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd984002dd0 con 0x7fd98807c820 2024-04-25T10:30:39.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.099+0000 7fb8c9d49640 1 -- 172.21.15.106:0/4186972120 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fb8a8074500 msgr2=0x7fb8a8076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.099+0000 7fb8c9d49640 1 --2- 172.21.15.106:0/4186972120 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fb8a8074500 0x7fb8a8076a10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8005880 tx=0x7fb8b8009040 comp rx=0 tx=0).stop 2024-04-25T10:30:39.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.099+0000 7fb8c9d49640 1 -- 172.21.15.106:0/4186972120 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb8bc0df5d0 msgr2=0x7fb8bc0da9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.099+0000 7fb8c9d49640 1 --2- 172.21.15.106:0/4186972120 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb8bc0df5d0 0x7fb8bc0da9d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb8c40519b0 tx=0x7fb8c40783f0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.099+0000 7fb8c9548640 1 -- 172.21.15.106:0/4186972120 reap_dead start 2024-04-25T10:30:39.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.099+0000 7f0115ffb640 1 -- 172.21.15.106:0/3575619058 shutdown_connections 2024-04-25T10:30:39.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.099+0000 7f0115ffb640 1 -- 172.21.15.106:0/3575619058 >> 172.21.15.106:0/3575619058 conn(0x7f0130068b10 msgr2=0x7f0130075f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:39.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.099+0000 7f587ee30640 1 -- 172.21.15.106:0/4127959366 --> [v2:172.21.15.106:6808/1441018239,v1:172.21.15.106:6809/1441018239] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5870186530 con 0x7f585c07c880 2024-04-25T10:30:39.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.099+0000 7f03c8ff9640 1 -- 172.21.15.106:0/2132973599 >> [v2:172.21.15.134:6824/729224739,v1:172.21.15.134:6825/729224739] conn(0x7f03bc07c850 msgr2=0x7f03bc07eca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.099+0000 7f03c8ff9640 1 --2- 172.21.15.106:0/2132973599 >> [v2:172.21.15.134:6824/729224739,v1:172.21.15.134:6825/729224739] conn(0x7f03bc07c850 0x7f03bc07eca0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.099+0000 7f03c8ff9640 1 -- 172.21.15.106:0/2132973599 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f03bc074580 msgr2=0x7f03bc076a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.099+0000 7f03c8ff9640 1 --2- 172.21.15.106:0/2132973599 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f03bc074580 0x7f03bc076a90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f03d0001840 tx=0x7f03d002e000 comp rx=0 tx=0).stop 2024-04-25T10:30:39.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.099+0000 7fb8c9d49640 1 -- 172.21.15.106:0/4186972120 shutdown_connections 2024-04-25T10:30:39.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.099+0000 7fb8c9d49640 1 -- 172.21.15.106:0/4186972120 >> 172.21.15.106:0/4186972120 conn(0x7fb8bc0514c0 msgr2=0x7fb8bc0df9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:39.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.099+0000 7f03c8ff9640 1 -- 172.21.15.106:0/2132973599 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f03dc14a810 msgr2=0x7f03dc0cd610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.099+0000 7f03c8ff9640 1 --2- 172.21.15.106:0/2132973599 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f03dc14a810 0x7f03dc0cd610 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f03cc009b50 tx=0x7f03cc00b510 comp rx=0 tx=0).stop 2024-04-25T10:30:39.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.099+0000 7f03dad9d640 1 -- 172.21.15.106:0/2132973599 reap_dead start 2024-04-25T10:30:39.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.099+0000 7f03c8ff9640 1 -- 172.21.15.106:0/2132973599 shutdown_connections 2024-04-25T10:30:39.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.099+0000 7f03c8ff9640 1 -- 172.21.15.106:0/2132973599 >> 172.21.15.106:0/2132973599 conn(0x7f03dc06c9b0 msgr2=0x7f03dc14fcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:39.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.099+0000 7fd9ab506640 1 -- 172.21.15.106:0/3575079349 >> [v2:172.21.15.134:6808/3132275652,v1:172.21.15.134:6809/3132275652] conn(0x7fd98807c820 msgr2=0x7fd98807ec70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.099+0000 7fd9ab506640 1 --2- 172.21.15.106:0/3575079349 >> [v2:172.21.15.134:6808/3132275652,v1:172.21.15.134:6809/3132275652] conn(0x7fd98807c820 0x7fd98807ec70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.099+0000 7f0115ffb640 1 -- 172.21.15.106:0/3575619058 shutdown_connections 2024-04-25T10:30:39.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.099+0000 7f03c8ff9640 1 -- 172.21.15.106:0/2132973599 shutdown_connections 2024-04-25T10:30:39.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.099+0000 7f0115ffb640 1 -- 172.21.15.106:0/3575619058 wait complete. 2024-04-25T10:30:39.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.099+0000 7f71abfff640 1 -- 172.21.15.106:0/1929653186 --> [v2:172.21.15.106:6816/1846968711,v1:172.21.15.106:6817/1846968711] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f71c8161460 con 0x7f71ac07c880 2024-04-25T10:30:39.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.099+0000 7f03c8ff9640 1 -- 172.21.15.106:0/2132973599 wait complete. 2024-04-25T10:30:39.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.099+0000 7fb8c9d49640 1 -- 172.21.15.106:0/4186972120 shutdown_connections 2024-04-25T10:30:39.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.100+0000 7fb8c9d49640 1 -- 172.21.15.106:0/4186972120 wait complete. 2024-04-25T10:30:39.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.099+0000 7fd9ab506640 1 -- 172.21.15.106:0/3575079349 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fd9880745b0 msgr2=0x7fd988076ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.099+0000 7fd9ab506640 1 --2- 172.21.15.106:0/3575079349 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fd9880745b0 0x7fd988076ac0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd9a0006fd0 tx=0x7fd9a0008040 comp rx=0 tx=0).stop 2024-04-25T10:30:39.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.099+0000 7fd9ab506640 1 -- 172.21.15.106:0/3575079349 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd9a406e9f0 msgr2=0x7fd9a4200530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.099+0000 7fd9ab506640 1 --2- 172.21.15.106:0/3575079349 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd9a406e9f0 0x7fd9a4200530 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd99400b460 tx=0x7fd99400b920 comp rx=0 tx=0).stop 2024-04-25T10:30:39.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.099+0000 7fd9a9aa3640 1 -- 172.21.15.106:0/3575079349 reap_dead start 2024-04-25T10:30:39.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.100+0000 7fd9ab506640 1 -- 172.21.15.106:0/3575079349 shutdown_connections 2024-04-25T10:30:39.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.100+0000 7fd9ab506640 1 -- 172.21.15.106:0/3575079349 >> 172.21.15.106:0/3575079349 conn(0x7fd9a4068ed0 msgr2=0x7fd9a4159a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:39.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.100+0000 7f71ba7fc640 1 -- 172.21.15.106:0/1929653186 <== osd.3 v2:172.21.15.106:6816/1846968711 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f71c8161460 con 0x7f71ac07c880 2024-04-25T10:30:39.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.100+0000 7f586e7fc640 1 -- 172.21.15.106:0/4127959366 <== osd.2 v2:172.21.15.106:6808/1441018239 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5870186530 con 0x7f585c07c880 2024-04-25T10:30:39.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.100+0000 7f71abfff640 1 -- 172.21.15.106:0/1929653186 >> [v2:172.21.15.106:6816/1846968711,v1:172.21.15.106:6817/1846968711] conn(0x7f71ac07c880 msgr2=0x7f71ac07ecd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.100+0000 7f71abfff640 1 --2- 172.21.15.106:0/1929653186 >> [v2:172.21.15.106:6816/1846968711,v1:172.21.15.106:6817/1846968711] conn(0x7f71ac07c880 0x7f71ac07ecd0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.101+0000 7f71abfff640 1 -- 172.21.15.106:0/1929653186 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f71ac0745b0 msgr2=0x7f71ac076ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.101+0000 7f71abfff640 1 --2- 172.21.15.106:0/1929653186 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f71ac0745b0 0x7f71ac076ac0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f71c4034000 tx=0x7f71c402e000 comp rx=0 tx=0).stop 2024-04-25T10:30:39.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.101+0000 7f71abfff640 1 -- 172.21.15.106:0/1929653186 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f71c80c7ec0 msgr2=0x7f71c8152540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.100+0000 7fd9ab506640 1 -- 172.21.15.106:0/3575079349 shutdown_connections 2024-04-25T10:30:39.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.100+0000 7fd9ab506640 1 -- 172.21.15.106:0/3575079349 wait complete. 2024-04-25T10:30:39.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.101+0000 7f71abfff640 1 --2- 172.21.15.106:0/1929653186 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f71c80c7ec0 0x7f71c8152540 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f71bc00b440 tx=0x7f71bc00b900 comp rx=0 tx=0).stop 2024-04-25T10:30:39.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.101+0000 7f71ce2a3640 1 -- 172.21.15.106:0/1929653186 reap_dead start 2024-04-25T10:30:39.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.101+0000 7f71abfff640 1 -- 172.21.15.106:0/1929653186 shutdown_connections 2024-04-25T10:30:39.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.101+0000 7f71abfff640 1 -- 172.21.15.106:0/1929653186 >> 172.21.15.106:0/1929653186 conn(0x7f71c806d8b0 msgr2=0x7f71c8154fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:39.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.101+0000 7f71abfff640 1 -- 172.21.15.106:0/1929653186 shutdown_connections 2024-04-25T10:30:39.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.101+0000 7f587ee30640 1 -- 172.21.15.106:0/4127959366 >> [v2:172.21.15.106:6808/1441018239,v1:172.21.15.106:6809/1441018239] conn(0x7f585c07c880 msgr2=0x7f585c07ecd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.101+0000 7f587ee30640 1 --2- 172.21.15.106:0/4127959366 >> [v2:172.21.15.106:6808/1441018239,v1:172.21.15.106:6809/1441018239] conn(0x7f585c07c880 0x7f585c07ecd0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.101+0000 7f71abfff640 1 -- 172.21.15.106:0/1929653186 wait complete. 2024-04-25T10:30:39.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.102+0000 7f587ee30640 1 -- 172.21.15.106:0/4127959366 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f585c0745b0 msgr2=0x7f585c076ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.102+0000 7f587ee30640 1 --2- 172.21.15.106:0/4127959366 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f585c0745b0 0x7f585c076ac0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5868006fd0 tx=0x7f5868008040 comp rx=0 tx=0).stop 2024-04-25T10:30:39.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.102+0000 7f587ee30640 1 -- 172.21.15.106:0/4127959366 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f58700d6570 msgr2=0x7f5870185bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.102+0000 7f587ee30640 1 --2- 172.21.15.106:0/4127959366 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f58700d6570 0x7f5870185bc0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5878076d10 tx=0x7f58780783f0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.102+0000 7f587e62f640 1 -- 172.21.15.106:0/4127959366 reap_dead start 2024-04-25T10:30:39.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.103+0000 7f587ee30640 1 -- 172.21.15.106:0/4127959366 shutdown_connections 2024-04-25T10:30:39.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.103+0000 7f587ee30640 1 -- 172.21.15.106:0/4127959366 >> 172.21.15.106:0/4127959366 conn(0x7f58700503b0 msgr2=0x7f58700c4e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:39.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.105+0000 7f587ee30640 1 -- 172.21.15.106:0/4127959366 shutdown_connections 2024-04-25T10:30:39.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.105+0000 7f587ee30640 1 -- 172.21.15.106:0/4127959366 wait complete. 2024-04-25T10:30:39.168 INFO:teuthology.orchestra.run.smithi106.stdout:55834574855 2024-04-25T10:30:39.169 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T10:30:39.170 INFO:teuthology.orchestra.run.smithi106.stdout:55834574855 2024-04-25T10:30:39.170 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T10:30:39.171 INFO:teuthology.orchestra.run.smithi106.stdout:55834574856 2024-04-25T10:30:39.171 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T10:30:39.172 INFO:teuthology.orchestra.run.smithi106.stdout:55834574856 2024-04-25T10:30:39.172 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T10:30:39.173 INFO:teuthology.orchestra.run.smithi106.stdout:55834574855 2024-04-25T10:30:39.173 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T10:30:39.174 INFO:teuthology.orchestra.run.smithi106.stdout:55834574855 2024-04-25T10:30:39.174 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T10:30:39.175 INFO:teuthology.orchestra.run.smithi106.stdout:55834574855 2024-04-25T10:30:39.175 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T10:30:39.176 INFO:teuthology.orchestra.run.smithi106.stdout:55834574855 2024-04-25T10:30:39.176 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T10:30:39.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.416+0000 7f4dc07d7640 1 Processor -- start 2024-04-25T10:30:39.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.416+0000 7f4dc07d7640 1 -- start start 2024-04-25T10:30:39.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.419+0000 7f4dc07d7640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f4db0050f00 0x7f4db0006ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.419+0000 7f4dc07d7640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4db0033510 0x7f4db0045ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.419+0000 7f4dc07d7640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4db0047570 con 0x7f4db00d5720 2024-04-25T10:30:39.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.419+0000 7f4dc07d7640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f4db0035010 con 0x7f4db0033510 2024-04-25T10:30:39.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.419+0000 7f4dc07d7640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f4db0035870 con 0x7f4db0050f00 2024-04-25T10:30:39.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.419+0000 7f4dbe573640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f4db0050f00 0x7f4db0006ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.419+0000 7f4dbe573640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f4db0050f00 0x7f4db0006ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.106:47916/0 (socket says 172.21.15.106:47916) 2024-04-25T10:30:39.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.419+0000 7f4dbe573640 1 -- 172.21.15.106:0/106990909 learned_addr learned my addr 172.21.15.106:0/106990909 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:39.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.419+0000 7f4dbdd72640 1 --2- 172.21.15.106:0/106990909 >> v2:172.21.15.106:3300/0 conn(0x7f4db0033510 0x7f4db0045ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.420+0000 7f4dbe573640 1 -- 172.21.15.106:0/106990909 >> v2:172.21.15.106:3300/0 conn(0x7f4db0033510 msgr2=0x7f4db0045ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:39.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.420+0000 7f4dbe573640 1 --2- 172.21.15.106:0/106990909 >> v2:172.21.15.106:3300/0 conn(0x7f4db0033510 0x7f4db0045ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.420+0000 7f4dbe573640 1 -- 172.21.15.106:0/106990909 >> v1:172.21.15.106:6789/0 conn(0x7f4db00d5720 legacy=0x7f4db0046da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.420+0000 7f4dbe573640 1 -- 172.21.15.106:0/106990909 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4db00352b0 con 0x7f4db0050f00 2024-04-25T10:30:39.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.420+0000 7ff6e3574640 1 Processor -- start 2024-04-25T10:30:39.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.420+0000 7ff6e3574640 1 -- start start 2024-04-25T10:30:39.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.421+0000 7f4dbe573640 1 --2- 172.21.15.106:0/106990909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4db0050f00 0x7f4db0006ac0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4db0007150 tx=0x7f4db402a280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8bdc57382e7f893 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.421+0000 7fb19e763640 1 Processor -- start 2024-04-25T10:30:39.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.422+0000 7fb19e763640 1 -- start start 2024-04-25T10:30:39.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.422+0000 7f4dbd571640 1 -- 172.21.15.106:0/106990909 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db4037070 con 0x7f4db0050f00 2024-04-25T10:30:39.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.422+0000 7f4dbd571640 1 -- 172.21.15.106:0/106990909 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4db4032440 con 0x7f4db0050f00 2024-04-25T10:30:39.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.422+0000 7fe69359e640 1 Processor -- start 2024-04-25T10:30:39.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.422+0000 7fc20ffff640 1 Processor -- start 2024-04-25T10:30:39.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.422+0000 7fc20ffff640 1 -- start start 2024-04-25T10:30:39.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.423+0000 7ff6e3574640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff6d4045be0 0x7ff6d40478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.423+0000 7ff6e3574640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff6d404d1c0 0x7ff6d40e6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.423+0000 7ff6e3574640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6d40dc2a0 con 0x7ff6d4006cd0 2024-04-25T10:30:39.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.423+0000 7ff6e3574640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7ff6d40339b0 con 0x7ff6d4045be0 2024-04-25T10:30:39.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.423+0000 7ff6e3574640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7ff6d4046b90 con 0x7ff6d404d1c0 2024-04-25T10:30:39.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.423+0000 7f4dbd571640 1 -- 172.21.15.106:0/106990909 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db4032810 con 0x7f4db0050f00 2024-04-25T10:30:39.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.423+0000 7f4dc07d7640 1 -- 172.21.15.106:0/106990909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4db0050f00 msgr2=0x7f4db0006ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.423+0000 7f4dc07d7640 1 --2- 172.21.15.106:0/106990909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4db0050f00 0x7f4db0006ac0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4db0007150 tx=0x7f4db402a280 comp rx=0 tx=0).stop 2024-04-25T10:30:39.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.423+0000 7ff6e2572640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7ff6d4006cd0 0x7ff6d404c410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:44322/0 (socket says 172.21.15.106:44322) 2024-04-25T10:30:39.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.423+0000 7ff6e1d71640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff6d4045be0 0x7ff6d40478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.423+0000 7ff6e2572640 1 -- 172.21.15.106:0/1976530794 learned_addr learned my addr 172.21.15.106:0/1976530794 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:30:39.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.423+0000 7f4dc07d7640 1 -- 172.21.15.106:0/106990909 shutdown_connections 2024-04-25T10:30:39.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.423+0000 7f4dc07d7640 1 --2- 172.21.15.106:0/106990909 >> v2:172.21.15.106:3300/0 conn(0x7f4db0033510 0x7f4db0045ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.423+0000 7f4dc07d7640 1 --2- 172.21.15.106:0/106990909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4db0050f00 0x7f4db0006ac0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.423+0000 7fb19e763640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb190009490 0x7fb1900d9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.423+0000 7f4dc07d7640 1 -- 172.21.15.106:0/106990909 >> 172.21.15.106:0/106990909 conn(0x7f4db00503e0 msgr2=0x7f4db00507f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:39.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.424+0000 7ff6e1d71640 1 -- 172.21.15.106:0/1976530794 >> v2:172.21.15.134:3300/0 conn(0x7ff6d404d1c0 msgr2=0x7ff6d40e6100 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T10:30:39.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.424+0000 7ff6e1d71640 1 --2- 172.21.15.106:0/1976530794 >> v2:172.21.15.134:3300/0 conn(0x7ff6d404d1c0 0x7ff6d40e6100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.424+0000 7ff6e1d71640 1 -- 172.21.15.106:0/1976530794 >> v1:172.21.15.106:6789/0 conn(0x7ff6d4006cd0 legacy=0x7ff6d404c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.424+0000 7ff6e1d71640 1 -- 172.21.15.106:0/1976530794 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6d40dc580 con 0x7ff6d4045be0 2024-04-25T10:30:39.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.424+0000 7fb19e763640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1900d9bc0 con 0x7fb190008460 2024-04-25T10:30:39.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.422+0000 7fe69359e640 1 -- start start 2024-04-25T10:30:39.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.424+0000 7fb19cf60640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fb190008460 0x7fb19004bec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:44324/0 (socket says 172.21.15.106:44324) 2024-04-25T10:30:39.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.424+0000 7fb19cf60640 1 -- 172.21.15.106:0/2777701362 learned_addr learned my addr 172.21.15.106:0/2777701362 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:30:39.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.424+0000 7fb19d761640 1 --2- 172.21.15.106:0/2777701362 >> v2:172.21.15.106:3300/0 conn(0x7fb190009490 0x7fb1900d9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.424+0000 7fb19e763640 1 -- 172.21.15.106:0/2777701362 --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb190007820 con 0x7fb190047bb0 2024-04-25T10:30:39.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.424+0000 7fb19e763640 1 -- 172.21.15.106:0/2777701362 --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fb1900dffa0 con 0x7fb190009490 2024-04-25T10:30:39.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.424+0000 7ff8ba7ea640 1 Processor -- start 2024-04-25T10:30:39.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.424+0000 7fe69359e640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fe6940ceb70 0x7fe694161770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.424+0000 7fe69359e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe69415ac40 0x7fe69406c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.424+0000 7f4dc07d7640 1 -- 172.21.15.106:0/106990909 shutdown_connections 2024-04-25T10:30:39.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.424+0000 7fe69359e640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6940c9740 con 0x7fe69406cb00 2024-04-25T10:30:39.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.424+0000 7fe69359e640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fe694161cb0 con 0x7fe6940ceb70 2024-04-25T10:30:39.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.424+0000 7fe69359e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe6940c9a20 con 0x7fe69415ac40 2024-04-25T10:30:39.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.424+0000 7fe69259c640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fe6940ceb70 0x7fe694161770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.424+0000 7fe68bfff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe69415ac40 0x7fe69406c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.424+0000 7fe692d9d640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fe69406cb00 0x7fe6940c69b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.106:41204/0 (socket says 172.21.15.106:41204) 2024-04-25T10:30:39.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.424+0000 7fe692d9d640 1 -- 172.21.15.106:0/1025829727 learned_addr learned my addr 172.21.15.106:0/1025829727 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:30:39.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.424+0000 7fc20ffff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc208037d90 0x7fc2080e0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.424+0000 7fc20ffff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc208037840 0x7fc2080ddd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.424+0000 7fc20ffff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fc20804fb60 0x7fc208052570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.424+0000 7fc20ffff640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fc2080e0f90 con 0x7fc208037840 2024-04-25T10:30:39.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.424+0000 7fc20ffff640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fc2080e1270 con 0x7fc20804fb60 2024-04-25T10:30:39.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.424+0000 7fc20ffff640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc2080e1550 con 0x7fc208037d90 2024-04-25T10:30:39.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.425+0000 7fe691d9b640 1 -- 172.21.15.106:0/1025829727 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 844164672 0 0) 0x7fe6940c9740 con 0x7fe69406cb00 2024-04-25T10:30:39.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.425+0000 7fe691d9b640 1 -- 172.21.15.106:0/1025829727 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe680003520 con 0x7fe69406cb00 2024-04-25T10:30:39.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.425+0000 7fc20effd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc208037d90 0x7fc2080e0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.425+0000 7fb18ffff640 1 -- 172.21.15.106:0/2777701362 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1661043546 0 0) 0x7fb1900d9bc0 con 0x7fb190008460 2024-04-25T10:30:39.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.424+0000 7f4dc07d7640 1 -- 172.21.15.106:0/106990909 wait complete. 2024-04-25T10:30:39.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.425+0000 7fc20effd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc208037d90 0x7fc2080e0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.106:48978/0 (socket says 172.21.15.106:48978) 2024-04-25T10:30:39.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.425+0000 7fc20effd640 1 -- 172.21.15.106:0/2851000337 learned_addr learned my addr 172.21.15.106:0/2851000337 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:39.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.425+0000 7fb18ffff640 1 -- 172.21.15.106:0/2777701362 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb184003520 con 0x7fb190008460 2024-04-25T10:30:39.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.425+0000 7fe68bfff640 1 -- 172.21.15.106:0/1025829727 >> v2:172.21.15.134:3300/0 conn(0x7fe6940ceb70 msgr2=0x7fe694161770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:39.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.425+0000 7fe68bfff640 1 --2- 172.21.15.106:0/1025829727 >> v2:172.21.15.134:3300/0 conn(0x7fe6940ceb70 0x7fe694161770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.425+0000 7fe68bfff640 1 -- 172.21.15.106:0/1025829727 >> v1:172.21.15.134:6789/0 conn(0x7fe69406cb00 legacy=0x7fe6940c69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.425+0000 7ff6e1d71640 1 --2- 172.21.15.106:0/1976530794 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6d4045be0 0x7ff6d40478c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff6d404be10 tx=0x7ff6d802b250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=672fb75b3ea04364 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.425+0000 7fc20effd640 1 -- 172.21.15.106:0/2851000337 >> v2:172.21.15.134:3300/0 conn(0x7fc20804fb60 msgr2=0x7fc208052570 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T10:30:39.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.425+0000 7fc20effd640 1 --2- 172.21.15.106:0/2851000337 >> v2:172.21.15.134:3300/0 conn(0x7fc20804fb60 0x7fc208052570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.425+0000 7fc20effd640 1 -- 172.21.15.106:0/2851000337 >> v2:172.21.15.106:3300/0 conn(0x7fc208037840 msgr2=0x7fc2080ddd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:39.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.425+0000 7fc20effd640 1 --2- 172.21.15.106:0/2851000337 >> v2:172.21.15.106:3300/0 conn(0x7fc208037840 0x7fc2080ddd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.425+0000 7fc20effd640 1 -- 172.21.15.106:0/2851000337 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2080dd530 con 0x7fc208037d90 2024-04-25T10:30:39.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.425+0000 7fe68bfff640 1 -- 172.21.15.106:0/1025829727 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6940c9d00 con 0x7fe69415ac40 2024-04-25T10:30:39.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.425+0000 7ff6e1570640 1 -- 172.21.15.106:0/1976530794 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d8038070 con 0x7ff6d4045be0 2024-04-25T10:30:39.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.425+0000 7fb19d761640 1 -- 172.21.15.106:0/2777701362 >> v1:172.21.15.134:6789/0 conn(0x7fb190047bb0 legacy=0x7fb190046e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.425+0000 7fb19d761640 1 -- 172.21.15.106:0/2777701362 >> v1:172.21.15.106:6789/0 conn(0x7fb190008460 legacy=0x7fb19004bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.426+0000 7fb19d761640 1 -- 172.21.15.106:0/2777701362 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1900e0430 con 0x7fb190009490 2024-04-25T10:30:39.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.426+0000 7ff6e1570640 1 -- 172.21.15.106:0/1976530794 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6d8033440 con 0x7ff6d4045be0 2024-04-25T10:30:39.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.424+0000 7f6fe359e640 1 Processor -- start 2024-04-25T10:30:39.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.426+0000 7ff6e1570640 1 -- 172.21.15.106:0/1976530794 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d8033810 con 0x7ff6d4045be0 2024-04-25T10:30:39.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.427+0000 7f4dc07d7640 1 Processor -- start 2024-04-25T10:30:39.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.427+0000 7f4dc07d7640 1 -- start start 2024-04-25T10:30:39.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.427+0000 7ff6e3574640 1 -- 172.21.15.106:0/1976530794 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6d4045be0 msgr2=0x7ff6d40478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.427+0000 7ff6e3574640 1 --2- 172.21.15.106:0/1976530794 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6d4045be0 0x7ff6d40478c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff6d404be10 tx=0x7ff6d802b250 comp rx=0 tx=0).stop 2024-04-25T10:30:39.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.427+0000 7ff6e3574640 1 -- 172.21.15.106:0/1976530794 shutdown_connections 2024-04-25T10:30:39.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.427+0000 7ff6e3574640 1 --2- 172.21.15.106:0/1976530794 >> v2:172.21.15.134:3300/0 conn(0x7ff6d404d1c0 0x7ff6d40e6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.427+0000 7ff6e3574640 1 --2- 172.21.15.106:0/1976530794 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6d4045be0 0x7ff6d40478c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.427+0000 7ff6e3574640 1 -- 172.21.15.106:0/1976530794 >> 172.21.15.106:0/1976530794 conn(0x7ff6d4050c40 msgr2=0x7ff6d4051050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:39.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.424+0000 7ff8ba7ea640 1 -- start start 2024-04-25T10:30:39.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.427+0000 7ff8ba7ea640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8b4156ce0 0x7ff8b40ccf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.427+0000 7ff8ba7ea640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff8b40ce5b0 0x7ff8b40ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.426+0000 7fe68bfff640 1 --2- 172.21.15.106:0/1025829727 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe69415ac40 0x7fe69406c5c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe67c0097a0 tx=0x7fe67c02a230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9681ea887c940ae2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.426+0000 7fe691d9b640 1 -- 172.21.15.106:0/1025829727 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe67c037070 con 0x7fe69415ac40 2024-04-25T10:30:39.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.426+0000 7fe691d9b640 1 -- 172.21.15.106:0/1025829727 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe67c032440 con 0x7fe69415ac40 2024-04-25T10:30:39.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.426+0000 7fe691d9b640 1 -- 172.21.15.106:0/1025829727 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe67c032810 con 0x7fe69415ac40 2024-04-25T10:30:39.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.427+0000 7fc20effd640 1 --2- 172.21.15.106:0/2851000337 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc208037d90 0x7fc2080e0900 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc208050470 tx=0x7fc20402a280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7cc98e073626fd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.426+0000 7fcee759e640 1 Processor -- start 2024-04-25T10:30:39.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.426+0000 7fcee759e640 1 -- start start 2024-04-25T10:30:39.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.427+0000 7fb19d761640 1 --2- 172.21.15.106:0/2777701362 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb190009490 0x7fb1900d9680 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb188009720 tx=0x7fb18802a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=680765ab5b2b9761 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.427+0000 7fb18ffff640 1 -- 172.21.15.106:0/2777701362 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb188037070 con 0x7fb190009490 2024-04-25T10:30:39.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.427+0000 7fb18ffff640 1 -- 172.21.15.106:0/2777701362 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb18802aad0 con 0x7fb190009490 2024-04-25T10:30:39.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.427+0000 7fb18ffff640 1 -- 172.21.15.106:0/2777701362 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb194001690 con 0x7fb190009490 2024-04-25T10:30:39.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.427+0000 7ff6e3574640 1 -- 172.21.15.106:0/1976530794 shutdown_connections 2024-04-25T10:30:39.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.427+0000 7ff6e3574640 1 -- 172.21.15.106:0/1976530794 wait complete. 2024-04-25T10:30:39.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.427+0000 7ff8b9fe9640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff8b40ce5b0 0x7ff8b40ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.427+0000 7ff8b9fe9640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff8b40ce5b0 0x7ff8b40ce9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47762/0 (socket says 172.21.15.106:47762) 2024-04-25T10:30:39.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.427+0000 7ff8b8fe7640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7ff8b415d340 0x7ff8b40c7070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.106:41214/0 (socket says 172.21.15.106:41214) 2024-04-25T10:30:39.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.427+0000 7ff8b9fe9640 1 -- 172.21.15.106:0/1855875262 learned_addr learned my addr 172.21.15.106:0/1855875262 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:39.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.427+0000 7fe69359e640 1 -- 172.21.15.106:0/1025829727 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe69415ac40 msgr2=0x7fe69406c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.427+0000 7fe69359e640 1 --2- 172.21.15.106:0/1025829727 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe69415ac40 0x7fe69406c5c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe67c0097a0 tx=0x7fe67c02a230 comp rx=0 tx=0).stop 2024-04-25T10:30:39.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.428+0000 7fe69359e640 1 -- 172.21.15.106:0/1025829727 shutdown_connections 2024-04-25T10:30:39.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.428+0000 7fe69359e640 1 --2- 172.21.15.106:0/1025829727 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe69415ac40 0x7fe69406c5c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.428+0000 7fe69359e640 1 --2- 172.21.15.106:0/1025829727 >> v2:172.21.15.134:3300/0 conn(0x7fe6940ceb70 0x7fe694161770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.428+0000 7fe69359e640 1 -- 172.21.15.106:0/1025829727 >> 172.21.15.106:0/1025829727 conn(0x7fe694070f00 msgr2=0x7fe6940ce2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:39.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.428+0000 7fc20dffb640 1 -- 172.21.15.106:0/2851000337 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc204037070 con 0x7fc208037d90 2024-04-25T10:30:39.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.428+0000 7fc20dffb640 1 -- 172.21.15.106:0/2851000337 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc204032440 con 0x7fc208037d90 2024-04-25T10:30:39.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.428+0000 7fb19e763640 1 -- 172.21.15.106:0/2777701362 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb190009490 msgr2=0x7fb1900d9680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.428+0000 7fb19e763640 1 --2- 172.21.15.106:0/2777701362 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb190009490 0x7fb1900d9680 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb188009720 tx=0x7fb18802a040 comp rx=0 tx=0).stop 2024-04-25T10:30:39.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.428+0000 7fe69359e640 1 -- 172.21.15.106:0/1025829727 shutdown_connections 2024-04-25T10:30:39.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.428+0000 7fe69359e640 1 -- 172.21.15.106:0/1025829727 wait complete. 2024-04-25T10:30:39.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.428+0000 7fcee759e640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fcee0008410 0x7fcee004be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.428+0000 7fcee759e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcee004c3d0 0x7fcee0006c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.428+0000 7fcee759e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fcee0007170 0x7fcee00e6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.428+0000 7fcee759e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fcee00e4340 con 0x7fcee0007170 2024-04-25T10:30:39.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.428+0000 7fcee759e640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fcee00da6d0 con 0x7fcee0008410 2024-04-25T10:30:39.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.428+0000 7fcee759e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fcee0008820 con 0x7fcee004c3d0 2024-04-25T10:30:39.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.427+0000 7ff8ba7ea640 1 -- 172.21.15.106:0/1855875262 --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8b40cf1d0 con 0x7ff8b415d340 2024-04-25T10:30:39.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.427+0000 7ff8ba7ea640 1 -- 172.21.15.106:0/1855875262 --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7ff8b4160e60 con 0x7ff8b40ce5b0 2024-04-25T10:30:39.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.427+0000 7ff8ba7ea640 1 -- 172.21.15.106:0/1855875262 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff8b40cd460 con 0x7ff8b4156ce0 2024-04-25T10:30:39.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.429+0000 7fb19e763640 1 -- 172.21.15.106:0/2777701362 shutdown_connections 2024-04-25T10:30:39.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.426+0000 7f6fe359e640 1 -- start start 2024-04-25T10:30:39.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.429+0000 7fb19e763640 1 --2- 172.21.15.106:0/2777701362 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb190009490 0x7fb1900d9680 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.429+0000 7fb19e763640 1 -- 172.21.15.106:0/2777701362 >> 172.21.15.106:0/2777701362 conn(0x7fb190051540 msgr2=0x7fb190051950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:39.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.429+0000 7ff8b97e8640 1 --2- 172.21.15.106:0/1855875262 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8b4156ce0 0x7ff8b40ccf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.429+0000 7f6fe359e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6fdc0d9830 0x7f6fdc0073b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.429+0000 7f6fe359e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6fdc050ab0 0x7f6fdc0078f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.429+0000 7f6fe359e640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fdc0dfee0 con 0x7f6fdc04abd0 2024-04-25T10:30:39.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.429+0000 7f6fe359e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f6fdc049fd0 con 0x7f6fdc050ab0 2024-04-25T10:30:39.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.429+0000 7fb19e763640 1 -- 172.21.15.106:0/2777701362 shutdown_connections 2024-04-25T10:30:39.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.429+0000 7fc20ffff640 1 -- 172.21.15.106:0/2851000337 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc208037d90 msgr2=0x7fc2080e0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.429+0000 7fc20dffb640 1 -- 172.21.15.106:0/2851000337 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc204037070 con 0x7fc208037d90 2024-04-25T10:30:39.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.429+0000 7fc20ffff640 1 --2- 172.21.15.106:0/2851000337 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc208037d90 0x7fc2080e0900 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc208050470 tx=0x7fc20402a280 comp rx=0 tx=0).stop 2024-04-25T10:30:39.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.429+0000 7fb19e763640 1 -- 172.21.15.106:0/2777701362 wait complete. 2024-04-25T10:30:39.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.428+0000 7fcee659c640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fcee0008410 0x7fcee004be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.429+0000 7fcee659c640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fcee0008410 0x7fcee004be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.106:47960/0 (socket says 172.21.15.106:47960) 2024-04-25T10:30:39.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.429+0000 7fcee659c640 1 -- 172.21.15.106:0/277733345 learned_addr learned my addr 172.21.15.106:0/277733345 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:39.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.429+0000 7fcee5d9b640 1 --2- 172.21.15.106:0/277733345 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcee004c3d0 0x7fcee0006c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.429+0000 7f4dc07d7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4db0033510 0x7f4db00d7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.429+0000 7f4dc07d7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4db0050f00 0x7f4db00db660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.429+0000 7f4dc07d7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4db00d5720 0x7f4db0179710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.429+0000 7f4dc07d7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4db0045380 con 0x7f4db0050f00 2024-04-25T10:30:39.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.429+0000 7f4dc07d7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4db00065a0 con 0x7f4db0033510 2024-04-25T10:30:39.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.429+0000 7f4dc07d7640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f4db00d5b30 con 0x7f4db00d5720 2024-04-25T10:30:39.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.429+0000 7fc20ffff640 1 -- 172.21.15.106:0/2851000337 shutdown_connections 2024-04-25T10:30:39.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.429+0000 7fc20ffff640 1 --2- 172.21.15.106:0/2851000337 >> v2:172.21.15.134:3300/0 conn(0x7fc20804fb60 0x7fc208052570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.429+0000 7fc20ffff640 1 --2- 172.21.15.106:0/2851000337 >> v2:172.21.15.106:3300/0 conn(0x7fc208037840 0x7fc2080ddd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.429+0000 7fc20ffff640 1 --2- 172.21.15.106:0/2851000337 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc208037d90 0x7fc2080e0900 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.429+0000 7fc20ffff640 1 -- 172.21.15.106:0/2851000337 >> 172.21.15.106:0/2851000337 conn(0x7fc20800e2e0 msgr2=0x7fc20800e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:39.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.429+0000 7fc20ffff640 1 -- 172.21.15.106:0/2851000337 shutdown_connections 2024-04-25T10:30:39.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.429+0000 7f4dbdd72640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4db0050f00 0x7f4db00db660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.429+0000 7f4dbed74640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4db00d5720 0x7f4db0179710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.429+0000 7f4dbdd72640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4db0050f00 0x7f4db00db660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47800/0 (socket says 172.21.15.106:47800) 2024-04-25T10:30:39.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.429+0000 7f4dbdd72640 1 -- 172.21.15.106:0/1526498517 learned_addr learned my addr 172.21.15.106:0/1526498517 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:39.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.429+0000 7fcee6d9d640 1 --2- 172.21.15.106:0/277733345 >> v2:172.21.15.106:3300/0 conn(0x7fcee0007170 0x7fcee00e6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.430+0000 7f4dbe573640 1 --2- 172.21.15.106:0/1526498517 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4db0033510 0x7f4db00d7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.430+0000 7ff6e3574640 1 Processor -- start 2024-04-25T10:30:39.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.430+0000 7f4dbdd72640 1 -- 172.21.15.106:0/1526498517 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4db00d5720 msgr2=0x7f4db0179710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.430+0000 7f4dbdd72640 1 --2- 172.21.15.106:0/1526498517 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4db00d5720 0x7f4db0179710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.430+0000 7f4dbdd72640 1 -- 172.21.15.106:0/1526498517 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4db0033510 msgr2=0x7f4db00d7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.430+0000 7f4dbdd72640 1 --2- 172.21.15.106:0/1526498517 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4db0033510 0x7f4db00d7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.430+0000 7f4dbdd72640 1 -- 172.21.15.106:0/1526498517 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4db00d8e60 con 0x7f4db0050f00 2024-04-25T10:30:39.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.430+0000 7f4dbdd72640 1 --2- 172.21.15.106:0/1526498517 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4db0050f00 0x7f4db00db660 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4db80519b0 tx=0x7f4db80783f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.430+0000 7ff6e3574640 1 -- start start 2024-04-25T10:30:39.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.430+0000 7fcee6d9d640 1 -- 172.21.15.106:0/277733345 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcee004c3d0 msgr2=0x7fcee0006c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:39.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.430+0000 7fcee6d9d640 1 --2- 172.21.15.106:0/277733345 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcee004c3d0 0x7fcee0006c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.430+0000 7fcee6d9d640 1 -- 172.21.15.106:0/277733345 >> v2:172.21.15.134:3300/0 conn(0x7fcee0008410 msgr2=0x7fcee004be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:39.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.430+0000 7fcee6d9d640 1 --2- 172.21.15.106:0/277733345 >> v2:172.21.15.134:3300/0 conn(0x7fcee0008410 0x7fcee004be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.430+0000 7fcee6d9d640 1 -- 172.21.15.106:0/277733345 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcee0047220 con 0x7fcee0007170 2024-04-25T10:30:39.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.429+0000 7ff8abfff640 1 -- 172.21.15.106:0/1855875262 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3167920883 0 0) 0x7ff8a4001110 con 0x7ff8b415d340 2024-04-25T10:30:39.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.429+0000 7ff8abfff640 1 -- 172.21.15.106:0/1855875262 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff8a0003520 con 0x7ff8b415d340 2024-04-25T10:30:39.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.430+0000 7ff8b97e8640 1 -- 172.21.15.106:0/1855875262 >> v2:172.21.15.106:3300/0 conn(0x7ff8b40ce5b0 msgr2=0x7ff8b40ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:39.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.430+0000 7ff8b97e8640 1 --2- 172.21.15.106:0/1855875262 >> v2:172.21.15.106:3300/0 conn(0x7ff8b40ce5b0 0x7ff8b40ce9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.430+0000 7ff8b97e8640 1 -- 172.21.15.106:0/1855875262 >> v1:172.21.15.134:6789/0 conn(0x7ff8b415d340 legacy=0x7ff8b40c7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.430+0000 7ff8b97e8640 1 -- 172.21.15.106:0/1855875262 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8b40cd700 con 0x7ff8b4156ce0 2024-04-25T10:30:39.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.431+0000 7f4daeffd640 1 -- 172.21.15.106:0/1526498517 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db806db50 con 0x7f4db0050f00 2024-04-25T10:30:39.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.431+0000 7f4daeffd640 1 -- 172.21.15.106:0/1526498517 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4db806dcf0 con 0x7f4db0050f00 2024-04-25T10:30:39.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.431+0000 7f4daeffd640 1 -- 172.21.15.106:0/1526498517 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db808e8b0 con 0x7f4db0050f00 2024-04-25T10:30:39.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.431+0000 7f4dc07d7640 1 -- 172.21.15.106:0/1526498517 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4db00d7990 con 0x7f4db0050f00 2024-04-25T10:30:39.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.431+0000 7f4dc07d7640 1 -- 172.21.15.106:0/1526498517 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4db00d88a0 con 0x7f4db0050f00 2024-04-25T10:30:39.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.431+0000 7ff8b97e8640 1 --2- 172.21.15.106:0/1855875262 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8b4156ce0 0x7ff8b40ccf20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff8b00097a0 tx=0x7ff8b0032a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bb6a23137caf75c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.429+0000 7f6fe359e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6fdc0e0180 con 0x7f6fdc0d9830 2024-04-25T10:30:39.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.429+0000 7f6fe259c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6fdc0d9830 0x7f6fdc0073b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.429+0000 7f6fe259c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6fdc0d9830 0x7f6fdc0073b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.106:49000/0 (socket says 172.21.15.106:49000) 2024-04-25T10:30:39.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.429+0000 7f6fe259c640 1 -- 172.21.15.106:0/1267551168 learned_addr learned my addr 172.21.15.106:0/1267551168 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:39.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.429+0000 7f6fe1d9b640 1 --2- 172.21.15.106:0/1267551168 >> v2:172.21.15.106:3300/0 conn(0x7f6fdc050ab0 0x7f6fdc0078f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.429+0000 7f6fe259c640 1 -- 172.21.15.106:0/1267551168 >> v2:172.21.15.106:3300/0 conn(0x7f6fdc050ab0 msgr2=0x7f6fdc0078f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:39.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.429+0000 7f6fe259c640 1 --2- 172.21.15.106:0/1267551168 >> v2:172.21.15.106:3300/0 conn(0x7f6fdc050ab0 0x7f6fdc0078f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.429+0000 7f6fe259c640 1 -- 172.21.15.106:0/1267551168 >> v1:172.21.15.134:6789/0 conn(0x7f6fdc04abd0 legacy=0x7f6fdc04afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:39.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.429+0000 7f6fe259c640 1 -- 172.21.15.106:0/1267551168 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fdc04a6b0 con 0x7f6fdc0d9830 2024-04-25T10:30:39.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.431+0000 7f6fe259c640 1 --2- 172.21.15.106:0/1267551168 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6fdc0d9830 0x7f6fdc0073b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc008810 tx=0x7f6fd4032a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db9ba478a02ae7bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.429+0000 7fc20ffff640 1 -- 172.21.15.106:0/2851000337 wait complete. 2024-04-25T10:30:39.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.432+0000 7fc20ffff640 1 Processor -- start 2024-04-25T10:30:39.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.432+0000 7f6fe159a640 1 -- 172.21.15.106:0/1267551168 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd403b070 con 0x7f6fdc0d9830 2024-04-25T10:30:39.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.432+0000 7f6fe159a640 1 -- 172.21.15.106:0/1267551168 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fd4037840 con 0x7f6fdc0d9830 2024-04-25T10:30:39.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.432+0000 7fb19e763640 1 Processor -- start 2024-04-25T10:30:39.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.432+0000 7fb19e763640 1 -- start start 2024-04-25T10:30:39.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.432+0000 7fcee6d9d640 1 --2- 172.21.15.106:0/277733345 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcee0007170 0x7fcee00e6df0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fcee004db30 tx=0x7fced802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd8a3f9b28861720 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.432+0000 7f6fe159a640 1 -- 172.21.15.106:0/1267551168 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd403b070 con 0x7f6fdc0d9830 2024-04-25T10:30:39.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.433+0000 7f4dc07d7640 1 -- 172.21.15.106:0/1526498517 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4db00055a0 con 0x7f4db0050f00 2024-04-25T10:30:39.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.432+0000 7ff6e3574640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6d4006cd0 0x7ff6d4047f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.433+0000 7ff6e3574640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6d4045be0 0x7ff6d4048f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.433+0000 7ff6e3574640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6d404d1c0 0x7ff6d4175950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.433+0000 7ff6e3574640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d4047e00 con 0x7ff6d4045be0 2024-04-25T10:30:39.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.433+0000 7ff6e3574640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d4047430 con 0x7ff6d4006cd0 2024-04-25T10:30:39.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.433+0000 7ff6e3574640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff6d40470c0 con 0x7ff6d404d1c0 2024-04-25T10:30:39.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.433+0000 7ff6e2572640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6d4006cd0 0x7ff6d4047f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.433+0000 7ff6e2572640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6d4006cd0 0x7ff6d4047f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.106:47986/0 (socket says 172.21.15.106:47986) 2024-04-25T10:30:39.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.433+0000 7ff6e2572640 1 -- 172.21.15.106:0/1826006075 learned_addr learned my addr 172.21.15.106:0/1826006075 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:39.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.433+0000 7ff6e1d71640 1 --2- 172.21.15.106:0/1826006075 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6d4045be0 0x7ff6d4048f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.433+0000 7fe69359e640 1 Processor -- start 2024-04-25T10:30:39.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.433+0000 7fe69359e640 1 -- start start 2024-04-25T10:30:39.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.432+0000 7fc20ffff640 1 -- start start 2024-04-25T10:30:39.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.433+0000 7f6fe359e640 1 -- 172.21.15.106:0/1267551168 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6fdc0d9830 msgr2=0x7f6fdc0073b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.433+0000 7f6fe359e640 1 --2- 172.21.15.106:0/1267551168 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6fdc0d9830 0x7f6fdc0073b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc008810 tx=0x7f6fd4032a20 comp rx=0 tx=0).stop 2024-04-25T10:30:39.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.433+0000 7ff6e2d73640 1 --2- 172.21.15.106:0/1826006075 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6d404d1c0 0x7ff6d4175950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.433+0000 7ff6e2572640 1 -- 172.21.15.106:0/1826006075 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6d404d1c0 msgr2=0x7ff6d4175950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.433+0000 7f6fe359e640 1 -- 172.21.15.106:0/1267551168 shutdown_connections 2024-04-25T10:30:39.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.433+0000 7f6fe359e640 1 --2- 172.21.15.106:0/1267551168 >> v2:172.21.15.106:3300/0 conn(0x7f6fdc050ab0 0x7f6fdc0078f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.433+0000 7fcee559a640 1 -- 172.21.15.106:0/277733345 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced8037070 con 0x7fcee0007170 2024-04-25T10:30:39.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.433+0000 7f6fe359e640 1 --2- 172.21.15.106:0/1267551168 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6fdc0d9830 0x7f6fdc0073b0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.433+0000 7f6fe359e640 1 -- 172.21.15.106:0/1267551168 >> 172.21.15.106:0/1267551168 conn(0x7f6fdc00e360 msgr2=0x7f6fdc00e770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:39.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.433+0000 7fcee559a640 1 -- 172.21.15.106:0/277733345 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fced8032440 con 0x7fcee0007170 2024-04-25T10:30:39.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.433+0000 7ff6e2572640 1 --2- 172.21.15.106:0/1826006075 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6d404d1c0 0x7ff6d4175950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.433+0000 7ff6e2572640 1 -- 172.21.15.106:0/1826006075 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6d4045be0 msgr2=0x7ff6d4048f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.433+0000 7ff6e2572640 1 --2- 172.21.15.106:0/1826006075 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6d4045be0 0x7ff6d4048f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.433+0000 7ff6e2572640 1 -- 172.21.15.106:0/1826006075 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6d40494a0 con 0x7ff6d4006cd0 2024-04-25T10:30:39.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.433+0000 7ff6e1d71640 1 --2- 172.21.15.106:0/1826006075 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6d4045be0 0x7ff6d4048f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:39.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.433+0000 7fcee559a640 1 -- 172.21.15.106:0/277733345 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced80327f0 con 0x7fcee0007170 2024-04-25T10:30:39.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.434+0000 7ff6e2572640 1 --2- 172.21.15.106:0/1826006075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6d4006cd0 0x7ff6d4047f80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc076d10 tx=0x7ff6dc0783f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.434+0000 7fb19e763640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb190008460 0x7fb1900d9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.434+0000 7fb19e763640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb190009490 0x7fb1901765a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.434+0000 7fb19e763640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb190047bb0 0x7fb19004b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.434+0000 7fb19e763640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb1900df950 con 0x7fb190047bb0 2024-04-25T10:30:39.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.434+0000 7fb19e763640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb1900df670 con 0x7fb190008460 2024-04-25T10:30:39.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.434+0000 7fb19e763640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb1900dfc30 con 0x7fb190009490 2024-04-25T10:30:39.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.434+0000 7fb19df62640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb190047bb0 0x7fb19004b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.434+0000 7fb19df62640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb190047bb0 0x7fb19004b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47820/0 (socket says 172.21.15.106:47820) 2024-04-25T10:30:39.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.434+0000 7fb19df62640 1 -- 172.21.15.106:0/2280543230 learned_addr learned my addr 172.21.15.106:0/2280543230 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:39.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.434+0000 7fb19cf60640 1 --2- 172.21.15.106:0/2280543230 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb190009490 0x7fb1901765a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.434+0000 7ff6d2ffd640 1 -- 172.21.15.106:0/1826006075 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc08ec40 con 0x7ff6d4006cd0 2024-04-25T10:30:39.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.434+0000 7ff6d2ffd640 1 -- 172.21.15.106:0/1826006075 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6dc08ede0 con 0x7ff6d4006cd0 2024-04-25T10:30:39.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.434+0000 7fe69359e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe69406cb00 0x7fe69415bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.434+0000 7fe69359e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe6940ceb70 0x7fe694154f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.434+0000 7fb19df62640 1 -- 172.21.15.106:0/2280543230 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb190009490 msgr2=0x7fb1901765a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.434+0000 7fb19df62640 1 --2- 172.21.15.106:0/2280543230 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb190009490 0x7fb1901765a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.434+0000 7fb19df62640 1 -- 172.21.15.106:0/2280543230 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb190008460 msgr2=0x7fb1900d9ec0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:30:39.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.434+0000 7fb19df62640 1 --2- 172.21.15.106:0/2280543230 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb190008460 0x7fb1900d9ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.434+0000 7ff6d2ffd640 1 -- 172.21.15.106:0/1826006075 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc06c7c0 con 0x7ff6d4006cd0 2024-04-25T10:30:39.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.435+0000 7fe69259c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe69406cb00 0x7fe69415bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.435+0000 7fe69359e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe69415ac40 0x7fe69414f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.435+0000 7fe69359e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe6940ca5a0 con 0x7fe6940ceb70 2024-04-25T10:30:39.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.435+0000 7fe69259c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe69406cb00 0x7fe69415bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.106:48002/0 (socket says 172.21.15.106:48002) 2024-04-25T10:30:39.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.435+0000 7fe69359e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe6940ca2c0 con 0x7fe69406cb00 2024-04-25T10:30:39.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.435+0000 7fe69259c640 1 -- 172.21.15.106:0/210111363 learned_addr learned my addr 172.21.15.106:0/210111363 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:39.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.435+0000 7fe69359e640 1 -- 172.21.15.106:0/210111363 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe69415b050 con 0x7fe69415ac40 2024-04-25T10:30:39.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.435+0000 7fc20ffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc208037840 0x7fc2080da830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.435+0000 7ff6e3574640 1 -- 172.21.15.106:0/1826006075 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6d4175e90 con 0x7ff6d4006cd0 2024-04-25T10:30:39.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.434+0000 7fb19df62640 1 -- 172.21.15.106:0/2280543230 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb190176ae0 con 0x7fb190047bb0 2024-04-25T10:30:39.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.435+0000 7fc20ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc208037d90 0x7fc2080d4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.435+0000 7fe692d9d640 1 --2- 172.21.15.106:0/210111363 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe69415ac40 0x7fe69414f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.435+0000 7fc20ffff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc20804fb60 0x7fc2080d1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.435+0000 7ff6e3574640 1 -- 172.21.15.106:0/1826006075 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6d40cd3e0 con 0x7ff6d4006cd0 2024-04-25T10:30:39.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.435+0000 7fb19df62640 1 --2- 172.21.15.106:0/2280543230 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb190047bb0 0x7fb19004b020 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb198075210 tx=0x7fb1980783f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.435+0000 7fe69259c640 1 -- 172.21.15.106:0/210111363 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe69415ac40 msgr2=0x7fe69414f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.435+0000 7fe69259c640 1 --2- 172.21.15.106:0/210111363 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe69415ac40 0x7fe69414f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.435+0000 7fe69259c640 1 -- 172.21.15.106:0/210111363 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe6940ceb70 msgr2=0x7fe694154f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.435+0000 7fe69259c640 1 --2- 172.21.15.106:0/210111363 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe6940ceb70 0x7fe694154f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.435+0000 7fe69259c640 1 -- 172.21.15.106:0/210111363 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe694155440 con 0x7fe69406cb00 2024-04-25T10:30:39.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.435+0000 7fc20ffff640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc20804edd0 con 0x7fc208037840 2024-04-25T10:30:39.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.435+0000 7fc20f7fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc20804fb60 0x7fc2080d1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.435+0000 7fc20e7fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc208037d90 0x7fc2080d4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.435+0000 7fc20f7fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc20804fb60 0x7fc2080d1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.106:49050/0 (socket says 172.21.15.106:49050) 2024-04-25T10:30:39.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.435+0000 7fc20f7fe640 1 -- 172.21.15.106:0/365299902 learned_addr learned my addr 172.21.15.106:0/365299902 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:39.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.435+0000 7fc20ffff640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc2080de270 con 0x7fc208037d90 2024-04-25T10:30:39.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.435+0000 7fc20ffff640 1 -- 172.21.15.106:0/365299902 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc20804ef50 con 0x7fc20804fb60 2024-04-25T10:30:39.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.435+0000 7f6fe359e640 1 -- 172.21.15.106:0/1267551168 shutdown_connections 2024-04-25T10:30:39.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.435+0000 7fe69259c640 1 --2- 172.21.15.106:0/210111363 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe69406cb00 0x7fe69415bb80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe68400d650 tx=0x7fe68400db10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.435+0000 7f6fe359e640 1 -- 172.21.15.106:0/1267551168 wait complete. 2024-04-25T10:30:39.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.434+0000 7ff8abfff640 1 -- 172.21.15.106:0/1855875262 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b003b070 con 0x7ff8b4156ce0 2024-04-25T10:30:39.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.434+0000 7ff8abfff640 1 -- 172.21.15.106:0/1855875262 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8b0037880 con 0x7ff8b4156ce0 2024-04-25T10:30:39.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.434+0000 7ff8abfff640 1 -- 172.21.15.106:0/1855875262 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b0037c50 con 0x7ff8b4156ce0 2024-04-25T10:30:39.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.435+0000 7ff8ba7ea640 1 -- 172.21.15.106:0/1855875262 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8b4156ce0 msgr2=0x7ff8b40ccf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.435+0000 7ff8ba7ea640 1 --2- 172.21.15.106:0/1855875262 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8b4156ce0 0x7ff8b40ccf20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff8b00097a0 tx=0x7ff8b0032a20 comp rx=0 tx=0).stop 2024-04-25T10:30:39.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.435+0000 7f4daeffd640 1 -- 172.21.15.106:0/1526498517 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4db808ea50 con 0x7f4db0050f00 2024-04-25T10:30:39.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.436+0000 7ff8ba7ea640 1 -- 172.21.15.106:0/1855875262 shutdown_connections 2024-04-25T10:30:39.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.436+0000 7ff8ba7ea640 1 --2- 172.21.15.106:0/1855875262 >> v2:172.21.15.106:3300/0 conn(0x7ff8b40ce5b0 0x7ff8b40ce9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.436+0000 7ff8ba7ea640 1 --2- 172.21.15.106:0/1855875262 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8b4156ce0 0x7ff8b40ccf20 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.436+0000 7ff8ba7ea640 1 -- 172.21.15.106:0/1855875262 >> 172.21.15.106:0/1855875262 conn(0x7ff8b406d770 msgr2=0x7ff8b406e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:39.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.436+0000 7ff8ba7ea640 1 -- 172.21.15.106:0/1855875262 shutdown_connections 2024-04-25T10:30:39.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.436+0000 7f4daeffd640 1 --2- 172.21.15.106:0/1526498517 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f4d9c0745b0 0x7f4d9c076ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.436+0000 7f4daeffd640 1 -- 172.21.15.106:0/1526498517 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4db81120f0 con 0x7f4db0050f00 2024-04-25T10:30:39.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.436+0000 7f4dbe573640 1 --2- 172.21.15.106:0/1526498517 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f4d9c0745b0 0x7f4d9c076ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.436+0000 7ff6e3574640 1 -- 172.21.15.106:0/1826006075 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6d40059c0 con 0x7ff6d4006cd0 2024-04-25T10:30:39.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.436+0000 7fcee759e640 1 -- 172.21.15.106:0/277733345 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcee0007170 msgr2=0x7fcee00e6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.436+0000 7fcee759e640 1 --2- 172.21.15.106:0/277733345 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcee0007170 0x7fcee00e6df0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fcee004db30 tx=0x7fced802a180 comp rx=0 tx=0).stop 2024-04-25T10:30:39.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.437+0000 7fe68affd640 1 -- 172.21.15.106:0/210111363 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68400b6c0 con 0x7fe69406cb00 2024-04-25T10:30:39.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.437+0000 7fe68affd640 1 -- 172.21.15.106:0/210111363 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe684019070 con 0x7fe69406cb00 2024-04-25T10:30:39.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.437+0000 7fe69359e640 1 -- 172.21.15.106:0/210111363 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6941f6c10 con 0x7fe69406cb00 2024-04-25T10:30:39.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.437+0000 7fe69359e640 1 -- 172.21.15.106:0/210111363 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6941f71d0 con 0x7fe69406cb00 2024-04-25T10:30:39.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.437+0000 7fcee759e640 1 -- 172.21.15.106:0/277733345 shutdown_connections 2024-04-25T10:30:39.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.437+0000 7fcee759e640 1 --2- 172.21.15.106:0/277733345 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcee0007170 0x7fcee00e6df0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.437+0000 7fcee759e640 1 --2- 172.21.15.106:0/277733345 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcee004c3d0 0x7fcee0006c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.437+0000 7fcee759e640 1 --2- 172.21.15.106:0/277733345 >> v2:172.21.15.134:3300/0 conn(0x7fcee0008410 0x7fcee004be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.437+0000 7fcee759e640 1 -- 172.21.15.106:0/277733345 >> 172.21.15.106:0/277733345 conn(0x7fcee004df90 msgr2=0x7fcee004e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:39.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.437+0000 7f4dbe573640 1 --2- 172.21.15.106:0/1526498517 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f4d9c0745b0 0x7f4d9c076ac0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4db4034000 tx=0x7f4db402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.437+0000 7ff6d2ffd640 1 -- 172.21.15.106:0/1826006075 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff6dc06c960 con 0x7ff6d4006cd0 2024-04-25T10:30:39.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.437+0000 7fe68affd640 1 -- 172.21.15.106:0/210111363 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68400ba60 con 0x7fe69406cb00 2024-04-25T10:30:39.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.437+0000 7ff6d2ffd640 1 --2- 172.21.15.106:0/1826006075 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7ff6c00788c0 0x7ff6c007add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.437+0000 7ff6e1d71640 1 --2- 172.21.15.106:0/1826006075 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7ff6c00788c0 0x7ff6c007add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.437+0000 7ff6d2ffd640 1 -- 172.21.15.106:0/1826006075 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff6dc1125a0 con 0x7ff6d4006cd0 2024-04-25T10:30:39.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.436+0000 7ff8ba7ea640 1 -- 172.21.15.106:0/1855875262 wait complete. 2024-04-25T10:30:39.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.436+0000 7fc20effd640 1 --2- 172.21.15.106:0/365299902 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc208037840 0x7fc2080da830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.438+0000 7fc20f7fe640 1 -- 172.21.15.106:0/365299902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc208037d90 msgr2=0x7fc2080d4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.438+0000 7fc20f7fe640 1 --2- 172.21.15.106:0/365299902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc208037d90 0x7fc2080d4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.438+0000 7fc20f7fe640 1 -- 172.21.15.106:0/365299902 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc208037840 msgr2=0x7fc2080da830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.438+0000 7fc20f7fe640 1 --2- 172.21.15.106:0/365299902 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc208037840 0x7fc2080da830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.438+0000 7fc20f7fe640 1 -- 172.21.15.106:0/365299902 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2080d4f60 con 0x7fc20804fb60 2024-04-25T10:30:39.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.438+0000 7ff6e1d71640 1 --2- 172.21.15.106:0/1826006075 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7ff6c00788c0 0x7ff6c007add0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff6d8035000 tx=0x7ff6d802f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.438+0000 7fc20f7fe640 1 --2- 172.21.15.106:0/365299902 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc20804fb60 0x7fc2080d1a20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc21006bf60 tx=0x7fc2100783f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.438+0000 7fcee759e640 1 -- 172.21.15.106:0/277733345 shutdown_connections 2024-04-25T10:30:39.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.438+0000 7fcee759e640 1 -- 172.21.15.106:0/277733345 wait complete. 2024-04-25T10:30:39.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.438+0000 7f6fe359e640 1 Processor -- start 2024-04-25T10:30:39.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.438+0000 7fc1f7fff640 1 -- 172.21.15.106:0/365299902 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc210092630 con 0x7fc20804fb60 2024-04-25T10:30:39.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.439+0000 7fe69359e640 1 -- 172.21.15.106:0/210111363 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe694077380 con 0x7fe69406cb00 2024-04-25T10:30:39.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.439+0000 7fc1f7fff640 1 -- 172.21.15.106:0/365299902 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc210092070 con 0x7fc20804fb60 2024-04-25T10:30:39.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.439+0000 7fc1f7fff640 1 -- 172.21.15.106:0/365299902 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc21006a610 con 0x7fc20804fb60 2024-04-25T10:30:39.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.438+0000 7f6fe359e640 1 -- start start 2024-04-25T10:30:39.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.439+0000 7fc20ffff640 1 -- 172.21.15.106:0/365299902 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2080d2200 con 0x7fc20804fb60 2024-04-25T10:30:39.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.439+0000 7fc20ffff640 1 -- 172.21.15.106:0/365299902 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc208173a40 con 0x7fc20804fb60 2024-04-25T10:30:39.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.439+0000 7ff8ba7ea640 1 Processor -- start 2024-04-25T10:30:39.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.439+0000 7fe68affd640 1 -- 172.21.15.106:0/210111363 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe68400bc00 con 0x7fe69406cb00 2024-04-25T10:30:39.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.439+0000 7ff8ba7ea640 1 -- start start 2024-04-25T10:30:39.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.440+0000 7fe68affd640 1 --2- 172.21.15.106:0/210111363 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fe674074530 0x7fe674076a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.440+0000 7fc20ffff640 1 -- 172.21.15.106:0/365299902 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2080059c0 con 0x7fc20804fb60 2024-04-25T10:30:39.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.440+0000 7fe68bfff640 1 --2- 172.21.15.106:0/210111363 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fe674074530 0x7fe674076a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.440+0000 7f6fe359e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fdc04abd0 0x7f6fdc053010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.440+0000 7f6fe359e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6fdc050ab0 0x7f6fdc0c95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.440+0000 7f6fe359e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6fdc0d9830 0x7f6fdc0dd610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.441+0000 7f6fe359e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6fdc00f390 con 0x7f6fdc0d9830 2024-04-25T10:30:39.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.441+0000 7f6fe359e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6fdc00f210 con 0x7f6fdc04abd0 2024-04-25T10:30:39.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.441+0000 7f6fe359e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6fdc0495e0 con 0x7f6fdc050ab0 2024-04-25T10:30:39.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.441+0000 7f6fe2d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6fdc0d9830 0x7f6fdc0dd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.441+0000 7f6fe2d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6fdc0d9830 0x7f6fdc0dd610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47844/0 (socket says 172.21.15.106:47844) 2024-04-25T10:30:39.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.441+0000 7f6fe2d9d640 1 -- 172.21.15.106:0/837716635 learned_addr learned my addr 172.21.15.106:0/837716635 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:39.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.441+0000 7f6fe259c640 1 --2- 172.21.15.106:0/837716635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fdc04abd0 0x7f6fdc053010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.441+0000 7fe68bfff640 1 --2- 172.21.15.106:0/210111363 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fe674074530 0x7fe674076a40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe67c034000 tx=0x7fe67c02c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.441+0000 7f6fe259c640 1 -- 172.21.15.106:0/837716635 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6fdc050ab0 msgr2=0x7f6fdc0c95a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:30:39.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.441+0000 7f6fe259c640 1 --2- 172.21.15.106:0/837716635 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6fdc050ab0 0x7f6fdc0c95a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.441+0000 7f6fe259c640 1 -- 172.21.15.106:0/837716635 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6fdc0d9830 msgr2=0x7f6fdc0dd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.441+0000 7f6fe259c640 1 --2- 172.21.15.106:0/837716635 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6fdc0d9830 0x7f6fdc0dd610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.441+0000 7f6fe259c640 1 -- 172.21.15.106:0/837716635 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fdc053550 con 0x7f6fdc04abd0 2024-04-25T10:30:39.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.441+0000 7fc1f7fff640 1 -- 172.21.15.106:0/365299902 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc21006d740 con 0x7fc20804fb60 2024-04-25T10:30:39.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.441+0000 7fcee759e640 1 Processor -- start 2024-04-25T10:30:39.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.441+0000 7f6fe259c640 1 --2- 172.21.15.106:0/837716635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fdc04abd0 0x7f6fdc053010 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6fd403a000 tx=0x7f6fd40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.441+0000 7fcee759e640 1 -- start start 2024-04-25T10:30:39.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.442+0000 7f6fe2d9d640 1 --2- 172.21.15.106:0/837716635 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6fdc0d9830 0x7f6fdc0dd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:39.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.442+0000 7fc1f7fff640 1 --2- 172.21.15.106:0/365299902 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fc1fc0744e0 0x7fc1fc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.442+0000 7ff8ba7ea640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8b40ce5b0 0x7ff8b415b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.442+0000 7fc1f7fff640 1 -- 172.21.15.106:0/365299902 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc210119db0 con 0x7fc20804fb60 2024-04-25T10:30:39.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.442+0000 7ff8ba7ea640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8b4156ce0 0x7ff8b4155520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.442+0000 7ff8b97e8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8b40ce5b0 0x7ff8b415b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.442+0000 7ff8b97e8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8b40ce5b0 0x7ff8b415b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.106:48022/0 (socket says 172.21.15.106:48022) 2024-04-25T10:30:39.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.442+0000 7ff8b97e8640 1 -- 172.21.15.106:0/2847170755 learned_addr learned my addr 172.21.15.106:0/2847170755 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:39.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.442+0000 7ff8b8fe7640 1 --2- 172.21.15.106:0/2847170755 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8b4156ce0 0x7ff8b4155520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.442+0000 7ff8ba7ea640 1 --2- 172.21.15.106:0/2847170755 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8b415d340 0x7ff8b4155a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.442+0000 7ff8b9fe9640 1 --2- 172.21.15.106:0/2847170755 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8b415d340 0x7ff8b4155a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.442+0000 7ff8ba7ea640 1 -- 172.21.15.106:0/2847170755 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b4160660 con 0x7ff8b415d340 2024-04-25T10:30:39.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.442+0000 7ff8ba7ea640 1 -- 172.21.15.106:0/2847170755 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b4160380 con 0x7ff8b40ce5b0 2024-04-25T10:30:39.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.442+0000 7ff8ba7ea640 1 -- 172.21.15.106:0/2847170755 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff8b4160940 con 0x7ff8b4156ce0 2024-04-25T10:30:39.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.443+0000 7fcee759e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcee0007170 0x7fcee00d7a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.443+0000 7fcee759e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcee0008410 0x7fcee00e03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.443+0000 7fcee759e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcee004c3d0 0x7fcee00d7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.443+0000 7fcee759e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fcee00e3fd0 con 0x7fcee0008410 2024-04-25T10:30:39.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.443+0000 7fcee759e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcee00e3cf0 con 0x7fcee004c3d0 2024-04-25T10:30:39.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.443+0000 7fcee759e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fcee0047bd0 con 0x7fcee0007170 2024-04-25T10:30:39.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.443+0000 7fc20effd640 1 --2- 172.21.15.106:0/365299902 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fc1fc0744e0 0x7fc1fc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.443+0000 7fcee5d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcee0008410 0x7fcee00e03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.443+0000 7fcee5d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcee0008410 0x7fcee00e03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47864/0 (socket says 172.21.15.106:47864) 2024-04-25T10:30:39.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.443+0000 7fcee5d9b640 1 -- 172.21.15.106:0/3049692349 learned_addr learned my addr 172.21.15.106:0/3049692349 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:39.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.443+0000 7fcee659c640 1 --2- 172.21.15.106:0/3049692349 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcee0007170 0x7fcee00d7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.443+0000 7fe68affd640 1 -- 172.21.15.106:0/210111363 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe684093b50 con 0x7fe69406cb00 2024-04-25T10:30:39.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.443+0000 7fcee6d9d640 1 --2- 172.21.15.106:0/3049692349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcee004c3d0 0x7fcee00d7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.443+0000 7ff8b8fe7640 1 -- 172.21.15.106:0/2847170755 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8b40ce5b0 msgr2=0x7ff8b415b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.443+0000 7ff8b8fe7640 1 --2- 172.21.15.106:0/2847170755 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8b40ce5b0 0x7ff8b415b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.443+0000 7ff8b8fe7640 1 -- 172.21.15.106:0/2847170755 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8b415d340 msgr2=0x7ff8b4155a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.443+0000 7ff8b8fe7640 1 --2- 172.21.15.106:0/2847170755 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8b415d340 0x7ff8b4155a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.443+0000 7ff8b8fe7640 1 -- 172.21.15.106:0/2847170755 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8b41f1ba0 con 0x7ff8b4156ce0 2024-04-25T10:30:39.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.443+0000 7ff8b9fe9640 1 --2- 172.21.15.106:0/2847170755 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8b415d340 0x7ff8b4155a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:39.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.443+0000 7ff6d2ffd640 1 -- 172.21.15.106:0/1826006075 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff6dc0dc640 con 0x7ff6d4006cd0 2024-04-25T10:30:39.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.443+0000 7fcee659c640 1 -- 172.21.15.106:0/3049692349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcee004c3d0 msgr2=0x7fcee00d7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.443+0000 7fcee659c640 1 --2- 172.21.15.106:0/3049692349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcee004c3d0 0x7fcee00d7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.443+0000 7fcee659c640 1 -- 172.21.15.106:0/3049692349 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcee0008410 msgr2=0x7fcee00e03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.443+0000 7fcee659c640 1 --2- 172.21.15.106:0/3049692349 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcee0008410 0x7fcee00e03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.443+0000 7fcee659c640 1 -- 172.21.15.106:0/3049692349 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcee00e0900 con 0x7fcee0007170 2024-04-25T10:30:39.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.443+0000 7fc20effd640 1 --2- 172.21.15.106:0/365299902 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fc1fc0744e0 0x7fc1fc0769f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc204034000 tx=0x7fc20402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.443+0000 7f6fd2ffd640 1 -- 172.21.15.106:0/837716635 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd404e070 con 0x7f6fdc04abd0 2024-04-25T10:30:39.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.443+0000 7f6fe359e640 1 -- 172.21.15.106:0/837716635 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fdc0de500 con 0x7f6fdc04abd0 2024-04-25T10:30:39.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.444+0000 7f6fe359e640 1 -- 172.21.15.106:0/837716635 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fdc0deac0 con 0x7f6fdc04abd0 2024-04-25T10:30:39.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.444+0000 7fcee659c640 1 --2- 172.21.15.106:0/3049692349 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcee0007170 0x7fcee00d7a30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fcedc00e840 tx=0x7fcedc00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.444+0000 7ff8b8fe7640 1 --2- 172.21.15.106:0/2847170755 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8b4156ce0 0x7ff8b4155520 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff8a400c3f0 tx=0x7ff8a400c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.444+0000 7fb18dffb640 1 -- 172.21.15.106:0/2280543230 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb19806d950 con 0x7fb190047bb0 2024-04-25T10:30:39.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.444+0000 7fb18dffb640 1 -- 172.21.15.106:0/2280543230 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb198092020 con 0x7fb190047bb0 2024-04-25T10:30:39.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.444+0000 7fb18dffb640 1 -- 172.21.15.106:0/2280543230 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb19806dd70 con 0x7fb190047bb0 2024-04-25T10:30:39.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.444+0000 7f4daeffd640 1 -- 172.21.15.106:0/1526498517 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4db80dc8f0 con 0x7f4db0050f00 2024-04-25T10:30:39.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.444+0000 7fb19e763640 1 -- 172.21.15.106:0/2280543230 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb190030f40 con 0x7fb190047bb0 2024-04-25T10:30:39.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.444+0000 7fb19e763640 1 -- 172.21.15.106:0/2280543230 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1900323a0 con 0x7fb190047bb0 2024-04-25T10:30:39.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.445+0000 7f6fe359e640 1 -- 172.21.15.106:0/837716635 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fdc0059c0 con 0x7f6fdc04abd0 2024-04-25T10:30:39.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.446+0000 7fb19e763640 1 -- 172.21.15.106:0/2280543230 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb190048820 con 0x7fb190047bb0 2024-04-25T10:30:39.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.446+0000 7fb18dffb640 1 -- 172.21.15.106:0/2280543230 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb19808e6b0 con 0x7fb190047bb0 2024-04-25T10:30:39.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.447+0000 7fb18dffb640 1 --2- 172.21.15.106:0/2280543230 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fb17c0744e0 0x7fb17c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.447+0000 7fb18dffb640 1 -- 172.21.15.106:0/2280543230 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb198111fd0 con 0x7fb190047bb0 2024-04-25T10:30:39.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.447+0000 7fb19d761640 1 --2- 172.21.15.106:0/2280543230 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fb17c0744e0 0x7fb17c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.447+0000 7ff8a9ffb640 1 -- 172.21.15.106:0/2847170755 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a40074e0 con 0x7ff8b4156ce0 2024-04-25T10:30:39.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.447+0000 7fced6ffd640 1 -- 172.21.15.106:0/3049692349 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcedc017070 con 0x7fcee0007170 2024-04-25T10:30:39.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.447+0000 7fced6ffd640 1 -- 172.21.15.106:0/3049692349 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcedc009e10 con 0x7fcee0007170 2024-04-25T10:30:39.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.448+0000 7fb19d761640 1 --2- 172.21.15.106:0/2280543230 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fb17c0744e0 0x7fb17c0769f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb188034000 tx=0x7fb18802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.447+0000 7ff8ba7ea640 1 -- 172.21.15.106:0/2847170755 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8b414e330 con 0x7ff8b4156ce0 2024-04-25T10:30:39.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.447+0000 7fcee759e640 1 -- 172.21.15.106:0/3049692349 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcee0179a90 con 0x7fcee0007170 2024-04-25T10:30:39.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.447+0000 7fcee759e640 1 -- 172.21.15.106:0/3049692349 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fcee017a050 con 0x7fcee0007170 2024-04-25T10:30:39.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.448+0000 7fced6ffd640 1 -- 172.21.15.106:0/3049692349 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcedc00cb00 con 0x7fcee0007170 2024-04-25T10:30:39.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.448+0000 7fc1f7fff640 1 -- 172.21.15.106:0/365299902 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc2100a3070 con 0x7fc20804fb60 2024-04-25T10:30:39.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.448+0000 7ff8a9ffb640 1 -- 172.21.15.106:0/2847170755 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8a4007680 con 0x7ff8b4156ce0 2024-04-25T10:30:39.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.450+0000 7fe68affd640 1 -- 172.21.15.106:0/210111363 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe68405fd30 con 0x7fe69406cb00 2024-04-25T10:30:39.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.449+0000 7f6fd2ffd640 1 -- 172.21.15.106:0/837716635 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fd4037840 con 0x7f6fdc04abd0 2024-04-25T10:30:39.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.450+0000 7f6fd2ffd640 1 -- 172.21.15.106:0/837716635 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd403b070 con 0x7f6fdc04abd0 2024-04-25T10:30:39.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.448+0000 7ff8a9ffb640 1 -- 172.21.15.106:0/2847170755 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a4004a20 con 0x7ff8b4156ce0 2024-04-25T10:30:39.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.451+0000 7fced6ffd640 1 -- 172.21.15.106:0/3049692349 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fcedc01a070 con 0x7fcee0007170 2024-04-25T10:30:39.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.451+0000 7fcee759e640 1 -- 172.21.15.106:0/3049692349 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcecc005290 con 0x7fcee0007170 2024-04-25T10:30:39.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.452+0000 7fced6ffd640 1 --2- 172.21.15.106:0/3049692349 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fcec8074580 0x7fcec8076a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.452+0000 7fcee5d9b640 1 --2- 172.21.15.106:0/3049692349 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fcec8074580 0x7fcec8076a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.452+0000 7fced6ffd640 1 -- 172.21.15.106:0/3049692349 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fcedc095260 con 0x7fcee0007170 2024-04-25T10:30:39.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.448+0000 7ff8ba7ea640 1 -- 172.21.15.106:0/2847170755 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8b41f2140 con 0x7ff8b4156ce0 2024-04-25T10:30:39.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.449+0000 7ff8ba7ea640 1 -- 172.21.15.106:0/2847170755 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8b4050740 con 0x7ff8b4156ce0 2024-04-25T10:30:39.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.451+0000 7ff8a9ffb640 1 -- 172.21.15.106:0/2847170755 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff8a401a020 con 0x7ff8b4156ce0 2024-04-25T10:30:39.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.451+0000 7ff8a9ffb640 1 --2- 172.21.15.106:0/2847170755 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7ff8980744b0 0x7ff8980769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.452+0000 7ff8b97e8640 1 --2- 172.21.15.106:0/2847170755 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7ff8980744b0 0x7ff8980769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.452+0000 7f6fd2ffd640 1 -- 172.21.15.106:0/837716635 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6fd4051070 con 0x7f6fdc04abd0 2024-04-25T10:30:39.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.453+0000 7f6fd2ffd640 1 --2- 172.21.15.106:0/837716635 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f6fc40746b0 0x7f6fc4076bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.452+0000 7ff8a9ffb640 1 -- 172.21.15.106:0/2847170755 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff8a4094970 con 0x7ff8b4156ce0 2024-04-25T10:30:39.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.453+0000 7fcee5d9b640 1 --2- 172.21.15.106:0/3049692349 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fcec8074580 0x7fcec8076a90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcee806c1a0 tx=0x7fcee8090040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.453+0000 7f6fe1d9b640 1 --2- 172.21.15.106:0/837716635 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f6fc40746b0 0x7f6fc4076bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.453+0000 7f6fd2ffd640 1 -- 172.21.15.106:0/837716635 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6fd4003820 con 0x7f6fdc04abd0 2024-04-25T10:30:39.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.453+0000 7ff8b97e8640 1 --2- 172.21.15.106:0/2847170755 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7ff8980744b0 0x7ff8980769c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff8b0009560 tx=0x7ff8b0008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.454+0000 7f6fd2ffd640 1 -- 172.21.15.106:0/837716635 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6fd4046070 con 0x7f6fdc04abd0 2024-04-25T10:30:39.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.454+0000 7f6fe1d9b640 1 --2- 172.21.15.106:0/837716635 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f6fc40746b0 0x7f6fc4076bc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc171f30 tx=0x7f6fd8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.457+0000 7ff8a9ffb640 1 -- 172.21.15.106:0/2847170755 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff8b40cf1d0 con 0x7ff8b4156ce0 2024-04-25T10:30:39.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.457+0000 7fb18dffb640 1 -- 172.21.15.106:0/2280543230 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb19809c030 con 0x7fb190047bb0 2024-04-25T10:30:39.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.462+0000 7fced6ffd640 1 -- 172.21.15.106:0/3049692349 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcedc05f380 con 0x7fcee0007170 2024-04-25T10:30:39.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.858+0000 7ff6e3574640 1 -- 172.21.15.106:0/1826006075 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7ff6d40d91a0 con 0x7ff6d4006cd0 2024-04-25T10:30:39.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.859+0000 7ff6d2ffd640 1 -- 172.21.15.106:0/1826006075 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff6dc0e0940 con 0x7ff6d4006cd0 2024-04-25T10:30:39.861 INFO:teuthology.orchestra.run.smithi106.stdout:55834574854 2024-04-25T10:30:39.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.862+0000 7ff6d0ff9640 1 -- 172.21.15.106:0/1826006075 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7ff6c00788c0 msgr2=0x7ff6c007add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.862+0000 7ff6d0ff9640 1 --2- 172.21.15.106:0/1826006075 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7ff6c00788c0 0x7ff6c007add0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff6d8035000 tx=0x7ff6d802f000 comp rx=0 tx=0).stop 2024-04-25T10:30:39.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.862+0000 7ff6d0ff9640 1 -- 172.21.15.106:0/1826006075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6d4006cd0 msgr2=0x7ff6d4047f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.862+0000 7ff6d0ff9640 1 --2- 172.21.15.106:0/1826006075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6d4006cd0 0x7ff6d4047f80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc076d10 tx=0x7ff6dc0783f0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.863+0000 7ff6d0ff9640 1 -- 172.21.15.106:0/1826006075 shutdown_connections 2024-04-25T10:30:39.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.863+0000 7ff6d0ff9640 1 --2- 172.21.15.106:0/1826006075 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7ff6c00788c0 0x7ff6c007add0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.863+0000 7ff6d0ff9640 1 --2- 172.21.15.106:0/1826006075 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6d404d1c0 0x7ff6d4175950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.863+0000 7ff6d0ff9640 1 --2- 172.21.15.106:0/1826006075 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6d4045be0 0x7ff6d4048f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.863+0000 7ff6d0ff9640 1 --2- 172.21.15.106:0/1826006075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6d4006cd0 0x7ff6d4047f80 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.863+0000 7ff6d0ff9640 1 -- 172.21.15.106:0/1826006075 >> 172.21.15.106:0/1826006075 conn(0x7ff6d4050c40 msgr2=0x7ff6d4035400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:39.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.863+0000 7ff6d0ff9640 1 -- 172.21.15.106:0/1826006075 shutdown_connections 2024-04-25T10:30:39.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.863+0000 7ff6d0ff9640 1 -- 172.21.15.106:0/1826006075 wait complete. 2024-04-25T10:30:39.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.868+0000 7fc20ffff640 1 -- 172.21.15.106:0/365299902 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fc20804e380 con 0x7fc20804fb60 2024-04-25T10:30:39.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.868+0000 7fc1f7fff640 1 -- 172.21.15.106:0/365299902 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc2100e4660 con 0x7fc20804fb60 2024-04-25T10:30:39.871 INFO:teuthology.orchestra.run.smithi106.stdout:55834574854 2024-04-25T10:30:39.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.869+0000 7f4dc07d7640 1 -- 172.21.15.106:0/1526498517 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f4db00480e0 con 0x7f4db0050f00 2024-04-25T10:30:39.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.870+0000 7f4daeffd640 1 -- 172.21.15.106:0/1526498517 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4db80dc1f0 con 0x7f4db0050f00 2024-04-25T10:30:39.872 INFO:teuthology.orchestra.run.smithi106.stdout:55834574854 2024-04-25T10:30:39.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.872+0000 7fc1f5ffb640 1 -- 172.21.15.106:0/365299902 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fc1fc0744e0 msgr2=0x7fc1fc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.872+0000 7fc1f5ffb640 1 --2- 172.21.15.106:0/365299902 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fc1fc0744e0 0x7fc1fc0769f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc204034000 tx=0x7fc20402e000 comp rx=0 tx=0).stop 2024-04-25T10:30:39.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.872+0000 7fc1f5ffb640 1 -- 172.21.15.106:0/365299902 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc20804fb60 msgr2=0x7fc2080d1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.872+0000 7fc1f5ffb640 1 --2- 172.21.15.106:0/365299902 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc20804fb60 0x7fc2080d1a20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc21006bf60 tx=0x7fc2100783f0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.872+0000 7fc1f5ffb640 1 -- 172.21.15.106:0/365299902 shutdown_connections 2024-04-25T10:30:39.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.872+0000 7fc1f5ffb640 1 --2- 172.21.15.106:0/365299902 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fc1fc0744e0 0x7fc1fc0769f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.872+0000 7fc1f5ffb640 1 --2- 172.21.15.106:0/365299902 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc20804fb60 0x7fc2080d1a20 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.872+0000 7fc1f5ffb640 1 --2- 172.21.15.106:0/365299902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc208037d90 0x7fc2080d4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.872+0000 7fc1f5ffb640 1 --2- 172.21.15.106:0/365299902 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc208037840 0x7fc2080da830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.873+0000 7fc1f5ffb640 1 -- 172.21.15.106:0/365299902 >> 172.21.15.106:0/365299902 conn(0x7fc20800e2e0 msgr2=0x7fc2080db300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:39.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.873+0000 7fc1f5ffb640 1 -- 172.21.15.106:0/365299902 shutdown_connections 2024-04-25T10:30:39.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.873+0000 7fc1f5ffb640 1 -- 172.21.15.106:0/365299902 wait complete. 2024-04-25T10:30:39.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.875+0000 7f4dc07d7640 1 -- 172.21.15.106:0/1526498517 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f4d9c0745b0 msgr2=0x7f4d9c076ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.875+0000 7f4dc07d7640 1 --2- 172.21.15.106:0/1526498517 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f4d9c0745b0 0x7f4d9c076ac0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4db4034000 tx=0x7f4db402e000 comp rx=0 tx=0).stop 2024-04-25T10:30:39.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.875+0000 7f4dc07d7640 1 -- 172.21.15.106:0/1526498517 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4db0050f00 msgr2=0x7f4db00db660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.876+0000 7f4dc07d7640 1 --2- 172.21.15.106:0/1526498517 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4db0050f00 0x7f4db00db660 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4db80519b0 tx=0x7f4db80783f0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.875+0000 7fe69359e640 1 -- 172.21.15.106:0/210111363 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fe6940cf180 con 0x7fe69406cb00 2024-04-25T10:30:39.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.876+0000 7f4dc07d7640 1 -- 172.21.15.106:0/1526498517 shutdown_connections 2024-04-25T10:30:39.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.876+0000 7f4dc07d7640 1 --2- 172.21.15.106:0/1526498517 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f4d9c0745b0 0x7f4d9c076ac0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.876+0000 7f4dc07d7640 1 --2- 172.21.15.106:0/1526498517 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4db00d5720 0x7f4db0179710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.876+0000 7f4dc07d7640 1 --2- 172.21.15.106:0/1526498517 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4db0050f00 0x7f4db00db660 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.876+0000 7f4dc07d7640 1 --2- 172.21.15.106:0/1526498517 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4db0033510 0x7f4db00d7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.876+0000 7fe68affd640 1 -- 172.21.15.106:0/210111363 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe68405f630 con 0x7fe69406cb00 2024-04-25T10:30:39.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.876+0000 7f4dc07d7640 1 -- 172.21.15.106:0/1526498517 >> 172.21.15.106:0/1526498517 conn(0x7f4db00503e0 msgr2=0x7f4db004c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:39.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.876+0000 7f4dc07d7640 1 -- 172.21.15.106:0/1526498517 shutdown_connections 2024-04-25T10:30:39.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.876+0000 7f4dc07d7640 1 -- 172.21.15.106:0/1526498517 wait complete. 2024-04-25T10:30:39.880 INFO:teuthology.orchestra.run.smithi106.stdout:55834574855 2024-04-25T10:30:39.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.876+0000 7f6fe359e640 1 -- 172.21.15.106:0/837716635 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f6fdc050890 con 0x7f6fdc04abd0 2024-04-25T10:30:39.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.877+0000 7f6fd2ffd640 1 -- 172.21.15.106:0/837716635 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6fd4088180 con 0x7f6fdc04abd0 2024-04-25T10:30:39.881 INFO:teuthology.orchestra.run.smithi106.stdout:55834574855 2024-04-25T10:30:39.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.877+0000 7ff8ba7ea640 1 -- 172.21.15.106:0/2847170755 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7ff8b40cdaf0 con 0x7ff8b4156ce0 2024-04-25T10:30:39.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.878+0000 7ff8a9ffb640 1 -- 172.21.15.106:0/2847170755 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff8a40670e0 con 0x7ff8b4156ce0 2024-04-25T10:30:39.882 INFO:teuthology.orchestra.run.smithi106.stdout:55834574854 2024-04-25T10:30:39.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.880+0000 7fe69359e640 1 -- 172.21.15.106:0/210111363 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fe674074530 msgr2=0x7fe674076a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.880+0000 7fe69359e640 1 --2- 172.21.15.106:0/210111363 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fe674074530 0x7fe674076a40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe67c034000 tx=0x7fe67c02c000 comp rx=0 tx=0).stop 2024-04-25T10:30:39.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.879+0000 7fb19e763640 1 -- 172.21.15.106:0/2280543230 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fb19017a180 con 0x7fb190047bb0 2024-04-25T10:30:39.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.880+0000 7fb18dffb640 1 -- 172.21.15.106:0/2280543230 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb1980dc6c0 con 0x7fb190047bb0 2024-04-25T10:30:39.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.880+0000 7fe69359e640 1 -- 172.21.15.106:0/210111363 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe69406cb00 msgr2=0x7fe69415bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.880+0000 7fe69359e640 1 --2- 172.21.15.106:0/210111363 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe69406cb00 0x7fe69415bb80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe68400d650 tx=0x7fe68400db10 comp rx=0 tx=0).stop 2024-04-25T10:30:39.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.880+0000 7fe69359e640 1 -- 172.21.15.106:0/210111363 shutdown_connections 2024-04-25T10:30:39.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.880+0000 7fe69359e640 1 --2- 172.21.15.106:0/210111363 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fe674074530 0x7fe674076a40 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.880+0000 7fe69359e640 1 --2- 172.21.15.106:0/210111363 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe69415ac40 0x7fe69414f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.880+0000 7fe69359e640 1 --2- 172.21.15.106:0/210111363 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe6940ceb70 0x7fe694154f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.880+0000 7fe69359e640 1 --2- 172.21.15.106:0/210111363 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe69406cb00 0x7fe69415bb80 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.880+0000 7fe69359e640 1 -- 172.21.15.106:0/210111363 >> 172.21.15.106:0/210111363 conn(0x7fe694070f00 msgr2=0x7fe6940cd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:39.884 INFO:teuthology.orchestra.run.smithi106.stdout:55834574854 2024-04-25T10:30:39.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.881+0000 7f6fe359e640 1 -- 172.21.15.106:0/837716635 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f6fc40746b0 msgr2=0x7f6fc4076bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.881+0000 7f6fe359e640 1 --2- 172.21.15.106:0/837716635 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f6fc40746b0 0x7f6fc4076bc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc171f30 tx=0x7f6fd8008040 comp rx=0 tx=0).stop 2024-04-25T10:30:39.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.881+0000 7f6fe359e640 1 -- 172.21.15.106:0/837716635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fdc04abd0 msgr2=0x7f6fdc053010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.881+0000 7f6fe359e640 1 --2- 172.21.15.106:0/837716635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fdc04abd0 0x7f6fdc053010 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6fd403a000 tx=0x7f6fd40046c0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.881+0000 7f6fe359e640 1 -- 172.21.15.106:0/837716635 shutdown_connections 2024-04-25T10:30:39.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.881+0000 7f6fe359e640 1 --2- 172.21.15.106:0/837716635 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f6fc40746b0 0x7f6fc4076bc0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.881+0000 7f6fe359e640 1 --2- 172.21.15.106:0/837716635 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6fdc0d9830 0x7f6fdc0dd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.881+0000 7f6fe359e640 1 --2- 172.21.15.106:0/837716635 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6fdc050ab0 0x7f6fdc0c95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.887 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.7 2024-04-25T10:30:39.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.882+0000 7ff8ba7ea640 1 -- 172.21.15.106:0/2847170755 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7ff8980744b0 msgr2=0x7ff8980769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.882+0000 7ff8ba7ea640 1 --2- 172.21.15.106:0/2847170755 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7ff8980744b0 0x7ff8980769c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff8b0009560 tx=0x7ff8b0008030 comp rx=0 tx=0).stop 2024-04-25T10:30:39.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.881+0000 7f6fe359e640 1 --2- 172.21.15.106:0/837716635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fdc04abd0 0x7f6fdc053010 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.881+0000 7f6fe359e640 1 -- 172.21.15.106:0/837716635 >> 172.21.15.106:0/837716635 conn(0x7f6fdc00e360 msgr2=0x7f6fdc038d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:39.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.882+0000 7ff8ba7ea640 1 -- 172.21.15.106:0/2847170755 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8b4156ce0 msgr2=0x7ff8b4155520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.882+0000 7ff8ba7ea640 1 --2- 172.21.15.106:0/2847170755 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8b4156ce0 0x7ff8b4155520 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff8a400c3f0 tx=0x7ff8a400c8b0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.881+0000 7f6fe359e640 1 -- 172.21.15.106:0/837716635 shutdown_connections 2024-04-25T10:30:39.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.881+0000 7f6fe359e640 1 -- 172.21.15.106:0/837716635 wait complete. 2024-04-25T10:30:39.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.882+0000 7ff8ba7ea640 1 -- 172.21.15.106:0/2847170755 shutdown_connections 2024-04-25T10:30:39.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.882+0000 7ff8ba7ea640 1 --2- 172.21.15.106:0/2847170755 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7ff8980744b0 0x7ff8980769c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.882+0000 7ff8ba7ea640 1 --2- 172.21.15.106:0/2847170755 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8b415d340 0x7ff8b4155a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.882+0000 7ff8ba7ea640 1 --2- 172.21.15.106:0/2847170755 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8b4156ce0 0x7ff8b4155520 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.882+0000 7ff8ba7ea640 1 --2- 172.21.15.106:0/2847170755 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8b40ce5b0 0x7ff8b415b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.882+0000 7ff8ba7ea640 1 -- 172.21.15.106:0/2847170755 >> 172.21.15.106:0/2847170755 conn(0x7ff8b406d770 msgr2=0x7ff8b415bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:39.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.882+0000 7ff8ba7ea640 1 -- 172.21.15.106:0/2847170755 shutdown_connections 2024-04-25T10:30:39.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.882+0000 7fe69359e640 1 -- 172.21.15.106:0/210111363 shutdown_connections 2024-04-25T10:30:39.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.882+0000 7ff8ba7ea640 1 -- 172.21.15.106:0/2847170755 wait complete. 2024-04-25T10:30:39.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.882+0000 7fe69359e640 1 -- 172.21.15.106:0/210111363 wait complete. 2024-04-25T10:30:39.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.883+0000 7fcee759e640 1 -- 172.21.15.106:0/3049692349 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fcecc005580 con 0x7fcee0007170 2024-04-25T10:30:39.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.883+0000 7fced6ffd640 1 -- 172.21.15.106:0/3049692349 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fcedc063680 con 0x7fcee0007170 2024-04-25T10:30:39.891 INFO:teuthology.orchestra.run.smithi106.stdout:55834574854 2024-04-25T10:30:39.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.884+0000 7fb19e763640 1 -- 172.21.15.106:0/2280543230 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fb17c0744e0 msgr2=0x7fb17c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.884+0000 7fb19e763640 1 --2- 172.21.15.106:0/2280543230 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fb17c0744e0 0x7fb17c0769f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb188034000 tx=0x7fb18802e000 comp rx=0 tx=0).stop 2024-04-25T10:30:39.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.884+0000 7fb19e763640 1 -- 172.21.15.106:0/2280543230 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb190047bb0 msgr2=0x7fb19004b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.884+0000 7fb19e763640 1 --2- 172.21.15.106:0/2280543230 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb190047bb0 0x7fb19004b020 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb198075210 tx=0x7fb1980783f0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.884+0000 7fb19e763640 1 -- 172.21.15.106:0/2280543230 shutdown_connections 2024-04-25T10:30:39.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.884+0000 7fb19e763640 1 --2- 172.21.15.106:0/2280543230 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fb17c0744e0 0x7fb17c0769f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.884+0000 7fb19e763640 1 --2- 172.21.15.106:0/2280543230 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb190047bb0 0x7fb19004b020 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.884+0000 7fb19e763640 1 --2- 172.21.15.106:0/2280543230 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb190009490 0x7fb1901765a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.884+0000 7fb19e763640 1 --2- 172.21.15.106:0/2280543230 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb190008460 0x7fb1900d9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.884+0000 7fb19e763640 1 -- 172.21.15.106:0/2280543230 >> 172.21.15.106:0/2280543230 conn(0x7fb190051540 msgr2=0x7fb190005d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:39.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.885+0000 7fb19e763640 1 -- 172.21.15.106:0/2280543230 shutdown_connections 2024-04-25T10:30:39.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.885+0000 7fb19e763640 1 -- 172.21.15.106:0/2280543230 wait complete. 2024-04-25T10:30:39.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.886+0000 7fcee759e640 1 -- 172.21.15.106:0/3049692349 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fcec8074580 msgr2=0x7fcec8076a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.886+0000 7fcee759e640 1 --2- 172.21.15.106:0/3049692349 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fcec8074580 0x7fcec8076a90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcee806c1a0 tx=0x7fcee8090040 comp rx=0 tx=0).stop 2024-04-25T10:30:39.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.886+0000 7fcee759e640 1 -- 172.21.15.106:0/3049692349 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcee0007170 msgr2=0x7fcee00d7a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.886+0000 7fcee759e640 1 --2- 172.21.15.106:0/3049692349 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcee0007170 0x7fcee00d7a30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fcedc00e840 tx=0x7fcedc00ed00 comp rx=0 tx=0).stop 2024-04-25T10:30:39.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.887+0000 7fcee759e640 1 -- 172.21.15.106:0/3049692349 shutdown_connections 2024-04-25T10:30:39.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.887+0000 7fcee759e640 1 --2- 172.21.15.106:0/3049692349 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fcec8074580 0x7fcec8076a90 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.887+0000 7fcee759e640 1 --2- 172.21.15.106:0/3049692349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcee004c3d0 0x7fcee00d7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.887+0000 7fcee759e640 1 --2- 172.21.15.106:0/3049692349 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcee0008410 0x7fcee00e03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.887+0000 7fcee759e640 1 --2- 172.21.15.106:0/3049692349 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcee0007170 0x7fcee00d7a30 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.887+0000 7fcee759e640 1 -- 172.21.15.106:0/3049692349 >> 172.21.15.106:0/3049692349 conn(0x7fcee004df90 msgr2=0x7fcee004bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:39.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.887+0000 7fcee759e640 1 -- 172.21.15.106:0/3049692349 shutdown_connections 2024-04-25T10:30:39.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:39.887+0000 7fcee759e640 1 -- 172.21.15.106:0/3049692349 wait complete. 2024-04-25T10:30:39.897 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.0 2024-04-25T10:30:39.903 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.1 2024-04-25T10:30:39.904 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.2 2024-04-25T10:30:39.905 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.4 2024-04-25T10:30:39.906 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.5 2024-04-25T10:30:39.906 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.6 2024-04-25T10:30:39.912 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.3 2024-04-25T10:30:40.888 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T10:30:40.898 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T10:30:40.904 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T10:30:40.905 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T10:30:40.905 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T10:30:40.906 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T10:30:40.907 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T10:30:40.913 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T10:30:41.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.055+0000 7f09f7534640 1 Processor -- start 2024-04-25T10:30:41.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.056+0000 7f09f7534640 1 -- start start 2024-04-25T10:30:41.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.060+0000 7f09f7534640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09f0072870 0x7f09f015b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.060+0000 7f09f7534640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f09f00ca120 0x7f09f00c94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.062+0000 7f09f7534640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f09f00c99f0 0x7f09f006ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.062+0000 7f09f4acf640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f09f00ca120 0x7f09f00c94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:41.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.062+0000 7f09f4acf640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f09f00ca120 0x7f09f00c94b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47866/0 (socket says 172.21.15.106:47866) 2024-04-25T10:30:41.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.062+0000 7f09f52d0640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09f0072870 0x7f09f015b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:41.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.062+0000 7f09f52d0640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09f0072870 0x7f09f015b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.106:49096/0 (socket says 172.21.15.106:49096) 2024-04-25T10:30:41.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.063+0000 7f09f4acf640 1 -- 172.21.15.106:0/1046589391 learned_addr learned my addr 172.21.15.106:0/1046589391 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:41.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.063+0000 7f09f5ad1640 1 --2- 172.21.15.106:0/1046589391 >> v2:172.21.15.134:3300/0 conn(0x7f09f00c99f0 0x7f09f006ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:41.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.064+0000 7f09f7534640 1 -- 172.21.15.106:0/1046589391 --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f09f00c8780 con 0x7f09f00ca120 2024-04-25T10:30:41.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.064+0000 7f09f7534640 1 -- 172.21.15.106:0/1046589391 --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f09f0161b70 con 0x7f09f00c99f0 2024-04-25T10:30:41.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.064+0000 7f09f7534640 1 -- 172.21.15.106:0/1046589391 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f09f0161e50 con 0x7f09f0072870 2024-04-25T10:30:41.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.065+0000 7f09f5ad1640 1 -- 172.21.15.106:0/1046589391 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09f0072870 msgr2=0x7f09f015b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:41.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.065+0000 7f09f5ad1640 1 --2- 172.21.15.106:0/1046589391 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09f0072870 0x7f09f015b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.065+0000 7f09f5ad1640 1 -- 172.21.15.106:0/1046589391 >> v2:172.21.15.106:3300/0 conn(0x7f09f00ca120 msgr2=0x7f09f00c94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:41.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.065+0000 7f09f5ad1640 1 --2- 172.21.15.106:0/1046589391 >> v2:172.21.15.106:3300/0 conn(0x7f09f00ca120 0x7f09f00c94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.065+0000 7f09f5ad1640 1 -- 172.21.15.106:0/1046589391 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09f00c89d0 con 0x7f09f00c99f0 2024-04-25T10:30:41.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.068+0000 7f09f5ad1640 1 --2- 172.21.15.106:0/1046589391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09f00c99f0 0x7f09f006ffd0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f09f0072800 tx=0x7f09e802a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ebfffcb0367b30 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:41.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.069+0000 7f09e7fff640 1 -- 172.21.15.106:0/1046589391 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e8031600 con 0x7f09f00c99f0 2024-04-25T10:30:41.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.069+0000 7f09e7fff640 1 -- 172.21.15.106:0/1046589391 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09e80317a0 con 0x7f09f00c99f0 2024-04-25T10:30:41.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.069+0000 7f09e7fff640 1 -- 172.21.15.106:0/1046589391 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e8031b70 con 0x7f09f00c99f0 2024-04-25T10:30:41.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.072+0000 7f09f7534640 1 -- 172.21.15.106:0/1046589391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09f00c99f0 msgr2=0x7f09f006ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:41.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.072+0000 7f09f7534640 1 --2- 172.21.15.106:0/1046589391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09f00c99f0 0x7f09f006ffd0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f09f0072800 tx=0x7f09e802a5d0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.073+0000 7f09f7534640 1 -- 172.21.15.106:0/1046589391 shutdown_connections 2024-04-25T10:30:41.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.073+0000 7f09f7534640 1 --2- 172.21.15.106:0/1046589391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09f00c99f0 0x7f09f006ffd0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.073+0000 7f09f7534640 1 --2- 172.21.15.106:0/1046589391 >> v2:172.21.15.106:3300/0 conn(0x7f09f00ca120 0x7f09f00c94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.073+0000 7f09f7534640 1 --2- 172.21.15.106:0/1046589391 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09f0072870 0x7f09f015b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.073+0000 7f09f7534640 1 -- 172.21.15.106:0/1046589391 >> 172.21.15.106:0/1046589391 conn(0x7f09f006c9b0 msgr2=0x7f09f006cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:41.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.074+0000 7f09f7534640 1 -- 172.21.15.106:0/1046589391 shutdown_connections 2024-04-25T10:30:41.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.074+0000 7f09f7534640 1 -- 172.21.15.106:0/1046589391 wait complete. 2024-04-25T10:30:41.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.078+0000 7f09f7534640 1 Processor -- start 2024-04-25T10:30:41.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.078+0000 7f09f7534640 1 -- start start 2024-04-25T10:30:41.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.081+0000 7f09f7534640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09f0072870 0x7f09f01fad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.081+0000 7f09f7534640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09f00c99f0 0x7f09f0160b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.081+0000 7f09f52d0640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09f0072870 0x7f09f01fad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:41.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.081+0000 7f09f4acf640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09f00c99f0 0x7f09f0160b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:41.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.081+0000 7f09f4acf640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09f00c99f0 0x7f09f0160b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.106:48042/0 (socket says 172.21.15.106:48042) 2024-04-25T10:30:41.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.082+0000 7f09f52d0640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09f0072870 0x7f09f01fad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.106:49106/0 (socket says 172.21.15.106:49106) 2024-04-25T10:30:41.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.082+0000 7f09f7534640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09f00ca120 0x7f09f014afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.082+0000 7f09f4acf640 1 -- 172.21.15.106:0/1757895629 learned_addr learned my addr 172.21.15.106:0/1757895629 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:41.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.082+0000 7f09f5ad1640 1 --2- 172.21.15.106:0/1757895629 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09f00ca120 0x7f09f014afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:41.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.082+0000 7f09f7534640 1 -- 172.21.15.106:0/1757895629 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f09f00c4ef0 con 0x7f09f00ca120 2024-04-25T10:30:41.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.082+0000 7f09f7534640 1 -- 172.21.15.106:0/1757895629 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f09f00c8f90 con 0x7f09f00c99f0 2024-04-25T10:30:41.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.082+0000 7f09f7534640 1 -- 172.21.15.106:0/1757895629 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f09f0071060 con 0x7f09f0072870 2024-04-25T10:30:41.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.083+0000 7f09f5ad1640 1 -- 172.21.15.106:0/1757895629 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09f0072870 msgr2=0x7f09f01fad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:41.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.083+0000 7f09f5ad1640 1 --2- 172.21.15.106:0/1757895629 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09f0072870 0x7f09f01fad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.083+0000 7f09f5ad1640 1 -- 172.21.15.106:0/1757895629 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09f00c99f0 msgr2=0x7f09f0160b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:41.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.083+0000 7f09f5ad1640 1 --2- 172.21.15.106:0/1757895629 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09f00c99f0 0x7f09f0160b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.083+0000 7f09f5ad1640 1 -- 172.21.15.106:0/1757895629 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09f0154de0 con 0x7f09f00ca120 2024-04-25T10:30:41.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.084+0000 7f09f5ad1640 1 --2- 172.21.15.106:0/1757895629 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09f00ca120 0x7f09f014afb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f09e8031000 tx=0x7f09e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:41.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.084+0000 7f09e5ffb640 1 -- 172.21.15.106:0/1757895629 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e803f070 con 0x7f09f00ca120 2024-04-25T10:30:41.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.084+0000 7f09f7534640 1 -- 172.21.15.106:0/1757895629 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09f01559a0 con 0x7f09f00ca120 2024-04-25T10:30:41.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.084+0000 7f09f7534640 1 -- 172.21.15.106:0/1757895629 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09f0155f60 con 0x7f09f00ca120 2024-04-25T10:30:41.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.085+0000 7f09e5ffb640 1 -- 172.21.15.106:0/1757895629 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09e8030a60 con 0x7f09f00ca120 2024-04-25T10:30:41.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.085+0000 7f09e5ffb640 1 -- 172.21.15.106:0/1757895629 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e8030e10 con 0x7f09f00ca120 2024-04-25T10:30:41.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.089+0000 7f09e5ffb640 1 -- 172.21.15.106:0/1757895629 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f09e8044460 con 0x7f09f00ca120 2024-04-25T10:30:41.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.090+0000 7f09e5ffb640 1 --2- 172.21.15.106:0/1757895629 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f09d4074580 0x7f09d4076a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.090+0000 7f09f52d0640 1 --2- 172.21.15.106:0/1757895629 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f09d4074580 0x7f09d4076a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:41.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.091+0000 7f09f52d0640 1 --2- 172.21.15.106:0/1757895629 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f09d4074580 0x7f09d4076a90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f09e0006fd0 tx=0x7f09e0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:41.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.091+0000 7f09e5ffb640 1 -- 172.21.15.106:0/1757895629 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f09e80b67b0 con 0x7f09f00ca120 2024-04-25T10:30:41.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.093+0000 7f09f7534640 1 -- 172.21.15.106:0/1757895629 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09f0159470 con 0x7f09f00ca120 2024-04-25T10:30:41.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.103+0000 7f09e5ffb640 1 -- 172.21.15.106:0/1757895629 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f09f0159470 con 0x7f09f00ca120 2024-04-25T10:30:41.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.133+0000 7f5fb3191640 1 Processor -- start 2024-04-25T10:30:41.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.133+0000 7f5fb3191640 1 -- start start 2024-04-25T10:30:41.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.136+0000 7f5fb3191640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f5fa4006a90 0x7f5fa4045e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.136+0000 7f5fb3191640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5fa4047540 con 0x7f5fa40d56f0 2024-04-25T10:30:41.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.136+0000 7f5fb3191640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5fa4034fe0 con 0x7f5fa40334e0 2024-04-25T10:30:41.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.136+0000 7f5fb3191640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f5fa4035840 con 0x7f5fa4006a90 2024-04-25T10:30:41.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.136+0000 7f5fb2990640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f5fa40d56f0 0x7f5fa404c220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:44334/0 (socket says 172.21.15.106:44334) 2024-04-25T10:30:41.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.136+0000 7f5fb2990640 1 -- 172.21.15.106:0/3733259471 learned_addr learned my addr 172.21.15.106:0/3733259471 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:30:41.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.137+0000 7f5fb198e640 1 --2- 172.21.15.106:0/3733259471 >> v2:172.21.15.134:3300/0 conn(0x7f5fa4006a90 0x7f5fa4045e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:41.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.137+0000 7f0771a51640 1 Processor -- start 2024-04-25T10:30:41.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.137+0000 7f5fb118d640 1 -- 172.21.15.106:0/3733259471 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2363327499 0 0) 0x7f5fa4047540 con 0x7f5fa40d56f0 2024-04-25T10:30:41.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.137+0000 7f47a359e640 1 Processor -- start 2024-04-25T10:30:41.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.137+0000 7f5fb118d640 1 -- 172.21.15.106:0/3733259471 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5f98003440 con 0x7f5fa40d56f0 2024-04-25T10:30:41.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.137+0000 7f0771a51640 1 -- start start 2024-04-25T10:30:41.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.137+0000 7f47a359e640 1 -- start start 2024-04-25T10:30:41.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.138+0000 7f5fb118d640 1 -- 172.21.15.106:0/3733259471 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3998942457 0 0) 0x7f5fa4034fe0 con 0x7f5fa40334e0 2024-04-25T10:30:41.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.138+0000 7f5fb118d640 1 -- 172.21.15.106:0/3733259471 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5fa4047540 con 0x7f5fa40334e0 2024-04-25T10:30:41.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.139+0000 7f5fb198e640 1 -- 172.21.15.106:0/3733259471 >> v1:172.21.15.134:6789/0 conn(0x7f5fa40334e0 legacy=0x7f5fa4046d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:41.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.139+0000 7f5fb198e640 1 -- 172.21.15.106:0/3733259471 >> v1:172.21.15.106:6789/0 conn(0x7f5fa40d56f0 legacy=0x7f5fa404c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:41.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.139+0000 7f5fb198e640 1 -- 172.21.15.106:0/3733259471 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fa4035280 con 0x7f5fa4006a90 2024-04-25T10:30:41.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.139+0000 7f0771a51640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f076c0c3350 0x7f076c0cd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.139+0000 7f0771a51640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f076c0cab60 con 0x7f076c0cc0d0 2024-04-25T10:30:41.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.139+0000 7f0771a51640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f076c0c4f90 con 0x7f076c153280 2024-04-25T10:30:41.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.139+0000 7f0771a51640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f076c0cc4e0 con 0x7f076c0c3350 2024-04-25T10:30:41.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.140+0000 7f076affd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f076c0c3350 0x7f076c0cd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:41.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.140+0000 7f076a7fc640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f076c0cc0d0 0x7f076c06c5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:44348/0 (socket says 172.21.15.106:44348) 2024-04-25T10:30:41.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.140+0000 7f076a7fc640 1 -- 172.21.15.106:0/2839733453 learned_addr learned my addr 172.21.15.106:0/2839733453 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:30:41.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.140+0000 7f5fb198e640 1 --2- 172.21.15.106:0/3733259471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fa4006a90 0x7f5fa4045e70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5fa80097a0 tx=0x7f5fa802a0b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6db5b8b1b40bf096 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:41.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.140+0000 7ff8aabd6640 1 Processor -- start 2024-04-25T10:30:41.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.140+0000 7f47a359e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f479c006cb0 0x7f479c0478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.140+0000 7f47a359e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f479c045be0 0x7f479c04c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.140+0000 7f47a359e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f479c0dc2a0 con 0x7f479c04d1c0 2024-04-25T10:30:41.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.140+0000 7f47a359e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f479c0339b0 con 0x7f479c045be0 2024-04-25T10:30:41.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.140+0000 7f47a359e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f479c046b90 con 0x7f479c006cb0 2024-04-25T10:30:41.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.140+0000 7f2336eab640 1 Processor -- start 2024-04-25T10:30:41.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.140+0000 7f47a2d9d640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f479c04d1c0 0x7f479c0ded70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:44360/0 (socket says 172.21.15.106:44360) 2024-04-25T10:30:41.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.140+0000 7f47a2d9d640 1 -- 172.21.15.106:0/1794379072 learned_addr learned my addr 172.21.15.106:0/1794379072 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:30:41.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.140+0000 7f47a1d9b640 1 --2- 172.21.15.106:0/1794379072 >> v2:172.21.15.106:3300/0 conn(0x7f479c045be0 0x7f479c04c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:41.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.140+0000 7f076affd640 1 -- 172.21.15.106:0/2839733453 >> v1:172.21.15.134:6789/0 conn(0x7f076c153280 legacy=0x7f076c1637e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:41.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.141+0000 7f2336eab640 1 -- start start 2024-04-25T10:30:41.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.141+0000 7f3771700640 1 Processor -- start 2024-04-25T10:30:41.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.141+0000 7f5fb118d640 1 -- 172.21.15.106:0/3733259471 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa8037070 con 0x7f5fa4006a90 2024-04-25T10:30:41.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.141+0000 7f3771700640 1 -- start start 2024-04-25T10:30:41.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.141+0000 7f5fb118d640 1 -- 172.21.15.106:0/3733259471 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fa802ab40 con 0x7f5fa4006a90 2024-04-25T10:30:41.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.141+0000 7f5fb118d640 1 -- 172.21.15.106:0/3733259471 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fac069660 con 0x7f5fa4006a90 2024-04-25T10:30:41.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.141+0000 7f1ee573c640 1 Processor -- start 2024-04-25T10:30:41.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.141+0000 7f1ee573c640 1 -- start start 2024-04-25T10:30:41.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.140+0000 7f076affd640 1 -- 172.21.15.106:0/2839733453 >> v1:172.21.15.106:6789/0 conn(0x7f076c0cc0d0 legacy=0x7f076c06c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:41.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.140+0000 7f076affd640 1 -- 172.21.15.106:0/2839733453 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f076c0c49f0 con 0x7f076c0c3350 2024-04-25T10:30:41.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.140+0000 7f0769ffb640 1 -- 172.21.15.106:0/2839733453 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3712150424 0 0) 0x7f076c0c4f90 con 0x7f076c153280 2024-04-25T10:30:41.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.142+0000 7f076affd640 1 --2- 172.21.15.106:0/2839733453 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f076c0c3350 0x7f076c0cd830 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f076000ba40 tx=0x7f076002c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4be74b364c59b410 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:41.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.141+0000 7f47a1d9b640 1 -- 172.21.15.106:0/1794379072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f479c006cb0 msgr2=0x7f479c0478c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T10:30:41.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.141+0000 7f47a1d9b640 1 --2- 172.21.15.106:0/1794379072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f479c006cb0 0x7f479c0478c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.141+0000 7f47a1d9b640 1 -- 172.21.15.106:0/1794379072 >> v1:172.21.15.106:6789/0 conn(0x7f479c04d1c0 legacy=0x7f479c0ded70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:41.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.141+0000 7f47a1d9b640 1 -- 172.21.15.106:0/1794379072 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f479c0dc580 con 0x7f479c045be0 2024-04-25T10:30:41.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.140+0000 7ff8aabd6640 1 -- start start 2024-04-25T10:30:41.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.142+0000 7ff8aabd6640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff8a40c7ad0 0x7ff8a406ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.142+0000 7ff8aabd6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8a4156f70 0x7ff8a406d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.142+0000 7ff8aabd6640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8a415d6d0 con 0x7ff8a406daa0 2024-04-25T10:30:41.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.142+0000 7ff8aabd6640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7ff8a415d9b0 con 0x7ff8a40c7ad0 2024-04-25T10:30:41.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.142+0000 7ff8aabd6640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff8a40c7150 con 0x7ff8a4156f70 2024-04-25T10:30:41.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.142+0000 7ff8a9173640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7ff8a406daa0 0x7ff8a40c8120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:44372/0 (socket says 172.21.15.106:44372) 2024-04-25T10:30:41.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.142+0000 7ff8a9173640 1 -- 172.21.15.106:0/2720529886 learned_addr learned my addr 172.21.15.106:0/2720529886 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:30:41.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.143+0000 7f47a1d9b640 1 --2- 172.21.15.106:0/1794379072 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f479c045be0 0x7f479c04c410 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f47980097a0 tx=0x7f479802c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b9af5e5e6a17d1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:41.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.143+0000 7f5fb3191640 1 -- 172.21.15.106:0/3733259471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fa4006a90 msgr2=0x7f5fa4045e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:41.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.143+0000 7f5fb3191640 1 --2- 172.21.15.106:0/3733259471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fa4006a90 0x7f5fa4045e70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5fa80097a0 tx=0x7f5fa802a0b0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.143+0000 7ff8a3fff640 1 --2- 172.21.15.106:0/2720529886 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8a4156f70 0x7ff8a406d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:41.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.143+0000 7f2336eab640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2330068670 0x7f23300731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.143+0000 7f5fb3191640 1 -- 172.21.15.106:0/3733259471 shutdown_connections 2024-04-25T10:30:41.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.143+0000 7f3771700640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f376c152f10 0x7f376c0c77a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.143+0000 7f3771700640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f376c070120 0x7f376c070530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.143+0000 7f2335ea9640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f2330074750 0x7f2330154960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.106:41244/0 (socket says 172.21.15.106:41244) 2024-04-25T10:30:41.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.143+0000 7ff8a37fe640 1 -- 172.21.15.106:0/2720529886 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3904879600 0 0) 0x7ff8a415d6d0 con 0x7ff8a406daa0 2024-04-25T10:30:41.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.143+0000 7f2335ea9640 1 -- 172.21.15.106:0/1135957026 learned_addr learned my addr 172.21.15.106:0/1135957026 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:30:41.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.143+0000 7f47a159a640 1 -- 172.21.15.106:0/1794379072 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4798039070 con 0x7f479c045be0 2024-04-25T10:30:41.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.143+0000 7f47a159a640 1 -- 172.21.15.106:0/1794379072 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4798034640 con 0x7f479c045be0 2024-04-25T10:30:41.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.143+0000 7f47a159a640 1 -- 172.21.15.106:0/1794379072 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4798034a50 con 0x7f479c045be0 2024-04-25T10:30:41.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.143+0000 7f23356a8640 1 --2- 172.21.15.106:0/1135957026 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2330068670 0x7f23300731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:41.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.143+0000 7ff8a37fe640 1 -- 172.21.15.106:0/2720529886 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff88c003520 con 0x7ff8a406daa0 2024-04-25T10:30:41.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.143+0000 7f5fb3191640 1 --2- 172.21.15.106:0/3733259471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fa4006a90 0x7f5fa4045e70 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.143+0000 7f5fb3191640 1 -- 172.21.15.106:0/3733259471 >> 172.21.15.106:0/3733259471 conn(0x7f5fa40503b0 msgr2=0x7f5fa40507c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:41.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.143+0000 7f2336eab640 1 --2- 172.21.15.106:0/1135957026 >> v2:172.21.15.134:3300/0 conn(0x7f2330073700 0x7f233015e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.143+0000 7f3771700640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f376c0c7ce0 con 0x7f376c158bd0 2024-04-25T10:30:41.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.143+0000 7f3763fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f376c070120 0x7f376c070530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:41.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.143+0000 7f3771700640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f376c0c52f0 con 0x7f376c152f10 2024-04-25T10:30:41.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.143+0000 7f3771700640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f376c073d00 con 0x7f376c070120 2024-04-25T10:30:41.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.143+0000 7f2336eab640 1 -- 172.21.15.106:0/1135957026 --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23300c9080 con 0x7f2330074750 2024-04-25T10:30:41.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.143+0000 7f3763fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f376c070120 0x7f376c070530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.106:49132/0 (socket says 172.21.15.106:49132) 2024-04-25T10:30:41.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.143+0000 7f3763fff640 1 -- 172.21.15.106:0/173501894 learned_addr learned my addr 172.21.15.106:0/173501894 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:41.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.143+0000 7f2336eab640 1 -- 172.21.15.106:0/1135957026 --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f23300c9360 con 0x7f2330073700 2024-04-25T10:30:41.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.143+0000 7f5fb3191640 1 -- 172.21.15.106:0/3733259471 shutdown_connections 2024-04-25T10:30:41.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.143+0000 7f1ee573c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1ee0157a20 0x7f1ee00ca290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.143+0000 7f1ee573c640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f1ee00c9ba0 0x7f1ee00ca7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.143+0000 7f1ee573c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ee01619d0 con 0x7f1ee00ce440 2024-04-25T10:30:41.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.143+0000 7f1ee573c640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f1ee00c9000 con 0x7f1ee00c9ba0 2024-04-25T10:30:41.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.143+0000 7f1ee573c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f1ee00c86d0 con 0x7f1ee0157a20 2024-04-25T10:30:41.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.144+0000 7f1ee4f3b640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f1ee00c9ba0 0x7f1ee00ca7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:41.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.143+0000 7f23366aa640 1 --2- 172.21.15.106:0/1135957026 >> v2:172.21.15.134:3300/0 conn(0x7f2330073700 0x7f233015e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:41.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.144+0000 7f1ee4f3b640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f1ee00c9ba0 0x7f1ee00ca7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.106:48070/0 (socket says 172.21.15.106:48070) 2024-04-25T10:30:41.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.144+0000 7f1ee4f3b640 1 -- 172.21.15.106:0/1151284196 learned_addr learned my addr 172.21.15.106:0/1151284196 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:41.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.144+0000 7f1edf7fe640 1 --2- 172.21.15.106:0/1151284196 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1ee0157a20 0x7f1ee00ca290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:41.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.144+0000 7f2336eab640 1 -- 172.21.15.106:0/1135957026 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f23300c9640 con 0x7f2330068670 2024-04-25T10:30:41.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.144+0000 7f5fb3191640 1 -- 172.21.15.106:0/3733259471 wait complete. 2024-04-25T10:30:41.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.144+0000 7ff8a8972640 1 --2- 172.21.15.106:0/2720529886 >> v2:172.21.15.134:3300/0 conn(0x7ff8a40c7ad0 0x7ff8a406ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:41.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.143+0000 7f0769ffb640 1 -- 172.21.15.106:0/2839733453 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f076002f070 con 0x7f076c0c3350 2024-04-25T10:30:41.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.143+0000 7f0769ffb640 1 -- 172.21.15.106:0/2839733453 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0760038070 con 0x7f076c0c3350 2024-04-25T10:30:41.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.143+0000 7f0769ffb640 1 -- 172.21.15.106:0/2839733453 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0760032a00 con 0x7f076c0c3350 2024-04-25T10:30:41.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.144+0000 7f2334ea7640 1 -- 172.21.15.106:0/1135957026 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1195171822 0 0) 0x7f23300c9080 con 0x7f2330074750 2024-04-25T10:30:41.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.144+0000 7f2334ea7640 1 -- 172.21.15.106:0/1135957026 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f231c003520 con 0x7f2330074750 2024-04-25T10:30:41.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.144+0000 7f3763fff640 1 -- 172.21.15.106:0/173501894 >> v2:172.21.15.106:3300/0 conn(0x7f376c152f10 msgr2=0x7f376c0c77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:41.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.144+0000 7ff8a3fff640 1 -- 172.21.15.106:0/2720529886 >> v2:172.21.15.134:3300/0 conn(0x7ff8a40c7ad0 msgr2=0x7ff8a406ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:41.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.144+0000 7ff8a3fff640 1 --2- 172.21.15.106:0/2720529886 >> v2:172.21.15.134:3300/0 conn(0x7ff8a40c7ad0 0x7ff8a406ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.144+0000 7ff8a3fff640 1 -- 172.21.15.106:0/2720529886 >> v1:172.21.15.106:6789/0 conn(0x7ff8a406daa0 legacy=0x7ff8a40c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:41.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.144+0000 7ff8a3fff640 1 -- 172.21.15.106:0/2720529886 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8a40c7430 con 0x7ff8a4156f70 2024-04-25T10:30:41.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.144+0000 7f3763fff640 1 --2- 172.21.15.106:0/173501894 >> v2:172.21.15.106:3300/0 conn(0x7f376c152f10 0x7f376c0c77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.144+0000 7f3763fff640 1 -- 172.21.15.106:0/173501894 >> v1:172.21.15.106:6789/0 conn(0x7f376c158bd0 legacy=0x7f376c158fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:41.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.144+0000 7f1ee4f3b640 1 -- 172.21.15.106:0/1151284196 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1ee0157a20 msgr2=0x7f1ee00ca290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:41.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.144+0000 7f1ee4f3b640 1 --2- 172.21.15.106:0/1151284196 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1ee0157a20 0x7f1ee00ca290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.144+0000 7f1ee4f3b640 1 -- 172.21.15.106:0/1151284196 >> v1:172.21.15.106:6789/0 conn(0x7f1ee00ce440 legacy=0x7f1ee00c9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:41.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.144+0000 7f1ee4f3b640 1 -- 172.21.15.106:0/1151284196 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ee00c89b0 con 0x7f1ee00c9ba0 2024-04-25T10:30:41.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.144+0000 7f3763fff640 1 -- 172.21.15.106:0/173501894 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f376c0c7f50 con 0x7f376c070120 2024-04-25T10:30:41.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.144+0000 7f0771a51640 1 -- 172.21.15.106:0/2839733453 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f076c0c3350 msgr2=0x7f076c0cd830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:41.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.144+0000 7f0771a51640 1 --2- 172.21.15.106:0/2839733453 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f076c0c3350 0x7f076c0cd830 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f076000ba40 tx=0x7f076002c8e0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.144+0000 7f0771a51640 1 -- 172.21.15.106:0/2839733453 shutdown_connections 2024-04-25T10:30:41.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.144+0000 7f0771a51640 1 --2- 172.21.15.106:0/2839733453 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f076c0c3350 0x7f076c0cd830 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.144+0000 7f0771a51640 1 -- 172.21.15.106:0/2839733453 >> 172.21.15.106:0/2839733453 conn(0x7f076c070ed0 msgr2=0x7f076c0ce2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:41.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.144+0000 7f2334ea7640 1 -- 172.21.15.106:0/1135957026 <== mon.1 v1:172.21.15.134:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3445820057 0 0) 0x7f231c003520 con 0x7f2330074750 2024-04-25T10:30:41.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.144+0000 7f2334ea7640 1 -- 172.21.15.106:0/1135957026 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2330068670 msgr2=0x7f23300731c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:41.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.144+0000 7f2334ea7640 1 --2- 172.21.15.106:0/1135957026 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2330068670 0x7f23300731c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.144+0000 7f2334ea7640 1 -- 172.21.15.106:0/1135957026 >> v2:172.21.15.134:3300/0 conn(0x7f2330073700 msgr2=0x7f233015e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:41.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.144+0000 7f2334ea7640 1 --2- 172.21.15.106:0/1135957026 >> v2:172.21.15.134:3300/0 conn(0x7f2330073700 0x7f233015e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.144+0000 7f2334ea7640 1 -- 172.21.15.106:0/1135957026 --> v1:172.21.15.134:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23300c9920 con 0x7f2330074750 2024-04-25T10:30:41.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.144+0000 7f2334ea7640 1 -- 172.21.15.106:0/1135957026 <== mon.1 v1:172.21.15.134:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3776459350 0 0) 0x7f232c001670 con 0x7f2330074750 2024-04-25T10:30:41.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.145+0000 7f2334ea7640 1 -- 172.21.15.106:0/1135957026 >> v1:172.21.15.134:6789/0 conn(0x7f2330074750 legacy=0x7f2330154960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:41.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.145+0000 7f2334ea7640 1 --2- 172.21.15.106:0/1135957026 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f231c0039d0 0x7f231c003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.145+0000 7f2334ea7640 1 --2- 172.21.15.106:0/1135957026 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f231c004800 0x7f231c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.145+0000 7f2334ea7640 1 --2- 172.21.15.106:0/1135957026 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f231c005150 0x7f231c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.145+0000 7f2334ea7640 1 -- 172.21.15.106:0/1135957026 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f23300c9360 con 0x7f231c0039d0 2024-04-25T10:30:41.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.145+0000 7f2334ea7640 1 -- 172.21.15.106:0/1135957026 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f23300c9640 con 0x7f231c005150 2024-04-25T10:30:41.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.145+0000 7f2334ea7640 1 -- 172.21.15.106:0/1135957026 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f231c00bf50 con 0x7f231c004800 2024-04-25T10:30:41.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.145+0000 7f2335ea9640 1 --2- 172.21.15.106:0/1135957026 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f231c0039d0 0x7f231c003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:41.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.145+0000 7f47a359e640 1 -- 172.21.15.106:0/1794379072 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f479c045be0 msgr2=0x7f479c04c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:41.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.145+0000 7f47a359e640 1 --2- 172.21.15.106:0/1794379072 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f479c045be0 0x7f479c04c410 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f47980097a0 tx=0x7f479802c3b0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.145+0000 7f0771a51640 1 -- 172.21.15.106:0/2839733453 shutdown_connections 2024-04-25T10:30:41.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.145+0000 7f0771a51640 1 -- 172.21.15.106:0/2839733453 wait complete. 2024-04-25T10:30:41.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.145+0000 7f3763fff640 1 --2- 172.21.15.106:0/173501894 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f376c070120 0x7f376c070530 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3754009870 tx=0x7f375402a2c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d07717bdb3963b99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:41.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.145+0000 7f47a359e640 1 -- 172.21.15.106:0/1794379072 shutdown_connections 2024-04-25T10:30:41.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.145+0000 7f47a359e640 1 --2- 172.21.15.106:0/1794379072 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f479c045be0 0x7f479c04c410 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.145+0000 7f47a359e640 1 --2- 172.21.15.106:0/1794379072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f479c006cb0 0x7f479c0478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.145+0000 7f47a359e640 1 -- 172.21.15.106:0/1794379072 >> 172.21.15.106:0/1794379072 conn(0x7f479c050c40 msgr2=0x7f479c051050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:41.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.145+0000 7ff8a3fff640 1 --2- 172.21.15.106:0/2720529886 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8a4156f70 0x7ff8a406d560 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff8980097a0 tx=0x7ff89802a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e069391236a26d87 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:41.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.145+0000 7f23356a8640 1 --2- 172.21.15.106:0/1135957026 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f231c004800 0x7f231c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:41.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.145+0000 7f23366aa640 1 --2- 172.21.15.106:0/1135957026 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f231c005150 0x7f231c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:41.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.145+0000 7f1ee4f3b640 1 --2- 172.21.15.106:0/1151284196 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ee00c9ba0 0x7f1ee00ca7d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1ed8009e50 tx=0x7f1ed802a6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd8eb209505d22e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:41.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.146+0000 7f1edeffd640 1 -- 172.21.15.106:0/1151284196 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed8036070 con 0x7f1ee00c9ba0 2024-04-25T10:30:41.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.146+0000 7f1edeffd640 1 -- 172.21.15.106:0/1151284196 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ed8031600 con 0x7f1ee00c9ba0 2024-04-25T10:30:41.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.146+0000 7f1edeffd640 1 -- 172.21.15.106:0/1151284196 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed80319d0 con 0x7f1ee00c9ba0 2024-04-25T10:30:41.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.146+0000 7ff8a37fe640 1 -- 172.21.15.106:0/2720529886 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff898037070 con 0x7ff8a4156f70 2024-04-25T10:30:41.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.146+0000 7f2335ea9640 1 -- 172.21.15.106:0/1135957026 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f231c004800 msgr2=0x7f231c004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:41.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.146+0000 7f376a7fc640 1 -- 172.21.15.106:0/173501894 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3754037070 con 0x7f376c070120 2024-04-25T10:30:41.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.146+0000 7ff8a37fe640 1 -- 172.21.15.106:0/2720529886 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff898032440 con 0x7ff8a4156f70 2024-04-25T10:30:41.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.146+0000 7ff8a37fe640 1 -- 172.21.15.106:0/2720529886 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff898032850 con 0x7ff8a4156f70 2024-04-25T10:30:41.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.146+0000 7f2335ea9640 1 --2- 172.21.15.106:0/1135957026 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f231c004800 0x7f231c004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.146+0000 7f2335ea9640 1 -- 172.21.15.106:0/1135957026 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f231c005150 msgr2=0x7f231c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:41.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.146+0000 7f2335ea9640 1 --2- 172.21.15.106:0/1135957026 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f231c005150 0x7f231c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.146+0000 7f2335ea9640 1 -- 172.21.15.106:0/1135957026 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23300c9920 con 0x7f231c0039d0 2024-04-25T10:30:41.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.146+0000 7f23366aa640 1 --2- 172.21.15.106:0/1135957026 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f231c005150 0x7f231c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:41.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.146+0000 7f23356a8640 1 --2- 172.21.15.106:0/1135957026 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f231c004800 0x7f231c004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:41.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.146+0000 7f5fb3191640 1 Processor -- start 2024-04-25T10:30:41.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.146+0000 7f5fb3191640 1 -- start start 2024-04-25T10:30:41.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.146+0000 7f376a7fc640 1 -- 172.21.15.106:0/173501894 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3754032440 con 0x7f376c070120 2024-04-25T10:30:41.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.146+0000 7f376a7fc640 1 -- 172.21.15.106:0/173501894 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37540327f0 con 0x7f376c070120 2024-04-25T10:30:41.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.147+0000 7f1ee573c640 1 -- 172.21.15.106:0/1151284196 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ee00c9ba0 msgr2=0x7f1ee00ca7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:41.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.147+0000 7f1ee573c640 1 --2- 172.21.15.106:0/1151284196 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ee00c9ba0 0x7f1ee00ca7d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1ed8009e50 tx=0x7f1ed802a6c0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.147+0000 7ff8aabd6640 1 -- 172.21.15.106:0/2720529886 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8a4156f70 msgr2=0x7ff8a406d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:41.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.147+0000 7ff8aabd6640 1 --2- 172.21.15.106:0/2720529886 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8a4156f70 0x7ff8a406d560 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff8980097a0 tx=0x7ff89802a1a0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.147+0000 7ff8aabd6640 1 -- 172.21.15.106:0/2720529886 shutdown_connections 2024-04-25T10:30:41.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.147+0000 7ff8aabd6640 1 --2- 172.21.15.106:0/2720529886 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8a4156f70 0x7ff8a406d560 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.147+0000 7ff8aabd6640 1 --2- 172.21.15.106:0/2720529886 >> v2:172.21.15.134:3300/0 conn(0x7ff8a40c7ad0 0x7ff8a406ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.147+0000 7ff8aabd6640 1 -- 172.21.15.106:0/2720529886 >> 172.21.15.106:0/2720529886 conn(0x7ff8a4068ed0 msgr2=0x7ff8a40d20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:41.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.147+0000 7f3771700640 1 -- 172.21.15.106:0/173501894 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f376c070120 msgr2=0x7f376c070530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:41.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.147+0000 7f3771700640 1 --2- 172.21.15.106:0/173501894 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f376c070120 0x7f376c070530 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3754009870 tx=0x7f375402a2c0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.147+0000 7ff8aabd6640 1 -- 172.21.15.106:0/2720529886 shutdown_connections 2024-04-25T10:30:41.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.147+0000 7ff8aabd6640 1 -- 172.21.15.106:0/2720529886 wait complete. 2024-04-25T10:30:41.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.147+0000 7f3771700640 1 -- 172.21.15.106:0/173501894 shutdown_connections 2024-04-25T10:30:41.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.147+0000 7f3771700640 1 --2- 172.21.15.106:0/173501894 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f376c070120 0x7f376c070530 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.147+0000 7f3771700640 1 --2- 172.21.15.106:0/173501894 >> v2:172.21.15.106:3300/0 conn(0x7f376c152f10 0x7f376c0c77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.147+0000 7f3771700640 1 -- 172.21.15.106:0/173501894 >> 172.21.15.106:0/173501894 conn(0x7f376c06c9e0 msgr2=0x7f376c06cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:41.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.147+0000 7f1ee573c640 1 -- 172.21.15.106:0/1151284196 shutdown_connections 2024-04-25T10:30:41.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.147+0000 7f1ee573c640 1 --2- 172.21.15.106:0/1151284196 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ee00c9ba0 0x7f1ee00ca7d0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.147+0000 7f1ee573c640 1 --2- 172.21.15.106:0/1151284196 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1ee0157a20 0x7f1ee00ca290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.147+0000 7f1ee573c640 1 -- 172.21.15.106:0/1151284196 >> 172.21.15.106:0/1151284196 conn(0x7f1ee006c9e0 msgr2=0x7f1ee006cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:41.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.147+0000 7f3771700640 1 -- 172.21.15.106:0/173501894 shutdown_connections 2024-04-25T10:30:41.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.147+0000 7f3771700640 1 -- 172.21.15.106:0/173501894 wait complete. 2024-04-25T10:30:41.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.147+0000 7f0771a51640 1 Processor -- start 2024-04-25T10:30:41.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.148+0000 7f2335ea9640 1 --2- 172.21.15.106:0/1135957026 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f231c0039d0 0x7f231c003e60 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f232c00c4e0 tx=0x7f231c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=665e055e33ee586b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:41.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.148+0000 7f0771a51640 1 -- start start 2024-04-25T10:30:41.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.147+0000 7f47a359e640 1 -- 172.21.15.106:0/1794379072 shutdown_connections 2024-04-25T10:30:41.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.147+0000 7f47a359e640 1 -- 172.21.15.106:0/1794379072 wait complete. 2024-04-25T10:30:41.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.148+0000 7f2334ea7640 1 -- 172.21.15.106:0/1135957026 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f232c036070 con 0x7f231c0039d0 2024-04-25T10:30:41.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.148+0000 7f2334ea7640 1 -- 172.21.15.106:0/1135957026 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f232c004aa0 con 0x7f231c0039d0 2024-04-25T10:30:41.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.148+0000 7f2334ea7640 1 -- 172.21.15.106:0/1135957026 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f232c036070 con 0x7f231c0039d0 2024-04-25T10:30:41.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.149+0000 7f5fb3191640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5fa4006a90 0x7f5fa40c9a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.149+0000 7f5fb3191640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5fa40334e0 0x7f5fa40c7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.149+0000 7f1ee573c640 1 -- 172.21.15.106:0/1151284196 shutdown_connections 2024-04-25T10:30:41.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.149+0000 7f5fb3191640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fa40d56f0 0x7f5fa40c81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.149+0000 7f1ee573c640 1 -- 172.21.15.106:0/1151284196 wait complete. 2024-04-25T10:30:41.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.149+0000 7f5fb3191640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa4045350 con 0x7f5fa4006a90 2024-04-25T10:30:41.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.149+0000 7f5fb3191640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa4006570 con 0x7f5fa40d56f0 2024-04-25T10:30:41.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.149+0000 7f5fb3191640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f5fa40d5b00 con 0x7f5fa40334e0 2024-04-25T10:30:41.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.149+0000 7f5fb2990640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fa40d56f0 0x7f5fa40c81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:41.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.149+0000 7f5fb2990640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fa40d56f0 0x7f5fa40c81e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.106:48090/0 (socket says 172.21.15.106:48090) 2024-04-25T10:30:41.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.149+0000 7f5fb2990640 1 -- 172.21.15.106:0/2958742256 learned_addr learned my addr 172.21.15.106:0/2958742256 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:41.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.149+0000 7f0771a51640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f076c0c3350 0x7f076c1f6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.150+0000 7f2336eab640 1 -- 172.21.15.106:0/1135957026 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f231c0039d0 msgr2=0x7f231c003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:41.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.150+0000 7f2336eab640 1 --2- 172.21.15.106:0/1135957026 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f231c0039d0 0x7f231c003e60 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f232c00c4e0 tx=0x7f231c005830 comp rx=0 tx=0).stop 2024-04-25T10:30:41.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.150+0000 7f23366aa640 1 -- 172.21.15.106:0/1135957026 reap_dead start 2024-04-25T10:30:41.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.150+0000 7f0771a51640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f076c0cc0d0 0x7f076c1584f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.150+0000 7f0771a51640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f076c153280 0x7f076c14ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.150+0000 7f076a7fc640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f076c0cc0d0 0x7f076c1584f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:41.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.150+0000 7f076affd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f076c0c3350 0x7f076c1f6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:41.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.150+0000 7f076a7fc640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f076c0cc0d0 0x7f076c1584f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.106:49154/0 (socket says 172.21.15.106:49154) 2024-04-25T10:30:41.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.150+0000 7f076a7fc640 1 -- 172.21.15.106:0/934720775 learned_addr learned my addr 172.21.15.106:0/934720775 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:41.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.150+0000 7f076b7fe640 1 --2- 172.21.15.106:0/934720775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f076c153280 0x7f076c14ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:41.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.150+0000 7f0771a51640 1 -- 172.21.15.106:0/934720775 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f076c0ca880 con 0x7f076c0c3350 2024-04-25T10:30:41.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.150+0000 7f0771a51640 1 -- 172.21.15.106:0/934720775 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f076c0ca5a0 con 0x7f076c153280 2024-04-25T10:30:41.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.150+0000 7f0771a51640 1 -- 172.21.15.106:0/934720775 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f076c153690 con 0x7f076c0cc0d0 2024-04-25T10:30:41.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.150+0000 7ff8aabd6640 1 Processor -- start 2024-04-25T10:30:41.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.150+0000 7ff8aabd6640 1 -- start start 2024-04-25T10:30:41.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.150+0000 7f2336eab640 1 -- 172.21.15.106:0/1135957026 shutdown_connections 2024-04-25T10:30:41.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.150+0000 7f2336eab640 1 -- 172.21.15.106:0/1135957026 >> 172.21.15.106:0/1135957026 conn(0x7f23300cb070 msgr2=0x7f23300cb480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:41.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.150+0000 7f5fb218f640 1 --2- 172.21.15.106:0/2958742256 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5fa4006a90 0x7f5fa40c9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:41.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.150+0000 7f5fb2990640 1 -- 172.21.15.106:0/2958742256 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5fa40334e0 msgr2=0x7f5fa40c7ca0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:30:41.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.150+0000 7f5fb2990640 1 --2- 172.21.15.106:0/2958742256 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5fa40334e0 0x7f5fa40c7ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.150+0000 7f5fb2990640 1 -- 172.21.15.106:0/2958742256 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5fa4006a90 msgr2=0x7f5fa40c9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:41.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.150+0000 7f5fb2990640 1 --2- 172.21.15.106:0/2958742256 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5fa4006a90 0x7f5fa40c9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.150+0000 7f5fb2990640 1 -- 172.21.15.106:0/2958742256 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fa40c9fb0 con 0x7f5fa40d56f0 2024-04-25T10:30:41.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.150+0000 7f2336eab640 1 -- 172.21.15.106:0/1135957026 shutdown_connections 2024-04-25T10:30:41.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.150+0000 7f3771700640 1 Processor -- start 2024-04-25T10:30:41.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.150+0000 7f2336eab640 1 -- 172.21.15.106:0/1135957026 wait complete. 2024-04-25T10:30:41.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.151+0000 7f3771700640 1 -- start start 2024-04-25T10:30:41.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.151+0000 7f076b7fe640 1 -- 172.21.15.106:0/934720775 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f076c0cc0d0 msgr2=0x7f076c1584f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:41.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.151+0000 7f076b7fe640 1 --2- 172.21.15.106:0/934720775 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f076c0cc0d0 0x7f076c1584f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.151+0000 7f076b7fe640 1 -- 172.21.15.106:0/934720775 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f076c0c3350 msgr2=0x7f076c1f6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:41.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.151+0000 7f076b7fe640 1 --2- 172.21.15.106:0/934720775 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f076c0c3350 0x7f076c1f6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.151+0000 7f076b7fe640 1 -- 172.21.15.106:0/934720775 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f076c1e6ff0 con 0x7f076c153280 2024-04-25T10:30:41.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.151+0000 7f076b7fe640 1 --2- 172.21.15.106:0/934720775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f076c153280 0x7f076c14ec10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f076400c780 tx=0x7f076400cc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:41.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.151+0000 7f5fb2990640 1 --2- 172.21.15.106:0/2958742256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fa40d56f0 0x7f5fa40c81e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5fac052040 tx=0x7f5fac0784b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:41.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.151+0000 7f0753fff640 1 -- 172.21.15.106:0/934720775 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0764003b70 con 0x7f076c153280 2024-04-25T10:30:41.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.151+0000 7f5fa2ffd640 1 -- 172.21.15.106:0/2958742256 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fac08e770 con 0x7f5fa40d56f0 2024-04-25T10:30:41.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.151+0000 7f5fa2ffd640 1 -- 172.21.15.106:0/2958742256 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fac08e910 con 0x7f5fa40d56f0 2024-04-25T10:30:41.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.152+0000 7f5fa2ffd640 1 -- 172.21.15.106:0/2958742256 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fac08ed20 con 0x7f5fa40d56f0 2024-04-25T10:30:41.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.152+0000 7f1ee573c640 1 Processor -- start 2024-04-25T10:30:41.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.152+0000 7f1ee573c640 1 -- start start 2024-04-25T10:30:41.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.151+0000 7f0771a51640 1 -- 172.21.15.106:0/934720775 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f076c1e7cb0 con 0x7f076c153280 2024-04-25T10:30:41.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.151+0000 7f0771a51640 1 -- 172.21.15.106:0/934720775 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f076c1e72d0 con 0x7f076c153280 2024-04-25T10:30:41.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.151+0000 7f0753fff640 1 -- 172.21.15.106:0/934720775 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0764003d30 con 0x7f076c153280 2024-04-25T10:30:41.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.153+0000 7f0753fff640 1 -- 172.21.15.106:0/934720775 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07640052b0 con 0x7f076c153280 2024-04-25T10:30:41.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.153+0000 7f5fb3191640 1 -- 172.21.15.106:0/2958742256 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fa4047960 con 0x7f5fa40d56f0 2024-04-25T10:30:41.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.153+0000 7f3771700640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f376c070120 0x7f376c1f23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.153+0000 7ff8aabd6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8a406daa0 0x7ff8a41fb430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.153+0000 7ff8aabd6640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8a40c7ad0 0x7ff8a414fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.153+0000 7f3771700640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f376c152f10 0x7f376c148450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.153+0000 7f3771700640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f376c158bd0 0x7f376c1e5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.153+0000 7ff8aabd6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8a4156f70 0x7ff8a40d0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.153+0000 7ff8a3fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8a40c7ad0 0x7ff8a414fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:41.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.153+0000 7ff8a3fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8a40c7ad0 0x7ff8a414fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47944/0 (socket says 172.21.15.106:47944) 2024-04-25T10:30:41.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.153+0000 7ff8a3fff640 1 -- 172.21.15.106:0/716811340 learned_addr learned my addr 172.21.15.106:0/716811340 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:41.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.153+0000 7ff8aabd6640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a40ce6c0 con 0x7ff8a40c7ad0 2024-04-25T10:30:41.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.153+0000 7f0771a51640 1 -- 172.21.15.106:0/934720775 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f074c005290 con 0x7f076c153280 2024-04-25T10:30:41.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.153+0000 7ff8aabd6640 1 -- 172.21.15.106:0/716811340 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a40ce360 con 0x7ff8a406daa0 2024-04-25T10:30:41.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.153+0000 7ff8aabd6640 1 -- 172.21.15.106:0/716811340 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff8a40c8830 con 0x7ff8a4156f70 2024-04-25T10:30:41.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.153+0000 7f5fb3191640 1 -- 172.21.15.106:0/2958742256 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fa40dbb30 con 0x7f5fa40d56f0 2024-04-25T10:30:41.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.153+0000 7f376affd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f376c070120 0x7f376c1f23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:41.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.153+0000 7f376affd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f376c070120 0x7f376c1f23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.106:49164/0 (socket says 172.21.15.106:49164) 2024-04-25T10:30:41.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.153+0000 7f376affd640 1 -- 172.21.15.106:0/2112450836 learned_addr learned my addr 172.21.15.106:0/2112450836 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:41.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.154+0000 7f376b7fe640 1 --2- 172.21.15.106:0/2112450836 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f376c158bd0 0x7f376c1e5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:41.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.150+0000 7f47a359e640 1 Processor -- start 2024-04-25T10:30:41.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.150+0000 7f47a359e640 1 -- start start 2024-04-25T10:30:41.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.154+0000 7f2336eab640 1 Processor -- start 2024-04-25T10:30:41.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.154+0000 7f3771700640 1 -- 172.21.15.106:0/2112450836 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f376c074510 con 0x7f376c152f10 2024-04-25T10:30:41.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.154+0000 7f3763fff640 1 --2- 172.21.15.106:0/2112450836 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f376c152f10 0x7f376c148450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:41.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.154+0000 7f3771700640 1 -- 172.21.15.106:0/2112450836 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f376c074230 con 0x7f376c158bd0 2024-04-25T10:30:41.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.154+0000 7f3771700640 1 -- 172.21.15.106:0/2112450836 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f376c0ca250 con 0x7f376c070120 2024-04-25T10:30:41.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.154+0000 7f1ee573c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ee00c9ba0 0x7f1ee0164c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.154+0000 7f1ee573c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1ee00ce440 0x7f1ee0151580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.154+0000 7f1ee573c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1ee0157a20 0x7f1ee0160540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.154+0000 7f1ee573c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee00cad10 con 0x7f1ee00ce440 2024-04-25T10:30:41.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.154+0000 7f1ee573c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee0161cb0 con 0x7f1ee00c9ba0 2024-04-25T10:30:41.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.154+0000 7f1ee573c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f1ee00cae90 con 0x7f1ee0157a20 2024-04-25T10:30:41.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.154+0000 7f1edf7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1ee00ce440 0x7f1ee0151580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:41.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.154+0000 7f2336eab640 1 -- start start 2024-04-25T10:30:41.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.154+0000 7f47a359e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f479c006cb0 0x7f479c0d12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.154+0000 7ff8a9173640 1 --2- 172.21.15.106:0/716811340 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8a4156f70 0x7ff8a40d0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:41.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.154+0000 7ff8a8972640 1 --2- 172.21.15.106:0/716811340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8a406daa0 0x7ff8a41fb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:41.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.154+0000 7ff8a3fff640 1 -- 172.21.15.106:0/716811340 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8a4156f70 msgr2=0x7ff8a40d0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:41.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.154+0000 7ff8a3fff640 1 --2- 172.21.15.106:0/716811340 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8a4156f70 0x7ff8a40d0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.154+0000 7ff8a3fff640 1 -- 172.21.15.106:0/716811340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8a406daa0 msgr2=0x7ff8a41fb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:41.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.154+0000 7ff8a3fff640 1 --2- 172.21.15.106:0/716811340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8a406daa0 0x7ff8a41fb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.154+0000 7ff8a3fff640 1 -- 172.21.15.106:0/716811340 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8a4160e50 con 0x7ff8a40c7ad0 2024-04-25T10:30:41.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.154+0000 7f47a359e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f479c045be0 0x7f479c17db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.154+0000 7f47a359e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f479c04d1c0 0x7f479c0c6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.154+0000 7f47a359e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f479c047e00 con 0x7f479c006cb0 2024-04-25T10:30:41.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.154+0000 7f47a359e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f479c047430 con 0x7f479c04d1c0 2024-04-25T10:30:41.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.154+0000 7f47a359e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f479c0470c0 con 0x7f479c045be0 2024-04-25T10:30:41.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.154+0000 7f1edffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ee00c9ba0 0x7f1ee0164c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:41.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.154+0000 7f1edffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ee00c9ba0 0x7f1ee0164c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.106:48122/0 (socket says 172.21.15.106:48122) 2024-04-25T10:30:41.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.154+0000 7f1edffff640 1 -- 172.21.15.106:0/72500340 learned_addr learned my addr 172.21.15.106:0/72500340 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:41.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.155+0000 7f1ee4f3b640 1 --2- 172.21.15.106:0/72500340 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1ee0157a20 0x7f1ee0160540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:41.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.154+0000 7f47a259c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f479c006cb0 0x7f479c0d12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:41.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.154+0000 7f47a259c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f479c006cb0 0x7f479c0d12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47962/0 (socket says 172.21.15.106:47962) 2024-04-25T10:30:41.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.155+0000 7f3763fff640 1 -- 172.21.15.106:0/2112450836 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f376c070120 msgr2=0x7f376c1f23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:41.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.155+0000 7f3763fff640 1 --2- 172.21.15.106:0/2112450836 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f376c070120 0x7f376c1f23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.155+0000 7f3763fff640 1 -- 172.21.15.106:0/2112450836 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f376c158bd0 msgr2=0x7f376c1e5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:41.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.154+0000 7f47a259c640 1 -- 172.21.15.106:0/2802364784 learned_addr learned my addr 172.21.15.106:0/2802364784 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:41.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.155+0000 7f3763fff640 1 --2- 172.21.15.106:0/2112450836 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f376c158bd0 0x7f376c1e5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.155+0000 7f1edf7fe640 1 -- 172.21.15.106:0/72500340 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1ee0157a20 msgr2=0x7f1ee0160540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:41.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.155+0000 7f3763fff640 1 -- 172.21.15.106:0/2112450836 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f376c1eee00 con 0x7f376c152f10 2024-04-25T10:30:41.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.155+0000 7f47a2d9d640 1 --2- 172.21.15.106:0/2802364784 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f479c04d1c0 0x7f479c0c6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:41.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.155+0000 7f1edf7fe640 1 --2- 172.21.15.106:0/72500340 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1ee0157a20 0x7f1ee0160540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.155+0000 7f376b7fe640 1 --2- 172.21.15.106:0/2112450836 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f376c158bd0 0x7f376c1e5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:41.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.155+0000 7f376affd640 1 --2- 172.21.15.106:0/2112450836 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f376c070120 0x7f376c1f23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:41.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.155+0000 7f47a259c640 1 -- 172.21.15.106:0/2802364784 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f479c045be0 msgr2=0x7f479c17db00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:30:41.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.155+0000 7f47a259c640 1 --2- 172.21.15.106:0/2802364784 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f479c045be0 0x7f479c17db00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.155+0000 7f47a259c640 1 -- 172.21.15.106:0/2802364784 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f479c04d1c0 msgr2=0x7f479c0c6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:41.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.155+0000 7f47a259c640 1 --2- 172.21.15.106:0/2802364784 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f479c04d1c0 0x7f479c0c6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.155+0000 7f47a259c640 1 -- 172.21.15.106:0/2802364784 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f479c17e040 con 0x7f479c006cb0 2024-04-25T10:30:41.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.155+0000 7f1edf7fe640 1 -- 172.21.15.106:0/72500340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ee00c9ba0 msgr2=0x7f1ee0164c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:41.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.155+0000 7f1edf7fe640 1 --2- 172.21.15.106:0/72500340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ee00c9ba0 0x7f1ee0164c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.155+0000 7f1edf7fe640 1 -- 172.21.15.106:0/72500340 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ee015e0d0 con 0x7f1ee00ce440 2024-04-25T10:30:41.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.155+0000 7f3763fff640 1 --2- 172.21.15.106:0/2112450836 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f376c152f10 0x7f376c148450 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3754031000 tx=0x7f37540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:41.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.155+0000 7f1edffff640 1 --2- 172.21.15.106:0/72500340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ee00c9ba0 0x7f1ee0164c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:41.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.155+0000 7f47a259c640 1 --2- 172.21.15.106:0/2802364784 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f479c006cb0 0x7f479c0d12b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f47a40514f0 tx=0x7f47a40783f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:41.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.156+0000 7f1edf7fe640 1 --2- 172.21.15.106:0/72500340 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1ee00ce440 0x7f1ee0151580 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1ed000d520 tx=0x7f1ed000d9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:41.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.156+0000 7f5fa0ff9640 1 -- 172.21.15.106:0/2958742256 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fa4005570 con 0x7f5fa40d56f0 2024-04-25T10:30:41.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.156+0000 7f0753fff640 1 -- 172.21.15.106:0/934720775 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0764005450 con 0x7f076c153280 2024-04-25T10:30:41.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.155+0000 7ff8a3fff640 1 --2- 172.21.15.106:0/716811340 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8a40c7ad0 0x7ff8a414fca0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff8980098c0 tx=0x7ff8980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:41.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.155+0000 7ff8a17fa640 1 -- 172.21.15.106:0/716811340 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff898037040 con 0x7ff8a40c7ad0 2024-04-25T10:30:41.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.156+0000 7ff8a17fa640 1 -- 172.21.15.106:0/716811340 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff89803b4f0 con 0x7ff8a40c7ad0 2024-04-25T10:30:41.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.156+0000 7ff8a17fa640 1 -- 172.21.15.106:0/716811340 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff89803b8e0 con 0x7ff8a40c7ad0 2024-04-25T10:30:41.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.156+0000 7ff8aabd6640 1 -- 172.21.15.106:0/716811340 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8a41fac30 con 0x7ff8a40c7ad0 2024-04-25T10:30:41.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.156+0000 7f2336eab640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f231c0039d0 0x7f23301e3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.156+0000 7f2335ea9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f231c0039d0 0x7f23301e3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:41.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.156+0000 7f2335ea9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f231c0039d0 0x7f23301e3bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47976/0 (socket says 172.21.15.106:47976) 2024-04-25T10:30:41.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.157+0000 7f2335ea9640 1 -- 172.21.15.106:0/2355381119 learned_addr learned my addr 172.21.15.106:0/2355381119 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:41.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.157+0000 7f2336eab640 1 --2- 172.21.15.106:0/2355381119 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f233015d890 0x7f23300c47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.157+0000 7f2336eab640 1 --2- 172.21.15.106:0/2355381119 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2330155cc0 0x7f23301560d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.157+0000 7f23356a8640 1 --2- 172.21.15.106:0/2355381119 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f233015d890 0x7f23300c47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:41.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.157+0000 7f23366aa640 1 --2- 172.21.15.106:0/2355381119 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2330155cc0 0x7f23301560d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:41.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.156+0000 7f1edcff9640 1 -- 172.21.15.106:0/72500340 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed00125f0 con 0x7f1ee00ce440 2024-04-25T10:30:41.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.157+0000 7f1ee573c640 1 -- 172.21.15.106:0/72500340 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ee01ea280 con 0x7f1ee00ce440 2024-04-25T10:30:41.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.157+0000 7f1ee573c640 1 -- 172.21.15.106:0/72500340 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ee01ea840 con 0x7f1ee00ce440 2024-04-25T10:30:41.207 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.156+0000 7f0753fff640 1 --2- 172.21.15.106:0/934720775 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f0758074500 0x7f0758076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.207 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.157+0000 7f0753fff640 1 -- 172.21.15.106:0/934720775 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f07640959d0 con 0x7f076c153280 2024-04-25T10:30:41.207 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.157+0000 7f076affd640 1 --2- 172.21.15.106:0/934720775 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f0758074500 0x7f0758076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:41.207 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.156+0000 7ff8aabd6640 1 -- 172.21.15.106:0/716811340 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8a40d1e80 con 0x7ff8a40c7ad0 2024-04-25T10:30:41.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.157+0000 7f2336eab640 1 -- 172.21.15.106:0/2355381119 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2330072e80 con 0x7f231c0039d0 2024-04-25T10:30:41.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.158+0000 7f2336eab640 1 -- 172.21.15.106:0/2355381119 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f233015ddb0 con 0x7f233015d890 2024-04-25T10:30:41.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.158+0000 7f2336eab640 1 -- 172.21.15.106:0/2355381119 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f233006ffc0 con 0x7f2330155cc0 2024-04-25T10:30:41.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.158+0000 7f076affd640 1 --2- 172.21.15.106:0/934720775 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f0758074500 0x7f0758076a10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0760037000 tx=0x7f0760032000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:41.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.156+0000 7f37637fe640 1 -- 172.21.15.106:0/2112450836 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3754041020 con 0x7f376c152f10 2024-04-25T10:30:41.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.156+0000 7f37637fe640 1 -- 172.21.15.106:0/2112450836 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37540325f0 con 0x7f376c152f10 2024-04-25T10:30:41.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.156+0000 7f37637fe640 1 -- 172.21.15.106:0/2112450836 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3754037040 con 0x7f376c152f10 2024-04-25T10:30:41.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.156+0000 7f3771700640 1 -- 172.21.15.106:0/2112450836 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f376c0c9730 con 0x7f376c152f10 2024-04-25T10:30:41.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.156+0000 7f3771700640 1 -- 172.21.15.106:0/2112450836 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f376c148de0 con 0x7f376c152f10 2024-04-25T10:30:41.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.158+0000 7ff8aabd6640 1 -- 172.21.15.106:0/716811340 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8a406a3d0 con 0x7ff8a40c7ad0 2024-04-25T10:30:41.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.158+0000 7f3771700640 1 -- 172.21.15.106:0/2112450836 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f376c1517c0 con 0x7f376c152f10 2024-04-25T10:30:41.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.158+0000 7f1edcff9640 1 -- 172.21.15.106:0/72500340 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ed0019070 con 0x7f1ee00ce440 2024-04-25T10:30:41.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.158+0000 7f4792ffd640 1 -- 172.21.15.106:0/2802364784 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a406d950 con 0x7f479c006cb0 2024-04-25T10:30:41.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.158+0000 7f4792ffd640 1 -- 172.21.15.106:0/2802364784 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47a4092020 con 0x7f479c006cb0 2024-04-25T10:30:41.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.158+0000 7f4792ffd640 1 -- 172.21.15.106:0/2802364784 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a406dd70 con 0x7f479c006cb0 2024-04-25T10:30:41.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.158+0000 7f47a359e640 1 -- 172.21.15.106:0/2802364784 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f479c04ef80 con 0x7f479c006cb0 2024-04-25T10:30:41.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.158+0000 7f47a359e640 1 -- 172.21.15.106:0/2802364784 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f479c04f540 con 0x7f479c006cb0 2024-04-25T10:30:41.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.158+0000 7f1ee573c640 1 -- 172.21.15.106:0/72500340 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ee0050740 con 0x7f1ee00ce440 2024-04-25T10:30:41.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.158+0000 7f2335ea9640 1 -- 172.21.15.106:0/2355381119 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2330155cc0 msgr2=0x7f23301560d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:41.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.158+0000 7f1edcff9640 1 -- 172.21.15.106:0/72500340 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed0012990 con 0x7f1ee00ce440 2024-04-25T10:30:41.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.158+0000 7f2335ea9640 1 --2- 172.21.15.106:0/2355381119 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2330155cc0 0x7f23301560d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.158+0000 7f2335ea9640 1 -- 172.21.15.106:0/2355381119 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f233015d890 msgr2=0x7f23300c47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:41.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.158+0000 7f2335ea9640 1 --2- 172.21.15.106:0/2355381119 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f233015d890 0x7f23300c47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.158+0000 7f2335ea9640 1 -- 172.21.15.106:0/2355381119 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2330156a10 con 0x7f231c0039d0 2024-04-25T10:30:41.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.159+0000 7f2335ea9640 1 --2- 172.21.15.106:0/2355381119 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f231c0039d0 0x7f23301e3bd0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f232c000c00 tx=0x7f232c001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:41.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.159+0000 7ff8a17fa640 1 -- 172.21.15.106:0/716811340 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff898044020 con 0x7ff8a40c7ad0 2024-04-25T10:30:41.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.159+0000 7f0753fff640 1 -- 172.21.15.106:0/934720775 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f07640602d0 con 0x7f076c153280 2024-04-25T10:30:41.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.159+0000 7ff8a17fa640 1 --2- 172.21.15.106:0/716811340 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7ff8900745b0 0x7ff890076ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.160+0000 7ff8a17fa640 1 -- 172.21.15.106:0/716811340 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff8980b53d0 con 0x7ff8a40c7ad0 2024-04-25T10:30:41.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.160+0000 7ff8a8972640 1 --2- 172.21.15.106:0/716811340 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7ff8900745b0 0x7ff890076ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:41.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.160+0000 7f47a359e640 1 -- 172.21.15.106:0/2802364784 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f479c0d21a0 con 0x7f479c006cb0 2024-04-25T10:30:41.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.161+0000 7ff8a8972640 1 --2- 172.21.15.106:0/716811340 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7ff8900745b0 0x7ff890076ac0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff894006fd0 tx=0x7ff894008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:41.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.161+0000 7f37637fe640 1 -- 172.21.15.106:0/2112450836 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3754044020 con 0x7f376c152f10 2024-04-25T10:30:41.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.161+0000 7f23267fc640 1 -- 172.21.15.106:0/2355381119 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f232c036070 con 0x7f231c0039d0 2024-04-25T10:30:41.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.161+0000 7f2336eab640 1 -- 172.21.15.106:0/2355381119 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23301e30f0 con 0x7f231c0039d0 2024-04-25T10:30:41.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.161+0000 7f2336eab640 1 -- 172.21.15.106:0/2355381119 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23301e36b0 con 0x7f231c0039d0 2024-04-25T10:30:41.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.161+0000 7f37637fe640 1 --2- 172.21.15.106:0/2112450836 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f374c074510 0x7f374c076a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.162+0000 7f376affd640 1 --2- 172.21.15.106:0/2112450836 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f374c074510 0x7f374c076a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:41.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.162+0000 7f1edcff9640 1 -- 172.21.15.106:0/72500340 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1ed0012b30 con 0x7f1ee00ce440 2024-04-25T10:30:41.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.162+0000 7f1edcff9640 1 --2- 172.21.15.106:0/72500340 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f1ecc0744b0 0x7f1ecc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.162+0000 7f1edcff9640 1 -- 172.21.15.106:0/72500340 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1ed0095fb0 con 0x7f1ee00ce440 2024-04-25T10:30:41.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.163+0000 7f376affd640 1 --2- 172.21.15.106:0/2112450836 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f374c074510 0x7f374c076a20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f375c00c530 tx=0x7f375c00b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:41.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.161+0000 7f5fa2ffd640 1 -- 172.21.15.106:0/2958742256 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5fac06e660 con 0x7f5fa40d56f0 2024-04-25T10:30:41.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.162+0000 7f5fa2ffd640 1 --2- 172.21.15.106:0/2958742256 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f5f90074580 0x7f5f90076a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.162+0000 7f5fa2ffd640 1 -- 172.21.15.106:0/2958742256 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5fac112ca0 con 0x7f5fa40d56f0 2024-04-25T10:30:41.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.162+0000 7f4792ffd640 1 -- 172.21.15.106:0/2802364784 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f47a408e6b0 con 0x7f479c006cb0 2024-04-25T10:30:41.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.162+0000 7f4792ffd640 1 --2- 172.21.15.106:0/2802364784 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f4784074500 0x7f4784076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.163+0000 7f5fb218f640 1 --2- 172.21.15.106:0/2958742256 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f5f90074580 0x7f5f90076a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:41.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.163+0000 7f47a1d9b640 1 --2- 172.21.15.106:0/2802364784 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f4784074500 0x7f4784076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:41.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.163+0000 7f5fb218f640 1 --2- 172.21.15.106:0/2958742256 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f5f90074580 0x7f5f90076a90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c0011e0 tx=0x7f5f9c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:41.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.163+0000 7f1edffff640 1 --2- 172.21.15.106:0/72500340 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f1ecc0744b0 0x7f1ecc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:41.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.164+0000 7f47a1d9b640 1 --2- 172.21.15.106:0/2802364784 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f4784074500 0x7f4784076a10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4798036000 tx=0x7f4798030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:41.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.164+0000 7f2336eab640 1 -- 172.21.15.106:0/2355381119 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23300cab50 con 0x7f231c0039d0 2024-04-25T10:30:41.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.164+0000 7f1edffff640 1 --2- 172.21.15.106:0/72500340 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f1ecc0744b0 0x7f1ecc0769c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1ed4006fd0 tx=0x7f1ed4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:41.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.164+0000 7f5fa2ffd640 1 -- 172.21.15.106:0/2958742256 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5fac0dd530 con 0x7f5fa40d56f0 2024-04-25T10:30:41.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.168+0000 7ff8a17fa640 1 -- 172.21.15.106:0/716811340 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff89807fbd0 con 0x7ff8a40c7ad0 2024-04-25T10:30:41.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.169+0000 7f37637fe640 1 -- 172.21.15.106:0/2112450836 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f375407c7e0 con 0x7f376c152f10 2024-04-25T10:30:41.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.173+0000 7f4792ffd640 1 -- 172.21.15.106:0/2802364784 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f47a40d8550 con 0x7f479c006cb0 2024-04-25T10:30:41.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.173+0000 7f1edcff9640 1 -- 172.21.15.106:0/72500340 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1ed0060840 con 0x7f1ee00ce440 2024-04-25T10:30:41.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.177+0000 7f23267fc640 1 -- 172.21.15.106:0/2355381119 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f232c042070 con 0x7f231c0039d0 2024-04-25T10:30:41.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.177+0000 7f23267fc640 1 -- 172.21.15.106:0/2355381119 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f232c00c0e0 con 0x7f231c0039d0 2024-04-25T10:30:41.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.178+0000 7f37637fe640 1 -- 172.21.15.106:0/2112450836 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f376c1517c0 con 0x7f376c152f10 2024-04-25T10:30:41.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.188+0000 7f4792ffd640 1 -- 172.21.15.106:0/2802364784 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f47a409c030 con 0x7f479c006cb0 2024-04-25T10:30:41.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.192+0000 7f23267fc640 1 -- 172.21.15.106:0/2355381119 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f232c02e070 con 0x7f231c0039d0 2024-04-25T10:30:41.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.193+0000 7f23267fc640 1 --2- 172.21.15.106:0/2355381119 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f23140744b0 0x7f23140769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.193+0000 7f23267fc640 1 -- 172.21.15.106:0/2355381119 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f232c0b6330 con 0x7f231c0039d0 2024-04-25T10:30:41.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.195+0000 7f23356a8640 1 --2- 172.21.15.106:0/2355381119 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f23140744b0 0x7f23140769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:41.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.196+0000 7f23356a8640 1 --2- 172.21.15.106:0/2355381119 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f23140744b0 0x7f23140769c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f23300710a0 tx=0x7f2320009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:41.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.200+0000 7f23267fc640 1 -- 172.21.15.106:0/2355381119 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f232c02c040 con 0x7f231c0039d0 2024-04-25T10:30:41.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.477+0000 7f09f7534640 1 -- 172.21.15.106:0/1757895629 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f09f006a3d0 con 0x7f09f00ca120 2024-04-25T10:30:41.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.478+0000 7f09e5ffb640 1 -- 172.21.15.106:0/1757895629 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f09e8080f20 con 0x7f09f00ca120 2024-04-25T10:30:41.481 INFO:teuthology.orchestra.run.smithi106.stdout:55834574855 2024-04-25T10:30:41.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.482+0000 7f09db7fe640 1 -- 172.21.15.106:0/1757895629 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f09d4074580 msgr2=0x7f09d4076a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:41.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.482+0000 7f09db7fe640 1 --2- 172.21.15.106:0/1757895629 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f09d4074580 0x7f09d4076a90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f09e0006fd0 tx=0x7f09e0008040 comp rx=0 tx=0).stop 2024-04-25T10:30:41.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.482+0000 7f09db7fe640 1 -- 172.21.15.106:0/1757895629 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09f00ca120 msgr2=0x7f09f014afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:41.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.482+0000 7f09db7fe640 1 --2- 172.21.15.106:0/1757895629 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09f00ca120 0x7f09f014afb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f09e8031000 tx=0x7f09e8004a40 comp rx=0 tx=0).stop 2024-04-25T10:30:41.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.483+0000 7f09db7fe640 1 -- 172.21.15.106:0/1757895629 shutdown_connections 2024-04-25T10:30:41.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.483+0000 7f09db7fe640 1 --2- 172.21.15.106:0/1757895629 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f09d4074580 0x7f09d4076a90 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.483+0000 7f09db7fe640 1 --2- 172.21.15.106:0/1757895629 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09f00ca120 0x7f09f014afb0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.483+0000 7f09db7fe640 1 --2- 172.21.15.106:0/1757895629 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09f00c99f0 0x7f09f0160b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.483+0000 7f09db7fe640 1 --2- 172.21.15.106:0/1757895629 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09f0072870 0x7f09f01fad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.483+0000 7f09db7fe640 1 -- 172.21.15.106:0/1757895629 >> 172.21.15.106:0/1757895629 conn(0x7f09f006c9b0 msgr2=0x7f09f0158640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:41.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.483+0000 7f09db7fe640 1 -- 172.21.15.106:0/1757895629 shutdown_connections 2024-04-25T10:30:41.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.483+0000 7f09db7fe640 1 -- 172.21.15.106:0/1757895629 wait complete. 2024-04-25T10:30:41.502 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.7 2024-04-25T10:30:41.503 DEBUG:teuthology.parallel:result is None 2024-04-25T10:30:41.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.543+0000 7f47a359e640 1 -- 172.21.15.106:0/2802364784 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f479c0d91a0 con 0x7f479c006cb0 2024-04-25T10:30:41.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.543+0000 7f4792ffd640 1 -- 172.21.15.106:0/2802364784 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f47a40dc9c0 con 0x7f479c006cb0 2024-04-25T10:30:41.546 INFO:teuthology.orchestra.run.smithi106.stdout:55834574855 2024-04-25T10:30:41.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.545+0000 7f47a359e640 1 -- 172.21.15.106:0/2802364784 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f4784074500 msgr2=0x7f4784076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:41.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.545+0000 7f47a359e640 1 --2- 172.21.15.106:0/2802364784 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f4784074500 0x7f4784076a10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4798036000 tx=0x7f4798030000 comp rx=0 tx=0).stop 2024-04-25T10:30:41.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.546+0000 7f47a359e640 1 -- 172.21.15.106:0/2802364784 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f479c006cb0 msgr2=0x7f479c0d12b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:41.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.546+0000 7f47a359e640 1 --2- 172.21.15.106:0/2802364784 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f479c006cb0 0x7f479c0d12b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f47a40514f0 tx=0x7f47a40783f0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.546+0000 7f47a359e640 1 -- 172.21.15.106:0/2802364784 shutdown_connections 2024-04-25T10:30:41.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.546+0000 7f47a359e640 1 --2- 172.21.15.106:0/2802364784 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f4784074500 0x7f4784076a10 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.546+0000 7f47a359e640 1 --2- 172.21.15.106:0/2802364784 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f479c04d1c0 0x7f479c0c6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.546+0000 7f47a359e640 1 --2- 172.21.15.106:0/2802364784 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f479c045be0 0x7f479c17db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.546+0000 7f47a359e640 1 --2- 172.21.15.106:0/2802364784 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f479c006cb0 0x7f479c0d12b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.546+0000 7f47a359e640 1 -- 172.21.15.106:0/2802364784 >> 172.21.15.106:0/2802364784 conn(0x7f479c050c40 msgr2=0x7f479c007150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:41.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.548+0000 7f47a359e640 1 -- 172.21.15.106:0/2802364784 shutdown_connections 2024-04-25T10:30:41.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.548+0000 7f47a359e640 1 -- 172.21.15.106:0/2802364784 wait complete. 2024-04-25T10:30:41.561 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.2 2024-04-25T10:30:41.562 DEBUG:teuthology.parallel:result is None 2024-04-25T10:30:41.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.584+0000 7f0771a51640 1 -- 172.21.15.106:0/934720775 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f074c005850 con 0x7f076c153280 2024-04-25T10:30:41.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.585+0000 7f0753fff640 1 -- 172.21.15.106:0/934720775 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f076405fbd0 con 0x7f076c153280 2024-04-25T10:30:41.589 INFO:teuthology.orchestra.run.smithi106.stdout:55834574855 2024-04-25T10:30:41.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.587+0000 7f5fa0ff9640 1 -- 172.21.15.106:0/2958742256 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f5fa40ce530 con 0x7f5fa40d56f0 2024-04-25T10:30:41.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.587+0000 7f5fa2ffd640 1 -- 172.21.15.106:0/2958742256 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5fac0dce30 con 0x7f5fa40d56f0 2024-04-25T10:30:41.590 INFO:teuthology.orchestra.run.smithi106.stdout:55834574855 2024-04-25T10:30:41.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.588+0000 7f0771a51640 1 -- 172.21.15.106:0/934720775 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f0758074500 msgr2=0x7f0758076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:41.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.588+0000 7f0771a51640 1 --2- 172.21.15.106:0/934720775 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f0758074500 0x7f0758076a10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0760037000 tx=0x7f0760032000 comp rx=0 tx=0).stop 2024-04-25T10:30:41.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.588+0000 7f0771a51640 1 -- 172.21.15.106:0/934720775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f076c153280 msgr2=0x7f076c14ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:41.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.588+0000 7f0771a51640 1 --2- 172.21.15.106:0/934720775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f076c153280 0x7f076c14ec10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f076400c780 tx=0x7f076400cc40 comp rx=0 tx=0).stop 2024-04-25T10:30:41.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.588+0000 7f0771a51640 1 -- 172.21.15.106:0/934720775 shutdown_connections 2024-04-25T10:30:41.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.588+0000 7f0771a51640 1 --2- 172.21.15.106:0/934720775 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f0758074500 0x7f0758076a10 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.588+0000 7f0771a51640 1 --2- 172.21.15.106:0/934720775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f076c153280 0x7f076c14ec10 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.588+0000 7f0771a51640 1 --2- 172.21.15.106:0/934720775 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f076c0cc0d0 0x7f076c1584f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.588+0000 7f0771a51640 1 --2- 172.21.15.106:0/934720775 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f076c0c3350 0x7f076c1f6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.588+0000 7f0771a51640 1 -- 172.21.15.106:0/934720775 >> 172.21.15.106:0/934720775 conn(0x7f076c070ed0 msgr2=0x7f076c1e46c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:41.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.588+0000 7f0771a51640 1 -- 172.21.15.106:0/934720775 shutdown_connections 2024-04-25T10:30:41.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.588+0000 7f0771a51640 1 -- 172.21.15.106:0/934720775 wait complete. 2024-04-25T10:30:41.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.588+0000 7ff8aabd6640 1 -- 172.21.15.106:0/716811340 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7ff8a4073ad0 con 0x7ff8a40c7ad0 2024-04-25T10:30:41.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.589+0000 7ff8a17fa640 1 -- 172.21.15.106:0/716811340 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff89807f4d0 con 0x7ff8a40c7ad0 2024-04-25T10:30:41.593 INFO:teuthology.orchestra.run.smithi106.stdout:55834574855 2024-04-25T10:30:41.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.590+0000 7f5fa0ff9640 1 -- 172.21.15.106:0/2958742256 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f5f90074580 msgr2=0x7f5f90076a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:41.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.590+0000 7f5fa0ff9640 1 --2- 172.21.15.106:0/2958742256 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f5f90074580 0x7f5f90076a90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c0011e0 tx=0x7f5f9c009040 comp rx=0 tx=0).stop 2024-04-25T10:30:41.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.590+0000 7f5fa0ff9640 1 -- 172.21.15.106:0/2958742256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fa40d56f0 msgr2=0x7f5fa40c81e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:41.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.590+0000 7f5fa0ff9640 1 --2- 172.21.15.106:0/2958742256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fa40d56f0 0x7f5fa40c81e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5fac052040 tx=0x7f5fac0784b0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.590+0000 7f5fa0ff9640 1 -- 172.21.15.106:0/2958742256 shutdown_connections 2024-04-25T10:30:41.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.590+0000 7f5fa0ff9640 1 --2- 172.21.15.106:0/2958742256 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f5f90074580 0x7f5f90076a90 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.590+0000 7f5fa0ff9640 1 --2- 172.21.15.106:0/2958742256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fa40d56f0 0x7f5fa40c81e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.590+0000 7f5fa0ff9640 1 --2- 172.21.15.106:0/2958742256 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5fa40334e0 0x7f5fa40c7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.590+0000 7f5fa0ff9640 1 --2- 172.21.15.106:0/2958742256 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5fa4006a90 0x7f5fa40c9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.590+0000 7f5fa0ff9640 1 -- 172.21.15.106:0/2958742256 >> 172.21.15.106:0/2958742256 conn(0x7f5fa40503b0 msgr2=0x7f5fa404bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:41.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.590+0000 7f5fa0ff9640 1 -- 172.21.15.106:0/2958742256 shutdown_connections 2024-04-25T10:30:41.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.590+0000 7f5fa0ff9640 1 -- 172.21.15.106:0/2958742256 wait complete. 2024-04-25T10:30:41.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.592+0000 7ff8aabd6640 1 -- 172.21.15.106:0/716811340 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7ff8900745b0 msgr2=0x7ff890076ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:41.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.592+0000 7ff8aabd6640 1 --2- 172.21.15.106:0/716811340 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7ff8900745b0 0x7ff890076ac0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff894006fd0 tx=0x7ff894008040 comp rx=0 tx=0).stop 2024-04-25T10:30:41.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.592+0000 7ff8aabd6640 1 -- 172.21.15.106:0/716811340 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8a40c7ad0 msgr2=0x7ff8a414fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:41.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.592+0000 7ff8aabd6640 1 --2- 172.21.15.106:0/716811340 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8a40c7ad0 0x7ff8a414fca0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff8980098c0 tx=0x7ff8980046c0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.592+0000 7ff8aabd6640 1 -- 172.21.15.106:0/716811340 shutdown_connections 2024-04-25T10:30:41.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.592+0000 7ff8aabd6640 1 --2- 172.21.15.106:0/716811340 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7ff8900745b0 0x7ff890076ac0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.592+0000 7ff8aabd6640 1 --2- 172.21.15.106:0/716811340 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8a4156f70 0x7ff8a40d0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.592+0000 7ff8aabd6640 1 --2- 172.21.15.106:0/716811340 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8a40c7ad0 0x7ff8a414fca0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.592+0000 7ff8aabd6640 1 --2- 172.21.15.106:0/716811340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8a406daa0 0x7ff8a41fb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.592+0000 7ff8aabd6640 1 -- 172.21.15.106:0/716811340 >> 172.21.15.106:0/716811340 conn(0x7ff8a4068ed0 msgr2=0x7ff8a40b6660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:41.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.593+0000 7ff8aabd6640 1 -- 172.21.15.106:0/716811340 shutdown_connections 2024-04-25T10:30:41.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.593+0000 7ff8aabd6640 1 -- 172.21.15.106:0/716811340 wait complete. 2024-04-25T10:30:41.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.594+0000 7f1ee573c640 1 -- 172.21.15.106:0/72500340 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f1ee01613d0 con 0x7f1ee00ce440 2024-04-25T10:30:41.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.595+0000 7f1edcff9640 1 -- 172.21.15.106:0/72500340 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1ed0060140 con 0x7f1ee00ce440 2024-04-25T10:30:41.598 INFO:teuthology.orchestra.run.smithi106.stdout:55834574856 2024-04-25T10:30:41.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.598+0000 7f1ee573c640 1 -- 172.21.15.106:0/72500340 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f1ecc0744b0 msgr2=0x7f1ecc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:41.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.598+0000 7f1ee573c640 1 --2- 172.21.15.106:0/72500340 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f1ecc0744b0 0x7f1ecc0769c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1ed4006fd0 tx=0x7f1ed4008040 comp rx=0 tx=0).stop 2024-04-25T10:30:41.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.598+0000 7f1ee573c640 1 -- 172.21.15.106:0/72500340 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1ee00ce440 msgr2=0x7f1ee0151580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:41.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.598+0000 7f1ee573c640 1 --2- 172.21.15.106:0/72500340 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1ee00ce440 0x7f1ee0151580 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1ed000d520 tx=0x7f1ed000d9e0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.599+0000 7f1ee573c640 1 -- 172.21.15.106:0/72500340 shutdown_connections 2024-04-25T10:30:41.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.599+0000 7f1ee573c640 1 --2- 172.21.15.106:0/72500340 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f1ecc0744b0 0x7f1ecc0769c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.599+0000 7f1ee573c640 1 --2- 172.21.15.106:0/72500340 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1ee0157a20 0x7f1ee0160540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.599+0000 7f1ee573c640 1 --2- 172.21.15.106:0/72500340 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1ee00ce440 0x7f1ee0151580 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.599+0000 7f1ee573c640 1 --2- 172.21.15.106:0/72500340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ee00c9ba0 0x7f1ee0164c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.599+0000 7f1ee573c640 1 -- 172.21.15.106:0/72500340 >> 172.21.15.106:0/72500340 conn(0x7f1ee006c9e0 msgr2=0x7f1ee00ca600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:41.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.599+0000 7f1ee573c640 1 -- 172.21.15.106:0/72500340 shutdown_connections 2024-04-25T10:30:41.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.599+0000 7f1ee573c640 1 -- 172.21.15.106:0/72500340 wait complete. 2024-04-25T10:30:41.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.605+0000 7f3771700640 1 -- 172.21.15.106:0/2112450836 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f376c050740 con 0x7f376c152f10 2024-04-25T10:30:41.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.605+0000 7f37637fe640 1 -- 172.21.15.106:0/2112450836 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f376c1eee00 con 0x7f376c152f10 2024-04-25T10:30:41.608 INFO:teuthology.orchestra.run.smithi106.stdout:55834574856 2024-04-25T10:30:41.609 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.6 2024-04-25T10:30:41.609 DEBUG:teuthology.parallel:result is None 2024-04-25T10:30:41.611 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.1 2024-04-25T10:30:41.612 DEBUG:teuthology.parallel:result is None 2024-04-25T10:30:41.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.609+0000 7f3771700640 1 -- 172.21.15.106:0/2112450836 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f374c074510 msgr2=0x7f374c076a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:41.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.609+0000 7f3771700640 1 --2- 172.21.15.106:0/2112450836 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f374c074510 0x7f374c076a20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f375c00c530 tx=0x7f375c00b040 comp rx=0 tx=0).stop 2024-04-25T10:30:41.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.609+0000 7f3771700640 1 -- 172.21.15.106:0/2112450836 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f376c152f10 msgr2=0x7f376c148450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:41.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.609+0000 7f3771700640 1 --2- 172.21.15.106:0/2112450836 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f376c152f10 0x7f376c148450 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3754031000 tx=0x7f37540046c0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.609+0000 7f3771700640 1 -- 172.21.15.106:0/2112450836 shutdown_connections 2024-04-25T10:30:41.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.609+0000 7f3771700640 1 --2- 172.21.15.106:0/2112450836 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f374c074510 0x7f374c076a20 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.609+0000 7f3771700640 1 --2- 172.21.15.106:0/2112450836 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f376c158bd0 0x7f376c1e5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.609+0000 7f3771700640 1 --2- 172.21.15.106:0/2112450836 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f376c152f10 0x7f376c148450 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.609+0000 7f3771700640 1 --2- 172.21.15.106:0/2112450836 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f376c070120 0x7f376c1f23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.609+0000 7f3771700640 1 -- 172.21.15.106:0/2112450836 >> 172.21.15.106:0/2112450836 conn(0x7f376c06c9e0 msgr2=0x7f376c075c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:41.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.609+0000 7f3771700640 1 -- 172.21.15.106:0/2112450836 shutdown_connections 2024-04-25T10:30:41.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.609+0000 7f3771700640 1 -- 172.21.15.106:0/2112450836 wait complete. 2024-04-25T10:30:41.613 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.3 2024-04-25T10:30:41.613 DEBUG:teuthology.parallel:result is None 2024-04-25T10:30:41.617 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.5 2024-04-25T10:30:41.617 DEBUG:teuthology.parallel:result is None 2024-04-25T10:30:41.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.618+0000 7f2336eab640 1 -- 172.21.15.106:0/2355381119 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f23301581e0 con 0x7f231c0039d0 2024-04-25T10:30:41.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.618+0000 7f23267fc640 1 -- 172.21.15.106:0/2355381119 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f232c080c80 con 0x7f231c0039d0 2024-04-25T10:30:41.620 INFO:teuthology.orchestra.run.smithi106.stdout:55834574855 2024-04-25T10:30:41.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.620+0000 7f2336eab640 1 -- 172.21.15.106:0/2355381119 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f23140744b0 msgr2=0x7f23140769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:41.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.620+0000 7f2336eab640 1 --2- 172.21.15.106:0/2355381119 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f23140744b0 0x7f23140769c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f23300710a0 tx=0x7f2320009190 comp rx=0 tx=0).stop 2024-04-25T10:30:41.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.620+0000 7f2336eab640 1 -- 172.21.15.106:0/2355381119 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f231c0039d0 msgr2=0x7f23301e3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:41.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.620+0000 7f2336eab640 1 --2- 172.21.15.106:0/2355381119 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f231c0039d0 0x7f23301e3bd0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f232c000c00 tx=0x7f232c001a40 comp rx=0 tx=0).stop 2024-04-25T10:30:41.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.620+0000 7f2336eab640 1 -- 172.21.15.106:0/2355381119 shutdown_connections 2024-04-25T10:30:41.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.620+0000 7f2336eab640 1 --2- 172.21.15.106:0/2355381119 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f23140744b0 0x7f23140769c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.620+0000 7f2336eab640 1 --2- 172.21.15.106:0/2355381119 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2330155cc0 0x7f23301560d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.620+0000 7f2336eab640 1 --2- 172.21.15.106:0/2355381119 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f233015d890 0x7f23300c47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.620+0000 7f2336eab640 1 --2- 172.21.15.106:0/2355381119 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f231c0039d0 0x7f23301e3bd0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.620+0000 7f2336eab640 1 -- 172.21.15.106:0/2355381119 >> 172.21.15.106:0/2355381119 conn(0x7f23300cb070 msgr2=0x7f23301518a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:41.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.620+0000 7f2336eab640 1 -- 172.21.15.106:0/2355381119 shutdown_connections 2024-04-25T10:30:41.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.620+0000 7f2336eab640 1 -- 172.21.15.106:0/2355381119 wait complete. 2024-04-25T10:30:41.624 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.4 2024-04-25T10:30:41.624 DEBUG:teuthology.parallel:result is None 2024-04-25T10:30:41.633 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.0 2024-04-25T10:30:41.634 DEBUG:teuthology.parallel:result is None 2024-04-25T10:30:41.634 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T10:30:41.634 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:30:41.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.779+0000 7fbcca6e0640 1 Processor -- start 2024-04-25T10:30:41.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.779+0000 7fbcca6e0640 1 -- start start 2024-04-25T10:30:41.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.780+0000 7fbcca6e0640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fbcc40ca170 0x7fbcc4156f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.780+0000 7fbcca6e0640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbcc406c5c0 0x7fbcc406c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.780+0000 7fbcca6e0640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fbcc415ce00 0x7fbcc415d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.780+0000 7fbcca6e0640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fbcc4073f00 con 0x7fbcc40ca170 2024-04-25T10:30:41.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.780+0000 7fbcca6e0640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fbcc4074990 con 0x7fbcc415ce00 2024-04-25T10:30:41.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.780+0000 7fbcca6e0640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fbcc4073980 con 0x7fbcc406c5c0 2024-04-25T10:30:41.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.780+0000 7fbcc3fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fbcc40ca170 0x7fbcc4156f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:41.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.780+0000 7fbcc3fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fbcc40ca170 0x7fbcc4156f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47986/0 (socket says 172.21.15.106:47986) 2024-04-25T10:30:41.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.780+0000 7fbcc3fff640 1 -- 172.21.15.106:0/1033521277 learned_addr learned my addr 172.21.15.106:0/1033521277 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:41.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.780+0000 7fbcc8c7d640 1 --2- 172.21.15.106:0/1033521277 >> v2:172.21.15.134:3300/0 conn(0x7fbcc415ce00 0x7fbcc415d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:41.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.780+0000 7fbcc37fe640 1 --2- 172.21.15.106:0/1033521277 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbcc406c5c0 0x7fbcc406c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:41.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.780+0000 7fbcc3fff640 1 -- 172.21.15.106:0/1033521277 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbcc406c5c0 msgr2=0x7fbcc406c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:41.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.780+0000 7fbcc3fff640 1 --2- 172.21.15.106:0/1033521277 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbcc406c5c0 0x7fbcc406c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.781+0000 7fbcc3fff640 1 -- 172.21.15.106:0/1033521277 >> v2:172.21.15.134:3300/0 conn(0x7fbcc415ce00 msgr2=0x7fbcc415d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:41.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.781+0000 7fbcc3fff640 1 --2- 172.21.15.106:0/1033521277 >> v2:172.21.15.134:3300/0 conn(0x7fbcc415ce00 0x7fbcc415d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.781+0000 7fbcc3fff640 1 -- 172.21.15.106:0/1033521277 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcc4160ea0 con 0x7fbcc40ca170 2024-04-25T10:30:41.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.781+0000 7fbcc3fff640 1 --2- 172.21.15.106:0/1033521277 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbcc40ca170 0x7fbcc4156f70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fbcc40ceeb0 tx=0x7fbcb402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ccaa94607341f6b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:41.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.781+0000 7fbcc2ffd640 1 -- 172.21.15.106:0/1033521277 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb4037070 con 0x7fbcc40ca170 2024-04-25T10:30:41.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.781+0000 7fbcc2ffd640 1 -- 172.21.15.106:0/1033521277 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcb4032440 con 0x7fbcc40ca170 2024-04-25T10:30:41.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.781+0000 7fbcc2ffd640 1 -- 172.21.15.106:0/1033521277 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb4032810 con 0x7fbcc40ca170 2024-04-25T10:30:41.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.782+0000 7fbcca6e0640 1 -- 172.21.15.106:0/1033521277 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbcc40ca170 msgr2=0x7fbcc4156f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:41.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.782+0000 7fbcca6e0640 1 --2- 172.21.15.106:0/1033521277 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbcc40ca170 0x7fbcc4156f70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fbcc40ceeb0 tx=0x7fbcb402a270 comp rx=0 tx=0).stop 2024-04-25T10:30:41.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.782+0000 7fbcca6e0640 1 -- 172.21.15.106:0/1033521277 shutdown_connections 2024-04-25T10:30:41.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.782+0000 7fbcca6e0640 1 --2- 172.21.15.106:0/1033521277 >> v2:172.21.15.134:3300/0 conn(0x7fbcc415ce00 0x7fbcc415d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.782+0000 7fbcca6e0640 1 --2- 172.21.15.106:0/1033521277 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbcc406c5c0 0x7fbcc406c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.782+0000 7fbcca6e0640 1 --2- 172.21.15.106:0/1033521277 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbcc40ca170 0x7fbcc4156f70 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.782+0000 7fbcca6e0640 1 -- 172.21.15.106:0/1033521277 >> 172.21.15.106:0/1033521277 conn(0x7fbcc40cef20 msgr2=0x7fbcc40cf330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:41.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.782+0000 7fbcca6e0640 1 -- 172.21.15.106:0/1033521277 shutdown_connections 2024-04-25T10:30:41.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.782+0000 7fbcca6e0640 1 -- 172.21.15.106:0/1033521277 wait complete. 2024-04-25T10:30:41.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.784+0000 7fbcca6e0640 1 Processor -- start 2024-04-25T10:30:41.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.784+0000 7fbcca6e0640 1 -- start start 2024-04-25T10:30:41.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.785+0000 7fbcca6e0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbcc406c5c0 0x7fbcc415f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.785+0000 7fbcca6e0640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbcc40ca170 0x7fbcc414f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.785+0000 7fbcca6e0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbcc415ce00 0x7fbcc414faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.785+0000 7fbcca6e0640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc4073c20 con 0x7fbcc415ce00 2024-04-25T10:30:41.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.785+0000 7fbcca6e0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc4074640 con 0x7fbcc406c5c0 2024-04-25T10:30:41.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.785+0000 7fbcca6e0640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fbcc4161140 con 0x7fbcc40ca170 2024-04-25T10:30:41.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.785+0000 7fbcc8c7d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbcc415ce00 0x7fbcc414faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:41.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.785+0000 7fbcc8c7d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbcc415ce00 0x7fbcc414faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47998/0 (socket says 172.21.15.106:47998) 2024-04-25T10:30:41.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.785+0000 7fbcc8c7d640 1 -- 172.21.15.106:0/3608068873 learned_addr learned my addr 172.21.15.106:0/3608068873 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:41.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.786+0000 7fbcc3fff640 1 --2- 172.21.15.106:0/3608068873 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbcc406c5c0 0x7fbcc415f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:41.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.786+0000 7fbcc37fe640 1 --2- 172.21.15.106:0/3608068873 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbcc40ca170 0x7fbcc414f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:41.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.786+0000 7fbcc8c7d640 1 -- 172.21.15.106:0/3608068873 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbcc40ca170 msgr2=0x7fbcc414f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:41.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.786+0000 7fbcc8c7d640 1 --2- 172.21.15.106:0/3608068873 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbcc40ca170 0x7fbcc414f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.786+0000 7fbcc8c7d640 1 -- 172.21.15.106:0/3608068873 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbcc406c5c0 msgr2=0x7fbcc415f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:41.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.786+0000 7fbcc8c7d640 1 --2- 172.21.15.106:0/3608068873 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbcc406c5c0 0x7fbcc415f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.786+0000 7fbcc8c7d640 1 -- 172.21.15.106:0/3608068873 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcc41e71d0 con 0x7fbcc415ce00 2024-04-25T10:30:41.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.786+0000 7fbcc8c7d640 1 --2- 172.21.15.106:0/3608068873 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbcc415ce00 0x7fbcc414faf0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc00efc0 tx=0x7fbcbc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:41.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.786+0000 7fbcc0ff9640 1 -- 172.21.15.106:0/3608068873 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc018020 con 0x7fbcc415ce00 2024-04-25T10:30:41.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.786+0000 7fbcc0ff9640 1 -- 172.21.15.106:0/3608068873 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcbc014480 con 0x7fbcc415ce00 2024-04-25T10:30:41.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.786+0000 7fbcca6e0640 1 -- 172.21.15.106:0/3608068873 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcc415ab50 con 0x7fbcc415ce00 2024-04-25T10:30:41.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.786+0000 7fbcc0ff9640 1 -- 172.21.15.106:0/3608068873 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc009e60 con 0x7fbcc415ce00 2024-04-25T10:30:41.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.786+0000 7fbcca6e0640 1 -- 172.21.15.106:0/3608068873 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcc415ba00 con 0x7fbcc415ce00 2024-04-25T10:30:41.791 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.787+0000 7fbcca6e0640 1 -- 172.21.15.106:0/3608068873 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcc406a3d0 con 0x7fbcc415ce00 2024-04-25T10:30:41.791 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.788+0000 7fbcc0ff9640 1 -- 172.21.15.106:0/3608068873 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbcbc005d10 con 0x7fbcc415ce00 2024-04-25T10:30:41.791 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.788+0000 7fbcc0ff9640 1 --2- 172.21.15.106:0/3608068873 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fbcb00744e0 0x7fbcb00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.788+0000 7fbcc3fff640 1 --2- 172.21.15.106:0/3608068873 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fbcb00744e0 0x7fbcb00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:41.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.788+0000 7fbcc0ff9640 1 -- 172.21.15.106:0/3608068873 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fbcbc095f90 con 0x7fbcc415ce00 2024-04-25T10:30:41.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.789+0000 7fbcc3fff640 1 --2- 172.21.15.106:0/3608068873 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fbcb00744e0 0x7fbcb00769f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbcb4034000 tx=0x7fbcb402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:41.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.793+0000 7fbcc0ff9640 1 -- 172.21.15.106:0/3608068873 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbcbc022030 con 0x7fbcc415ce00 2024-04-25T10:30:41.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.964+0000 7fbcca6e0640 1 -- 172.21.15.106:0/3608068873 --> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbcc4074420 con 0x7fbcb00744e0 2024-04-25T10:30:41.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.970+0000 7fbcc0ff9640 1 -- 172.21.15.106:0/3608068873 <== mgr.4107 v2:172.21.15.106:6832/1744210167 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251936 (secure 0 0 0) 0x7fbcc4074420 con 0x7fbcb00744e0 2024-04-25T10:30:41.975 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:30:41.977 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-25T10:30:41.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.973+0000 7fbcca6e0640 1 -- 172.21.15.106:0/3608068873 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fbcb00744e0 msgr2=0x7fbcb00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:41.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.973+0000 7fbcca6e0640 1 --2- 172.21.15.106:0/3608068873 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fbcb00744e0 0x7fbcb00769f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbcb4034000 tx=0x7fbcb402e000 comp rx=0 tx=0).stop 2024-04-25T10:30:41.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.973+0000 7fbcca6e0640 1 -- 172.21.15.106:0/3608068873 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbcc415ce00 msgr2=0x7fbcc414faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:41.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.973+0000 7fbcca6e0640 1 --2- 172.21.15.106:0/3608068873 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbcc415ce00 0x7fbcc414faf0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc00efc0 tx=0x7fbcbc00c490 comp rx=0 tx=0).stop 2024-04-25T10:30:41.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.973+0000 7fbcca6e0640 1 -- 172.21.15.106:0/3608068873 shutdown_connections 2024-04-25T10:30:41.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.973+0000 7fbcca6e0640 1 --2- 172.21.15.106:0/3608068873 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fbcb00744e0 0x7fbcb00769f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.973+0000 7fbcca6e0640 1 --2- 172.21.15.106:0/3608068873 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbcc415ce00 0x7fbcc414faf0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.973+0000 7fbcca6e0640 1 --2- 172.21.15.106:0/3608068873 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbcc40ca170 0x7fbcc414f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.973+0000 7fbcca6e0640 1 --2- 172.21.15.106:0/3608068873 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbcc406c5c0 0x7fbcc415f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.973+0000 7fbcca6e0640 1 -- 172.21.15.106:0/3608068873 >> 172.21.15.106:0/3608068873 conn(0x7fbcc40cef20 msgr2=0x7fbcc4071a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:41.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.973+0000 7fbcca6e0640 1 -- 172.21.15.106:0/3608068873 shutdown_connections 2024-04-25T10:30:41.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:41.973+0000 7fbcca6e0640 1 -- 172.21.15.106:0/3608068873 wait complete. 2024-04-25T10:30:41.987 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-04-25T10:30:40.097718+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218952,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755768,"statfs":{"total":773094113280,"available":772869906432,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002087"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795275+0000","last_change":"2024-04-25T10:30:32.440983+0000","last_active":"2024-04-25T10:30:34.795275+0000","last_peered":"2024-04-25T10:30:34.795275+0000","last_clean":"2024-04-25T10:30:34.795275+0000","last_became_active":"2024-04-25T10:30:32.440552+0000","last_became_peered":"2024-04-25T10:30:32.440552+0000","last_unstale":"2024-04-25T10:30:34.795275+0000","last_undegraded":"2024-04-25T10:30:34.795275+0000","last_fullsized":"2024-04-25T10:30:34.795275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:47.963061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.412686+0000","last_change":"2024-04-25T10:30:32.425695+0000","last_active":"2024-04-25T10:30:33.412686+0000","last_peered":"2024-04-25T10:30:33.412686+0000","last_clean":"2024-04-25T10:30:33.412686+0000","last_became_active":"2024-04-25T10:30:32.425368+0000","last_became_peered":"2024-04-25T10:30:32.425368+0000","last_unstale":"2024-04-25T10:30:33.412686+0000","last_undegraded":"2024-04-25T10:30:33.412686+0000","last_fullsized":"2024-04-25T10:30:33.412686+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:39.120522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.196620+0000","last_change":"2024-04-25T10:30:32.430144+0000","last_active":"2024-04-25T10:30:34.196620+0000","last_peered":"2024-04-25T10:30:34.196620+0000","last_clean":"2024-04-25T10:30:34.196620+0000","last_became_active":"2024-04-25T10:30:32.429754+0000","last_became_peered":"2024-04-25T10:30:32.429754+0000","last_unstale":"2024-04-25T10:30:34.196620+0000","last_undegraded":"2024-04-25T10:30:34.196620+0000","last_fullsized":"2024-04-25T10:30:34.196620+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:11.448907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794948+0000","last_change":"2024-04-25T10:30:32.442478+0000","last_active":"2024-04-25T10:30:34.794948+0000","last_peered":"2024-04-25T10:30:34.794948+0000","last_clean":"2024-04-25T10:30:34.794948+0000","last_became_active":"2024-04-25T10:30:32.442182+0000","last_became_peered":"2024-04-25T10:30:32.442182+0000","last_unstale":"2024-04-25T10:30:34.794948+0000","last_undegraded":"2024-04-25T10:30:34.794948+0000","last_fullsized":"2024-04-25T10:30:34.794948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:22.185978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794683+0000","last_change":"2024-04-25T10:30:32.438231+0000","last_active":"2024-04-25T10:30:34.794683+0000","last_peered":"2024-04-25T10:30:34.794683+0000","last_clean":"2024-04-25T10:30:34.794683+0000","last_became_active":"2024-04-25T10:30:32.437656+0000","last_became_peered":"2024-04-25T10:30:32.437656+0000","last_unstale":"2024-04-25T10:30:34.794683+0000","last_undegraded":"2024-04-25T10:30:34.794683+0000","last_fullsized":"2024-04-25T10:30:34.794683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:56.915999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795645+0000","last_change":"2024-04-25T10:30:32.439221+0000","last_active":"2024-04-25T10:30:34.795645+0000","last_peered":"2024-04-25T10:30:34.795645+0000","last_clean":"2024-04-25T10:30:34.795645+0000","last_became_active":"2024-04-25T10:30:32.438923+0000","last_became_peered":"2024-04-25T10:30:32.438923+0000","last_unstale":"2024-04-25T10:30:34.795645+0000","last_undegraded":"2024-04-25T10:30:34.795645+0000","last_fullsized":"2024-04-25T10:30:34.795645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:40.469409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794157+0000","last_change":"2024-04-25T10:30:32.429453+0000","last_active":"2024-04-25T10:30:34.794157+0000","last_peered":"2024-04-25T10:30:34.794157+0000","last_clean":"2024-04-25T10:30:34.794157+0000","last_became_active":"2024-04-25T10:30:32.429131+0000","last_became_peered":"2024-04-25T10:30:32.429131+0000","last_unstale":"2024-04-25T10:30:34.794157+0000","last_undegraded":"2024-04-25T10:30:34.794157+0000","last_fullsized":"2024-04-25T10:30:34.794157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:26.785922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415260+0000","last_change":"2024-04-25T10:30:32.438860+0000","last_active":"2024-04-25T10:30:33.415260+0000","last_peered":"2024-04-25T10:30:33.415260+0000","last_clean":"2024-04-25T10:30:33.415260+0000","last_became_active":"2024-04-25T10:30:32.438491+0000","last_became_peered":"2024-04-25T10:30:32.438491+0000","last_unstale":"2024-04-25T10:30:33.415260+0000","last_undegraded":"2024-04-25T10:30:33.415260+0000","last_fullsized":"2024-04-25T10:30:33.415260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:08.610707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795467+0000","last_change":"2024-04-25T10:30:32.440331+0000","last_active":"2024-04-25T10:30:34.795467+0000","last_peered":"2024-04-25T10:30:34.795467+0000","last_clean":"2024-04-25T10:30:34.795467+0000","last_became_active":"2024-04-25T10:30:32.440043+0000","last_became_peered":"2024-04-25T10:30:32.440043+0000","last_unstale":"2024-04-25T10:30:34.795467+0000","last_undegraded":"2024-04-25T10:30:34.795467+0000","last_fullsized":"2024-04-25T10:30:34.795467+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:17.684685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794636+0000","last_change":"2024-04-25T10:30:32.435388+0000","last_active":"2024-04-25T10:30:34.794636+0000","last_peered":"2024-04-25T10:30:34.794636+0000","last_clean":"2024-04-25T10:30:34.794636+0000","last_became_active":"2024-04-25T10:30:32.434983+0000","last_became_peered":"2024-04-25T10:30:32.434983+0000","last_unstale":"2024-04-25T10:30:34.794636+0000","last_undegraded":"2024-04-25T10:30:34.794636+0000","last_fullsized":"2024-04-25T10:30:34.794636+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:39.256604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.793928+0000","last_change":"2024-04-25T10:30:32.435884+0000","last_active":"2024-04-25T10:30:34.793928+0000","last_peered":"2024-04-25T10:30:34.793928+0000","last_clean":"2024-04-25T10:30:34.793928+0000","last_became_active":"2024-04-25T10:30:32.434645+0000","last_became_peered":"2024-04-25T10:30:32.434645+0000","last_unstale":"2024-04-25T10:30:34.793928+0000","last_undegraded":"2024-04-25T10:30:34.793928+0000","last_fullsized":"2024-04-25T10:30:34.793928+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:07.778437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795720+0000","last_change":"2024-04-25T10:30:32.436403+0000","last_active":"2024-04-25T10:30:34.795720+0000","last_peered":"2024-04-25T10:30:34.795720+0000","last_clean":"2024-04-25T10:30:34.795720+0000","last_became_active":"2024-04-25T10:30:32.435908+0000","last_became_peered":"2024-04-25T10:30:32.435908+0000","last_unstale":"2024-04-25T10:30:34.795720+0000","last_undegraded":"2024-04-25T10:30:34.795720+0000","last_fullsized":"2024-04-25T10:30:34.795720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:03.339316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415872+0000","last_change":"2024-04-25T10:30:32.432110+0000","last_active":"2024-04-25T10:30:33.415872+0000","last_peered":"2024-04-25T10:30:33.415872+0000","last_clean":"2024-04-25T10:30:33.415872+0000","last_became_active":"2024-04-25T10:30:32.431792+0000","last_became_peered":"2024-04-25T10:30:32.431792+0000","last_unstale":"2024-04-25T10:30:33.415872+0000","last_undegraded":"2024-04-25T10:30:33.415872+0000","last_fullsized":"2024-04-25T10:30:33.415872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:34.185989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.196476+0000","last_change":"2024-04-25T10:30:32.433676+0000","last_active":"2024-04-25T10:30:34.196476+0000","last_peered":"2024-04-25T10:30:34.196476+0000","last_clean":"2024-04-25T10:30:34.196476+0000","last_became_active":"2024-04-25T10:30:32.433329+0000","last_became_peered":"2024-04-25T10:30:32.433329+0000","last_unstale":"2024-04-25T10:30:34.196476+0000","last_undegraded":"2024-04-25T10:30:34.196476+0000","last_fullsized":"2024-04-25T10:30:34.196476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:59.942368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.412754+0000","last_change":"2024-04-25T10:30:32.423582+0000","last_active":"2024-04-25T10:30:33.412754+0000","last_peered":"2024-04-25T10:30:33.412754+0000","last_clean":"2024-04-25T10:30:33.412754+0000","last_became_active":"2024-04-25T10:30:32.422991+0000","last_became_peered":"2024-04-25T10:30:32.422991+0000","last_unstale":"2024-04-25T10:30:33.412754+0000","last_undegraded":"2024-04-25T10:30:33.412754+0000","last_fullsized":"2024-04-25T10:30:33.412754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:11:47.132928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794545+0000","last_change":"2024-04-25T10:30:32.440235+0000","last_active":"2024-04-25T10:30:34.794545+0000","last_peered":"2024-04-25T10:30:34.794545+0000","last_clean":"2024-04-25T10:30:34.794545+0000","last_became_active":"2024-04-25T10:30:32.439191+0000","last_became_peered":"2024-04-25T10:30:32.439191+0000","last_unstale":"2024-04-25T10:30:34.794545+0000","last_undegraded":"2024-04-25T10:30:34.794545+0000","last_fullsized":"2024-04-25T10:30:34.794545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:51.423086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795170+0000","last_change":"2024-04-25T10:30:32.442418+0000","last_active":"2024-04-25T10:30:34.795170+0000","last_peered":"2024-04-25T10:30:34.795170+0000","last_clean":"2024-04-25T10:30:34.795170+0000","last_became_active":"2024-04-25T10:30:32.442125+0000","last_became_peered":"2024-04-25T10:30:32.442125+0000","last_unstale":"2024-04-25T10:30:34.795170+0000","last_undegraded":"2024-04-25T10:30:34.795170+0000","last_fullsized":"2024-04-25T10:30:34.795170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:14.213809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415344+0000","last_change":"2024-04-25T10:30:32.434758+0000","last_active":"2024-04-25T10:30:33.415344+0000","last_peered":"2024-04-25T10:30:33.415344+0000","last_clean":"2024-04-25T10:30:33.415344+0000","last_became_active":"2024-04-25T10:30:32.434492+0000","last_became_peered":"2024-04-25T10:30:32.434492+0000","last_unstale":"2024-04-25T10:30:33.415344+0000","last_undegraded":"2024-04-25T10:30:33.415344+0000","last_fullsized":"2024-04-25T10:30:33.415344+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:41.454407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.796066+0000","last_change":"2024-04-25T10:30:32.440399+0000","last_active":"2024-04-25T10:30:34.796066+0000","last_peered":"2024-04-25T10:30:34.796066+0000","last_clean":"2024-04-25T10:30:34.796066+0000","last_became_active":"2024-04-25T10:30:32.438227+0000","last_became_peered":"2024-04-25T10:30:32.438227+0000","last_unstale":"2024-04-25T10:30:34.796066+0000","last_undegraded":"2024-04-25T10:30:34.796066+0000","last_fullsized":"2024-04-25T10:30:34.796066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:49.571671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794149+0000","last_change":"2024-04-25T10:30:32.431609+0000","last_active":"2024-04-25T10:30:34.794149+0000","last_peered":"2024-04-25T10:30:34.794149+0000","last_clean":"2024-04-25T10:30:34.794149+0000","last_became_active":"2024-04-25T10:30:32.431314+0000","last_became_peered":"2024-04-25T10:30:32.431314+0000","last_unstale":"2024-04-25T10:30:34.794149+0000","last_undegraded":"2024-04-25T10:30:34.794149+0000","last_fullsized":"2024-04-25T10:30:34.794149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:33.421989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.196668+0000","last_change":"2024-04-25T10:30:32.432612+0000","last_active":"2024-04-25T10:30:34.196668+0000","last_peered":"2024-04-25T10:30:34.196668+0000","last_clean":"2024-04-25T10:30:34.196668+0000","last_became_active":"2024-04-25T10:30:32.432271+0000","last_became_peered":"2024-04-25T10:30:32.432271+0000","last_unstale":"2024-04-25T10:30:34.196668+0000","last_undegraded":"2024-04-25T10:30:34.196668+0000","last_fullsized":"2024-04-25T10:30:34.196668+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:14.281493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795827+0000","last_change":"2024-04-25T10:30:32.444618+0000","last_active":"2024-04-25T10:30:34.795827+0000","last_peered":"2024-04-25T10:30:34.795827+0000","last_clean":"2024-04-25T10:30:34.795827+0000","last_became_active":"2024-04-25T10:30:32.444273+0000","last_became_peered":"2024-04-25T10:30:32.444273+0000","last_unstale":"2024-04-25T10:30:34.795827+0000","last_undegraded":"2024-04-25T10:30:34.795827+0000","last_fullsized":"2024-04-25T10:30:34.795827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:09.295251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415348+0000","last_change":"2024-04-25T10:30:32.426578+0000","last_active":"2024-04-25T10:30:33.415348+0000","last_peered":"2024-04-25T10:30:33.415348+0000","last_clean":"2024-04-25T10:30:33.415348+0000","last_became_active":"2024-04-25T10:30:32.426223+0000","last_became_peered":"2024-04-25T10:30:32.426223+0000","last_unstale":"2024-04-25T10:30:33.415348+0000","last_undegraded":"2024-04-25T10:30:33.415348+0000","last_fullsized":"2024-04-25T10:30:33.415348+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:53.700585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794721+0000","last_change":"2024-04-25T10:30:32.439767+0000","last_active":"2024-04-25T10:30:34.794721+0000","last_peered":"2024-04-25T10:30:34.794721+0000","last_clean":"2024-04-25T10:30:34.794721+0000","last_became_active":"2024-04-25T10:30:32.438801+0000","last_became_peered":"2024-04-25T10:30:32.438801+0000","last_unstale":"2024-04-25T10:30:34.794721+0000","last_undegraded":"2024-04-25T10:30:34.794721+0000","last_fullsized":"2024-04-25T10:30:34.794721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:58.231512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415544+0000","last_change":"2024-04-25T10:30:32.427424+0000","last_active":"2024-04-25T10:30:33.415544+0000","last_peered":"2024-04-25T10:30:33.415544+0000","last_clean":"2024-04-25T10:30:33.415544+0000","last_became_active":"2024-04-25T10:30:32.427081+0000","last_became_peered":"2024-04-25T10:30:32.427081+0000","last_unstale":"2024-04-25T10:30:33.415544+0000","last_undegraded":"2024-04-25T10:30:33.415544+0000","last_fullsized":"2024-04-25T10:30:33.415544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:47.372451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794727+0000","last_change":"2024-04-25T10:30:32.432472+0000","last_active":"2024-04-25T10:30:34.794727+0000","last_peered":"2024-04-25T10:30:34.794727+0000","last_clean":"2024-04-25T10:30:34.794727+0000","last_became_active":"2024-04-25T10:30:32.432121+0000","last_became_peered":"2024-04-25T10:30:32.432121+0000","last_unstale":"2024-04-25T10:30:34.794727+0000","last_undegraded":"2024-04-25T10:30:34.794727+0000","last_fullsized":"2024-04-25T10:30:34.794727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:34.191339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.412571+0000","last_change":"2024-04-25T10:30:32.425108+0000","last_active":"2024-04-25T10:30:33.412571+0000","last_peered":"2024-04-25T10:30:33.412571+0000","last_clean":"2024-04-25T10:30:33.412571+0000","last_became_active":"2024-04-25T10:30:32.424794+0000","last_became_peered":"2024-04-25T10:30:32.424794+0000","last_unstale":"2024-04-25T10:30:33.412571+0000","last_undegraded":"2024-04-25T10:30:33.412571+0000","last_fullsized":"2024-04-25T10:30:33.412571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:01.282758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415734+0000","last_change":"2024-04-25T10:30:32.426672+0000","last_active":"2024-04-25T10:30:33.415734+0000","last_peered":"2024-04-25T10:30:33.415734+0000","last_clean":"2024-04-25T10:30:33.415734+0000","last_became_active":"2024-04-25T10:30:32.426348+0000","last_became_peered":"2024-04-25T10:30:32.426348+0000","last_unstale":"2024-04-25T10:30:33.415734+0000","last_undegraded":"2024-04-25T10:30:33.415734+0000","last_fullsized":"2024-04-25T10:30:33.415734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:57.645772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.196609+0000","last_change":"2024-04-25T10:30:32.427989+0000","last_active":"2024-04-25T10:30:34.196609+0000","last_peered":"2024-04-25T10:30:34.196609+0000","last_clean":"2024-04-25T10:30:34.196609+0000","last_became_active":"2024-04-25T10:30:32.427676+0000","last_became_peered":"2024-04-25T10:30:32.427676+0000","last_unstale":"2024-04-25T10:30:34.196609+0000","last_undegraded":"2024-04-25T10:30:34.196609+0000","last_fullsized":"2024-04-25T10:30:34.196609+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:15:02.332186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795935+0000","last_change":"2024-04-25T10:30:32.441112+0000","last_active":"2024-04-25T10:30:34.795935+0000","last_peered":"2024-04-25T10:30:34.795935+0000","last_clean":"2024-04-25T10:30:34.795935+0000","last_became_active":"2024-04-25T10:30:32.440815+0000","last_became_peered":"2024-04-25T10:30:32.440815+0000","last_unstale":"2024-04-25T10:30:34.795935+0000","last_undegraded":"2024-04-25T10:30:34.795935+0000","last_fullsized":"2024-04-25T10:30:34.795935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:02.956759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415178+0000","last_change":"2024-04-25T10:30:32.430902+0000","last_active":"2024-04-25T10:30:33.415178+0000","last_peered":"2024-04-25T10:30:33.415178+0000","last_clean":"2024-04-25T10:30:33.415178+0000","last_became_active":"2024-04-25T10:30:32.430615+0000","last_became_peered":"2024-04-25T10:30:32.430615+0000","last_unstale":"2024-04-25T10:30:33.415178+0000","last_undegraded":"2024-04-25T10:30:33.415178+0000","last_fullsized":"2024-04-25T10:30:33.415178+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:12.747595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795247+0000","last_change":"2024-04-25T10:30:32.435379+0000","last_active":"2024-04-25T10:30:34.795247+0000","last_peered":"2024-04-25T10:30:34.795247+0000","last_clean":"2024-04-25T10:30:34.795247+0000","last_became_active":"2024-04-25T10:30:32.435055+0000","last_became_peered":"2024-04-25T10:30:32.435055+0000","last_unstale":"2024-04-25T10:30:34.795247+0000","last_undegraded":"2024-04-25T10:30:34.795247+0000","last_fullsized":"2024-04-25T10:30:34.795247+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:13.623762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794490+0000","last_change":"2024-04-25T10:30:32.422102+0000","last_active":"2024-04-25T10:30:34.794490+0000","last_peered":"2024-04-25T10:30:34.794490+0000","last_clean":"2024-04-25T10:30:34.794490+0000","last_became_active":"2024-04-25T10:30:32.421766+0000","last_became_peered":"2024-04-25T10:30:32.421766+0000","last_unstale":"2024-04-25T10:30:34.794490+0000","last_undegraded":"2024-04-25T10:30:34.794490+0000","last_fullsized":"2024-04-25T10:30:34.794490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:30.300572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.120320+0000","last_change":"2024-04-25T10:30:31.418065+0000","last_active":"2024-04-25T10:30:35.120320+0000","last_peered":"2024-04-25T10:30:35.120320+0000","last_clean":"2024-04-25T10:30:35.120320+0000","last_became_active":"2024-04-25T10:30:31.417451+0000","last_became_peered":"2024-04-25T10:30:31.417451+0000","last_unstale":"2024-04-25T10:30:35.120320+0000","last_undegraded":"2024-04-25T10:30:35.120320+0000","last_fullsized":"2024-04-25T10:30:35.120320+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:17.262861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.196777+0000","last_change":"2024-04-25T10:30:32.426913+0000","last_active":"2024-04-25T10:30:34.196777+0000","last_peered":"2024-04-25T10:30:34.196777+0000","last_clean":"2024-04-25T10:30:34.196777+0000","last_became_active":"2024-04-25T10:30:32.426616+0000","last_became_peered":"2024-04-25T10:30:32.426616+0000","last_unstale":"2024-04-25T10:30:34.196777+0000","last_undegraded":"2024-04-25T10:30:34.196777+0000","last_fullsized":"2024-04-25T10:30:34.196777+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:50.614932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.117929+0000","last_change":"2024-04-25T10:30:31.416432+0000","last_active":"2024-04-25T10:30:35.117929+0000","last_peered":"2024-04-25T10:30:35.117929+0000","last_clean":"2024-04-25T10:30:35.117929+0000","last_became_active":"2024-04-25T10:30:31.415112+0000","last_became_peered":"2024-04-25T10:30:31.415112+0000","last_unstale":"2024-04-25T10:30:35.117929+0000","last_undegraded":"2024-04-25T10:30:35.117929+0000","last_fullsized":"2024-04-25T10:30:35.117929+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:22.674365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415646+0000","last_change":"2024-04-25T10:30:31.418059+0000","last_active":"2024-04-25T10:30:33.415646+0000","last_peered":"2024-04-25T10:30:33.415646+0000","last_clean":"2024-04-25T10:30:33.415646+0000","last_became_active":"2024-04-25T10:30:31.417716+0000","last_became_peered":"2024-04-25T10:30:31.417716+0000","last_unstale":"2024-04-25T10:30:33.415646+0000","last_undegraded":"2024-04-25T10:30:33.415646+0000","last_fullsized":"2024-04-25T10:30:33.415646+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:57.752218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415428+0000","last_change":"2024-04-25T10:30:32.428078+0000","last_active":"2024-04-25T10:30:33.415428+0000","last_peered":"2024-04-25T10:30:33.415428+0000","last_clean":"2024-04-25T10:30:33.415428+0000","last_became_active":"2024-04-25T10:30:32.427700+0000","last_became_peered":"2024-04-25T10:30:32.427700+0000","last_unstale":"2024-04-25T10:30:33.415428+0000","last_undegraded":"2024-04-25T10:30:33.415428+0000","last_fullsized":"2024-04-25T10:30:33.415428+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:26.931791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795722+0000","last_change":"2024-04-25T10:30:31.415030+0000","last_active":"2024-04-25T10:30:34.795722+0000","last_peered":"2024-04-25T10:30:34.795722+0000","last_clean":"2024-04-25T10:30:34.795722+0000","last_became_active":"2024-04-25T10:30:31.414542+0000","last_became_peered":"2024-04-25T10:30:31.414542+0000","last_unstale":"2024-04-25T10:30:34.795722+0000","last_undegraded":"2024-04-25T10:30:34.795722+0000","last_fullsized":"2024-04-25T10:30:34.795722+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:20.082420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.412471+0000","last_change":"2024-04-25T10:30:32.425207+0000","last_active":"2024-04-25T10:30:33.412471+0000","last_peered":"2024-04-25T10:30:33.412471+0000","last_clean":"2024-04-25T10:30:33.412471+0000","last_became_active":"2024-04-25T10:30:32.424849+0000","last_became_peered":"2024-04-25T10:30:32.424849+0000","last_unstale":"2024-04-25T10:30:33.412471+0000","last_undegraded":"2024-04-25T10:30:33.412471+0000","last_fullsized":"2024-04-25T10:30:33.412471+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:11.533686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794249+0000","last_change":"2024-04-25T10:30:31.407700+0000","last_active":"2024-04-25T10:30:34.794249+0000","last_peered":"2024-04-25T10:30:34.794249+0000","last_clean":"2024-04-25T10:30:34.794249+0000","last_became_active":"2024-04-25T10:30:31.407376+0000","last_became_peered":"2024-04-25T10:30:31.407376+0000","last_unstale":"2024-04-25T10:30:34.794249+0000","last_undegraded":"2024-04-25T10:30:34.794249+0000","last_fullsized":"2024-04-25T10:30:34.794249+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:50.112742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794157+0000","last_change":"2024-04-25T10:30:32.444599+0000","last_active":"2024-04-25T10:30:34.794157+0000","last_peered":"2024-04-25T10:30:34.794157+0000","last_clean":"2024-04-25T10:30:34.794157+0000","last_became_active":"2024-04-25T10:30:32.444250+0000","last_became_peered":"2024-04-25T10:30:32.444250+0000","last_unstale":"2024-04-25T10:30:34.794157+0000","last_undegraded":"2024-04-25T10:30:34.794157+0000","last_fullsized":"2024-04-25T10:30:34.794157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:55.711525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794280+0000","last_change":"2024-04-25T10:30:31.405051+0000","last_active":"2024-04-25T10:30:34.794280+0000","last_peered":"2024-04-25T10:30:34.794280+0000","last_clean":"2024-04-25T10:30:34.794280+0000","last_became_active":"2024-04-25T10:30:31.404669+0000","last_became_peered":"2024-04-25T10:30:31.404669+0000","last_unstale":"2024-04-25T10:30:34.794280+0000","last_undegraded":"2024-04-25T10:30:34.794280+0000","last_fullsized":"2024-04-25T10:30:34.794280+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:45.260847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.196763+0000","last_change":"2024-04-25T10:30:32.435804+0000","last_active":"2024-04-25T10:30:34.196763+0000","last_peered":"2024-04-25T10:30:34.196763+0000","last_clean":"2024-04-25T10:30:34.196763+0000","last_became_active":"2024-04-25T10:30:32.435471+0000","last_became_peered":"2024-04-25T10:30:32.435471+0000","last_unstale":"2024-04-25T10:30:34.196763+0000","last_undegraded":"2024-04-25T10:30:34.196763+0000","last_fullsized":"2024-04-25T10:30:34.196763+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:48.213846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:37.436569+0000","last_change":"2024-04-25T10:30:31.403560+0000","last_active":"2024-04-25T10:30:37.436569+0000","last_peered":"2024-04-25T10:30:37.436569+0000","last_clean":"2024-04-25T10:30:37.436569+0000","last_became_active":"2024-04-25T10:30:31.403167+0000","last_became_peered":"2024-04-25T10:30:31.403167+0000","last_unstale":"2024-04-25T10:30:37.436569+0000","last_undegraded":"2024-04-25T10:30:37.436569+0000","last_fullsized":"2024-04-25T10:30:37.436569+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:54.592832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795908+0000","last_change":"2024-04-25T10:30:32.424101+0000","last_active":"2024-04-25T10:30:34.795908+0000","last_peered":"2024-04-25T10:30:34.795908+0000","last_clean":"2024-04-25T10:30:34.795908+0000","last_became_active":"2024-04-25T10:30:32.423780+0000","last_became_peered":"2024-04-25T10:30:32.423780+0000","last_unstale":"2024-04-25T10:30:34.795908+0000","last_undegraded":"2024-04-25T10:30:34.795908+0000","last_fullsized":"2024-04-25T10:30:34.795908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:29.175348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:37.435669+0000","last_change":"2024-04-25T10:30:31.418201+0000","last_active":"2024-04-25T10:30:37.435669+0000","last_peered":"2024-04-25T10:30:37.435669+0000","last_clean":"2024-04-25T10:30:37.435669+0000","last_became_active":"2024-04-25T10:30:31.417824+0000","last_became_peered":"2024-04-25T10:30:31.417824+0000","last_unstale":"2024-04-25T10:30:37.435669+0000","last_undegraded":"2024-04-25T10:30:37.435669+0000","last_fullsized":"2024-04-25T10:30:37.435669+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:38.584225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795830+0000","last_change":"2024-04-25T10:30:32.431139+0000","last_active":"2024-04-25T10:30:34.795830+0000","last_peered":"2024-04-25T10:30:34.795830+0000","last_clean":"2024-04-25T10:30:34.795830+0000","last_became_active":"2024-04-25T10:30:32.429622+0000","last_became_peered":"2024-04-25T10:30:32.429622+0000","last_unstale":"2024-04-25T10:30:34.795830+0000","last_undegraded":"2024-04-25T10:30:34.795830+0000","last_fullsized":"2024-04-25T10:30:34.795830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:39.993647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:37.434076+0000","last_change":"2024-04-25T10:30:31.414775+0000","last_active":"2024-04-25T10:30:37.434076+0000","last_peered":"2024-04-25T10:30:37.434076+0000","last_clean":"2024-04-25T10:30:37.434076+0000","last_became_active":"2024-04-25T10:30:31.414422+0000","last_became_peered":"2024-04-25T10:30:31.414422+0000","last_unstale":"2024-04-25T10:30:37.434076+0000","last_undegraded":"2024-04-25T10:30:37.434076+0000","last_fullsized":"2024-04-25T10:30:37.434076+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:37.803505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.196648+0000","last_change":"2024-04-25T10:30:32.432884+0000","last_active":"2024-04-25T10:30:34.196648+0000","last_peered":"2024-04-25T10:30:34.196648+0000","last_clean":"2024-04-25T10:30:34.196648+0000","last_became_active":"2024-04-25T10:30:32.431743+0000","last_became_peered":"2024-04-25T10:30:32.431743+0000","last_unstale":"2024-04-25T10:30:34.196648+0000","last_undegraded":"2024-04-25T10:30:34.196648+0000","last_fullsized":"2024-04-25T10:30:34.196648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:15:06.737631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795999+0000","last_change":"2024-04-25T10:30:31.419497+0000","last_active":"2024-04-25T10:30:34.795999+0000","last_peered":"2024-04-25T10:30:34.795999+0000","last_clean":"2024-04-25T10:30:34.795999+0000","last_became_active":"2024-04-25T10:30:31.419135+0000","last_became_peered":"2024-04-25T10:30:31.419135+0000","last_unstale":"2024-04-25T10:30:34.795999+0000","last_undegraded":"2024-04-25T10:30:34.795999+0000","last_fullsized":"2024-04-25T10:30:34.795999+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:59.403927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794867+0000","last_change":"2024-04-25T10:30:32.434197+0000","last_active":"2024-04-25T10:30:34.794867+0000","last_peered":"2024-04-25T10:30:34.794867+0000","last_clean":"2024-04-25T10:30:34.794867+0000","last_became_active":"2024-04-25T10:30:32.433623+0000","last_became_peered":"2024-04-25T10:30:32.433623+0000","last_unstale":"2024-04-25T10:30:34.794867+0000","last_undegraded":"2024-04-25T10:30:34.794867+0000","last_fullsized":"2024-04-25T10:30:34.794867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:08.027472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.793813+0000","last_change":"2024-04-25T10:30:31.416901+0000","last_active":"2024-04-25T10:30:34.793813+0000","last_peered":"2024-04-25T10:30:34.793813+0000","last_clean":"2024-04-25T10:30:34.793813+0000","last_became_active":"2024-04-25T10:30:31.415706+0000","last_became_peered":"2024-04-25T10:30:31.415706+0000","last_unstale":"2024-04-25T10:30:34.793813+0000","last_undegraded":"2024-04-25T10:30:34.793813+0000","last_fullsized":"2024-04-25T10:30:34.793813+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:23.243039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795344+0000","last_change":"2024-04-25T10:30:32.435089+0000","last_active":"2024-04-25T10:30:34.795344+0000","last_peered":"2024-04-25T10:30:34.795344+0000","last_clean":"2024-04-25T10:30:34.795344+0000","last_became_active":"2024-04-25T10:30:32.434737+0000","last_became_peered":"2024-04-25T10:30:32.434737+0000","last_unstale":"2024-04-25T10:30:34.795344+0000","last_undegraded":"2024-04-25T10:30:34.795344+0000","last_fullsized":"2024-04-25T10:30:34.795344+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:19.738850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:37.436320+0000","last_change":"2024-04-25T10:30:31.405163+0000","last_active":"2024-04-25T10:30:37.436320+0000","last_peered":"2024-04-25T10:30:37.436320+0000","last_clean":"2024-04-25T10:30:37.436320+0000","last_became_active":"2024-04-25T10:30:31.404775+0000","last_became_peered":"2024-04-25T10:30:31.404775+0000","last_unstale":"2024-04-25T10:30:37.436320+0000","last_undegraded":"2024-04-25T10:30:37.436320+0000","last_fullsized":"2024-04-25T10:30:37.436320+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:10:21.595653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794446+0000","last_change":"2024-04-25T10:30:32.432061+0000","last_active":"2024-04-25T10:30:34.794446+0000","last_peered":"2024-04-25T10:30:34.794446+0000","last_clean":"2024-04-25T10:30:34.794446+0000","last_became_active":"2024-04-25T10:30:32.431651+0000","last_became_peered":"2024-04-25T10:30:32.431651+0000","last_unstale":"2024-04-25T10:30:34.794446+0000","last_undegraded":"2024-04-25T10:30:34.794446+0000","last_fullsized":"2024-04-25T10:30:34.794446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:39.062490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.122181+0000","last_change":"2024-04-25T10:30:31.420258+0000","last_active":"2024-04-25T10:30:35.122181+0000","last_peered":"2024-04-25T10:30:35.122181+0000","last_clean":"2024-04-25T10:30:35.122181+0000","last_became_active":"2024-04-25T10:30:31.419902+0000","last_became_peered":"2024-04-25T10:30:31.419902+0000","last_unstale":"2024-04-25T10:30:35.122181+0000","last_undegraded":"2024-04-25T10:30:35.122181+0000","last_fullsized":"2024-04-25T10:30:35.122181+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:43.919867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795443+0000","last_change":"2024-04-25T10:30:32.433568+0000","last_active":"2024-04-25T10:30:34.795443+0000","last_peered":"2024-04-25T10:30:34.795443+0000","last_clean":"2024-04-25T10:30:34.795443+0000","last_became_active":"2024-04-25T10:30:32.432572+0000","last_became_peered":"2024-04-25T10:30:32.432572+0000","last_unstale":"2024-04-25T10:30:34.795443+0000","last_undegraded":"2024-04-25T10:30:34.795443+0000","last_fullsized":"2024-04-25T10:30:34.795443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:31.771919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:37.435148+0000","last_change":"2024-04-25T10:30:31.419308+0000","last_active":"2024-04-25T10:30:37.435148+0000","last_peered":"2024-04-25T10:30:37.435148+0000","last_clean":"2024-04-25T10:30:37.435148+0000","last_became_active":"2024-04-25T10:30:31.418978+0000","last_became_peered":"2024-04-25T10:30:31.418978+0000","last_unstale":"2024-04-25T10:30:37.435148+0000","last_undegraded":"2024-04-25T10:30:37.435148+0000","last_fullsized":"2024-04-25T10:30:37.435148+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:33.784804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795670+0000","last_change":"2024-04-25T10:30:32.436109+0000","last_active":"2024-04-25T10:30:34.795670+0000","last_peered":"2024-04-25T10:30:34.795670+0000","last_clean":"2024-04-25T10:30:34.795670+0000","last_became_active":"2024-04-25T10:30:32.434701+0000","last_became_peered":"2024-04-25T10:30:32.434701+0000","last_unstale":"2024-04-25T10:30:34.795670+0000","last_undegraded":"2024-04-25T10:30:34.795670+0000","last_fullsized":"2024-04-25T10:30:34.795670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:19.471867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.119322+0000","last_change":"2024-04-25T10:30:31.416044+0000","last_active":"2024-04-25T10:30:35.119322+0000","last_peered":"2024-04-25T10:30:35.119322+0000","last_clean":"2024-04-25T10:30:35.119322+0000","last_became_active":"2024-04-25T10:30:31.415707+0000","last_became_peered":"2024-04-25T10:30:31.415707+0000","last_unstale":"2024-04-25T10:30:35.119322+0000","last_undegraded":"2024-04-25T10:30:35.119322+0000","last_fullsized":"2024-04-25T10:30:35.119322+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:31.424135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795321+0000","last_change":"2024-04-25T10:30:32.442971+0000","last_active":"2024-04-25T10:30:34.795321+0000","last_peered":"2024-04-25T10:30:34.795321+0000","last_clean":"2024-04-25T10:30:34.795321+0000","last_became_active":"2024-04-25T10:30:32.442656+0000","last_became_peered":"2024-04-25T10:30:32.442656+0000","last_unstale":"2024-04-25T10:30:34.795321+0000","last_undegraded":"2024-04-25T10:30:34.795321+0000","last_fullsized":"2024-04-25T10:30:34.795321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:29.951142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415622+0000","last_change":"2024-04-25T10:30:31.404793+0000","last_active":"2024-04-25T10:30:33.415622+0000","last_peered":"2024-04-25T10:30:33.415622+0000","last_clean":"2024-04-25T10:30:33.415622+0000","last_became_active":"2024-04-25T10:30:31.404392+0000","last_became_peered":"2024-04-25T10:30:31.404392+0000","last_unstale":"2024-04-25T10:30:33.415622+0000","last_undegraded":"2024-04-25T10:30:33.415622+0000","last_fullsized":"2024-04-25T10:30:33.415622+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:00.965450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795922+0000","last_change":"2024-04-25T10:30:32.441438+0000","last_active":"2024-04-25T10:30:34.795922+0000","last_peered":"2024-04-25T10:30:34.795922+0000","last_clean":"2024-04-25T10:30:34.795922+0000","last_became_active":"2024-04-25T10:30:32.440011+0000","last_became_peered":"2024-04-25T10:30:32.440011+0000","last_unstale":"2024-04-25T10:30:34.795922+0000","last_undegraded":"2024-04-25T10:30:34.795922+0000","last_fullsized":"2024-04-25T10:30:34.795922+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:09.127517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":110,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.793865+0000","last_change":"2024-04-25T10:30:21.429098+0000","last_active":"2024-04-25T10:30:34.793865+0000","last_peered":"2024-04-25T10:30:34.793865+0000","last_clean":"2024-04-25T10:30:34.793865+0000","last_became_active":"2024-04-25T10:30:21.427958+0000","last_became_peered":"2024-04-25T10:30:21.427958+0000","last_unstale":"2024-04-25T10:30:34.793865+0000","last_undegraded":"2024-04-25T10:30:34.793865+0000","last_fullsized":"2024-04-25T10:30:34.793865+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:20.186206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:20.186206+0000","last_clean_scrub_stamp":"2024-04-25T10:30:20.186206+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:46.046488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.793946+0000","last_change":"2024-04-25T10:30:31.404974+0000","last_active":"2024-04-25T10:30:34.793946+0000","last_peered":"2024-04-25T10:30:34.793946+0000","last_clean":"2024-04-25T10:30:34.793946+0000","last_became_active":"2024-04-25T10:30:31.404549+0000","last_became_peered":"2024-04-25T10:30:31.404549+0000","last_unstale":"2024-04-25T10:30:34.793946+0000","last_undegraded":"2024-04-25T10:30:34.793946+0000","last_fullsized":"2024-04-25T10:30:34.793946+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:06.191321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415412+0000","last_change":"2024-04-25T10:30:32.434409+0000","last_active":"2024-04-25T10:30:33.415412+0000","last_peered":"2024-04-25T10:30:33.415412+0000","last_clean":"2024-04-25T10:30:33.415412+0000","last_became_active":"2024-04-25T10:30:32.433830+0000","last_became_peered":"2024-04-25T10:30:32.433830+0000","last_unstale":"2024-04-25T10:30:33.415412+0000","last_undegraded":"2024-04-25T10:30:33.415412+0000","last_fullsized":"2024-04-25T10:30:33.415412+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:57.864627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.793793+0000","last_change":"2024-04-25T10:30:31.416025+0000","last_active":"2024-04-25T10:30:34.793793+0000","last_peered":"2024-04-25T10:30:34.793793+0000","last_clean":"2024-04-25T10:30:34.793793+0000","last_became_active":"2024-04-25T10:30:31.415675+0000","last_became_peered":"2024-04-25T10:30:31.415675+0000","last_unstale":"2024-04-25T10:30:34.793793+0000","last_undegraded":"2024-04-25T10:30:34.793793+0000","last_fullsized":"2024-04-25T10:30:34.793793+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:28:48.185715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.196505+0000","last_change":"2024-04-25T10:30:32.424630+0000","last_active":"2024-04-25T10:30:34.196505+0000","last_peered":"2024-04-25T10:30:34.196505+0000","last_clean":"2024-04-25T10:30:34.196505+0000","last_became_active":"2024-04-25T10:30:32.424303+0000","last_became_peered":"2024-04-25T10:30:32.424303+0000","last_unstale":"2024-04-25T10:30:34.196505+0000","last_undegraded":"2024-04-25T10:30:34.196505+0000","last_fullsized":"2024-04-25T10:30:34.196505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:55.535319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415284+0000","last_change":"2024-04-25T10:30:31.419145+0000","last_active":"2024-04-25T10:30:33.415284+0000","last_peered":"2024-04-25T10:30:33.415284+0000","last_clean":"2024-04-25T10:30:33.415284+0000","last_became_active":"2024-04-25T10:30:31.418817+0000","last_became_peered":"2024-04-25T10:30:31.418817+0000","last_unstale":"2024-04-25T10:30:33.415284+0000","last_undegraded":"2024-04-25T10:30:33.415284+0000","last_fullsized":"2024-04-25T10:30:33.415284+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:01.608205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.412646+0000","last_change":"2024-04-25T10:30:32.419911+0000","last_active":"2024-04-25T10:30:33.412646+0000","last_peered":"2024-04-25T10:30:33.412646+0000","last_clean":"2024-04-25T10:30:33.412646+0000","last_became_active":"2024-04-25T10:30:32.419486+0000","last_became_peered":"2024-04-25T10:30:32.419486+0000","last_unstale":"2024-04-25T10:30:33.412646+0000","last_undegraded":"2024-04-25T10:30:33.412646+0000","last_fullsized":"2024-04-25T10:30:33.412646+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:54.967873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.796060+0000","last_change":"2024-04-25T10:30:31.405646+0000","last_active":"2024-04-25T10:30:34.796060+0000","last_peered":"2024-04-25T10:30:34.796060+0000","last_clean":"2024-04-25T10:30:34.796060+0000","last_became_active":"2024-04-25T10:30:31.405238+0000","last_became_peered":"2024-04-25T10:30:31.405238+0000","last_unstale":"2024-04-25T10:30:34.796060+0000","last_undegraded":"2024-04-25T10:30:34.796060+0000","last_fullsized":"2024-04-25T10:30:34.796060+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:13.159787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795730+0000","last_change":"2024-04-25T10:30:32.426130+0000","last_active":"2024-04-25T10:30:34.795730+0000","last_peered":"2024-04-25T10:30:34.795730+0000","last_clean":"2024-04-25T10:30:34.795730+0000","last_became_active":"2024-04-25T10:30:32.425783+0000","last_became_peered":"2024-04-25T10:30:32.425783+0000","last_unstale":"2024-04-25T10:30:34.795730+0000","last_undegraded":"2024-04-25T10:30:34.795730+0000","last_fullsized":"2024-04-25T10:30:34.795730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:30.844640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.412747+0000","last_change":"2024-04-25T10:30:31.412736+0000","last_active":"2024-04-25T10:30:33.412747+0000","last_peered":"2024-04-25T10:30:33.412747+0000","last_clean":"2024-04-25T10:30:33.412747+0000","last_became_active":"2024-04-25T10:30:31.412413+0000","last_became_peered":"2024-04-25T10:30:31.412413+0000","last_unstale":"2024-04-25T10:30:33.412747+0000","last_undegraded":"2024-04-25T10:30:33.412747+0000","last_fullsized":"2024-04-25T10:30:33.412747+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:19:56.236327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795688+0000","last_change":"2024-04-25T10:30:32.440858+0000","last_active":"2024-04-25T10:30:34.795688+0000","last_peered":"2024-04-25T10:30:34.795688+0000","last_clean":"2024-04-25T10:30:34.795688+0000","last_became_active":"2024-04-25T10:30:32.440519+0000","last_became_peered":"2024-04-25T10:30:32.440519+0000","last_unstale":"2024-04-25T10:30:34.795688+0000","last_undegraded":"2024-04-25T10:30:34.795688+0000","last_fullsized":"2024-04-25T10:30:34.795688+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:21.288748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.796138+0000","last_change":"2024-04-25T10:30:31.417958+0000","last_active":"2024-04-25T10:30:34.796138+0000","last_peered":"2024-04-25T10:30:34.796138+0000","last_clean":"2024-04-25T10:30:34.796138+0000","last_became_active":"2024-04-25T10:30:31.417541+0000","last_became_peered":"2024-04-25T10:30:31.417541+0000","last_unstale":"2024-04-25T10:30:34.796138+0000","last_undegraded":"2024-04-25T10:30:34.796138+0000","last_fullsized":"2024-04-25T10:30:34.796138+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:35.373351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.412553+0000","last_change":"2024-04-25T10:30:32.420451+0000","last_active":"2024-04-25T10:30:33.412553+0000","last_peered":"2024-04-25T10:30:33.412553+0000","last_clean":"2024-04-25T10:30:33.412553+0000","last_became_active":"2024-04-25T10:30:32.420109+0000","last_became_peered":"2024-04-25T10:30:32.420109+0000","last_unstale":"2024-04-25T10:30:33.412553+0000","last_undegraded":"2024-04-25T10:30:33.412553+0000","last_fullsized":"2024-04-25T10:30:33.412553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:22.067828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.412601+0000","last_change":"2024-04-25T10:30:31.406005+0000","last_active":"2024-04-25T10:30:33.412601+0000","last_peered":"2024-04-25T10:30:33.412601+0000","last_clean":"2024-04-25T10:30:33.412601+0000","last_became_active":"2024-04-25T10:30:31.404739+0000","last_became_peered":"2024-04-25T10:30:31.404739+0000","last_unstale":"2024-04-25T10:30:33.412601+0000","last_undegraded":"2024-04-25T10:30:33.412601+0000","last_fullsized":"2024-04-25T10:30:33.412601+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:31.771123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415711+0000","last_change":"2024-04-25T10:30:32.437513+0000","last_active":"2024-04-25T10:30:33.415711+0000","last_peered":"2024-04-25T10:30:33.415711+0000","last_clean":"2024-04-25T10:30:33.415711+0000","last_became_active":"2024-04-25T10:30:32.437171+0000","last_became_peered":"2024-04-25T10:30:32.437171+0000","last_unstale":"2024-04-25T10:30:33.415711+0000","last_undegraded":"2024-04-25T10:30:33.415711+0000","last_fullsized":"2024-04-25T10:30:33.415711+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:50.924843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:37.436706+0000","last_change":"2024-04-25T10:30:31.413140+0000","last_active":"2024-04-25T10:30:37.436706+0000","last_peered":"2024-04-25T10:30:37.436706+0000","last_clean":"2024-04-25T10:30:37.436706+0000","last_became_active":"2024-04-25T10:30:31.412821+0000","last_became_peered":"2024-04-25T10:30:31.412821+0000","last_unstale":"2024-04-25T10:30:37.436706+0000","last_undegraded":"2024-04-25T10:30:37.436706+0000","last_fullsized":"2024-04-25T10:30:37.436706+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:37.177254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.196799+0000","last_change":"2024-04-25T10:30:32.434333+0000","last_active":"2024-04-25T10:30:34.196799+0000","last_peered":"2024-04-25T10:30:34.196799+0000","last_clean":"2024-04-25T10:30:34.196799+0000","last_became_active":"2024-04-25T10:30:32.434000+0000","last_became_peered":"2024-04-25T10:30:32.434000+0000","last_unstale":"2024-04-25T10:30:34.196799+0000","last_undegraded":"2024-04-25T10:30:34.196799+0000","last_fullsized":"2024-04-25T10:30:34.196799+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:54.057457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794573+0000","last_change":"2024-04-25T10:30:31.419475+0000","last_active":"2024-04-25T10:30:34.794573+0000","last_peered":"2024-04-25T10:30:34.794573+0000","last_clean":"2024-04-25T10:30:34.794573+0000","last_became_active":"2024-04-25T10:30:31.417605+0000","last_became_peered":"2024-04-25T10:30:31.417605+0000","last_unstale":"2024-04-25T10:30:34.794573+0000","last_undegraded":"2024-04-25T10:30:34.794573+0000","last_fullsized":"2024-04-25T10:30:34.794573+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:00.948269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794414+0000","last_change":"2024-04-25T10:30:32.419693+0000","last_active":"2024-04-25T10:30:34.794414+0000","last_peered":"2024-04-25T10:30:34.794414+0000","last_clean":"2024-04-25T10:30:34.794414+0000","last_became_active":"2024-04-25T10:30:32.419370+0000","last_became_peered":"2024-04-25T10:30:32.419370+0000","last_unstale":"2024-04-25T10:30:34.794414+0000","last_undegraded":"2024-04-25T10:30:34.794414+0000","last_fullsized":"2024-04-25T10:30:34.794414+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:31.199065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.412581+0000","last_change":"2024-04-25T10:30:31.410767+0000","last_active":"2024-04-25T10:30:33.412581+0000","last_peered":"2024-04-25T10:30:33.412581+0000","last_clean":"2024-04-25T10:30:33.412581+0000","last_became_active":"2024-04-25T10:30:31.410404+0000","last_became_peered":"2024-04-25T10:30:31.410404+0000","last_unstale":"2024-04-25T10:30:33.412581+0000","last_undegraded":"2024-04-25T10:30:33.412581+0000","last_fullsized":"2024-04-25T10:30:33.412581+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:37.431532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415585+0000","last_change":"2024-04-25T10:30:32.433687+0000","last_active":"2024-04-25T10:30:33.415585+0000","last_peered":"2024-04-25T10:30:33.415585+0000","last_clean":"2024-04-25T10:30:33.415585+0000","last_became_active":"2024-04-25T10:30:32.433389+0000","last_became_peered":"2024-04-25T10:30:32.433389+0000","last_unstale":"2024-04-25T10:30:33.415585+0000","last_undegraded":"2024-04-25T10:30:33.415585+0000","last_fullsized":"2024-04-25T10:30:33.415585+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:39.155317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.412506+0000","last_change":"2024-04-25T10:30:31.419118+0000","last_active":"2024-04-25T10:30:33.412506+0000","last_peered":"2024-04-25T10:30:33.412506+0000","last_clean":"2024-04-25T10:30:33.412506+0000","last_became_active":"2024-04-25T10:30:31.418822+0000","last_became_peered":"2024-04-25T10:30:31.418822+0000","last_unstale":"2024-04-25T10:30:33.412506+0000","last_undegraded":"2024-04-25T10:30:33.412506+0000","last_fullsized":"2024-04-25T10:30:33.412506+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:22.755042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415890+0000","last_change":"2024-04-25T10:30:32.427894+0000","last_active":"2024-04-25T10:30:33.415890+0000","last_peered":"2024-04-25T10:30:33.415890+0000","last_clean":"2024-04-25T10:30:33.415890+0000","last_became_active":"2024-04-25T10:30:32.427504+0000","last_became_peered":"2024-04-25T10:30:32.427504+0000","last_unstale":"2024-04-25T10:30:33.415890+0000","last_undegraded":"2024-04-25T10:30:33.415890+0000","last_fullsized":"2024-04-25T10:30:33.415890+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:53.877484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:37.436274+0000","last_change":"2024-04-25T10:30:31.417115+0000","last_active":"2024-04-25T10:30:37.436274+0000","last_peered":"2024-04-25T10:30:37.436274+0000","last_clean":"2024-04-25T10:30:37.436274+0000","last_became_active":"2024-04-25T10:30:31.416766+0000","last_became_peered":"2024-04-25T10:30:31.416766+0000","last_unstale":"2024-04-25T10:30:37.436274+0000","last_undegraded":"2024-04-25T10:30:37.436274+0000","last_fullsized":"2024-04-25T10:30:37.436274+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:06.021270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795407+0000","last_change":"2024-04-25T10:30:32.431674+0000","last_active":"2024-04-25T10:30:34.795407+0000","last_peered":"2024-04-25T10:30:34.795407+0000","last_clean":"2024-04-25T10:30:34.795407+0000","last_became_active":"2024-04-25T10:30:32.431395+0000","last_became_peered":"2024-04-25T10:30:32.431395+0000","last_unstale":"2024-04-25T10:30:34.795407+0000","last_undegraded":"2024-04-25T10:30:34.795407+0000","last_fullsized":"2024-04-25T10:30:34.795407+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:49.726323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795508+0000","last_change":"2024-04-25T10:30:32.437700+0000","last_active":"2024-04-25T10:30:34.795508+0000","last_peered":"2024-04-25T10:30:34.795508+0000","last_clean":"2024-04-25T10:30:34.795508+0000","last_became_active":"2024-04-25T10:30:32.437242+0000","last_became_peered":"2024-04-25T10:30:32.437242+0000","last_unstale":"2024-04-25T10:30:34.795508+0000","last_undegraded":"2024-04-25T10:30:34.795508+0000","last_fullsized":"2024-04-25T10:30:34.795508+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:06.499414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:37.435895+0000","last_change":"2024-04-25T10:30:31.419678+0000","last_active":"2024-04-25T10:30:37.435895+0000","last_peered":"2024-04-25T10:30:37.435895+0000","last_clean":"2024-04-25T10:30:37.435895+0000","last_became_active":"2024-04-25T10:30:31.419269+0000","last_became_peered":"2024-04-25T10:30:31.419269+0000","last_unstale":"2024-04-25T10:30:37.435895+0000","last_undegraded":"2024-04-25T10:30:37.435895+0000","last_fullsized":"2024-04-25T10:30:37.435895+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:33.034260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795138+0000","last_change":"2024-04-25T10:30:32.434410+0000","last_active":"2024-04-25T10:30:34.795138+0000","last_peered":"2024-04-25T10:30:34.795138+0000","last_clean":"2024-04-25T10:30:34.795138+0000","last_became_active":"2024-04-25T10:30:32.434093+0000","last_became_peered":"2024-04-25T10:30:32.434093+0000","last_unstale":"2024-04-25T10:30:34.795138+0000","last_undegraded":"2024-04-25T10:30:34.795138+0000","last_fullsized":"2024-04-25T10:30:34.795138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:46.964704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:37.434829+0000","last_change":"2024-04-25T10:30:31.418360+0000","last_active":"2024-04-25T10:30:37.434829+0000","last_peered":"2024-04-25T10:30:37.434829+0000","last_clean":"2024-04-25T10:30:37.434829+0000","last_became_active":"2024-04-25T10:30:31.418005+0000","last_became_peered":"2024-04-25T10:30:31.418005+0000","last_unstale":"2024-04-25T10:30:37.434829+0000","last_undegraded":"2024-04-25T10:30:37.434829+0000","last_fullsized":"2024-04-25T10:30:37.434829+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:15.556859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.196705+0000","last_change":"2024-04-25T10:30:32.439515+0000","last_active":"2024-04-25T10:30:34.196705+0000","last_peered":"2024-04-25T10:30:34.196705+0000","last_clean":"2024-04-25T10:30:34.196705+0000","last_became_active":"2024-04-25T10:30:32.439101+0000","last_became_peered":"2024-04-25T10:30:32.439101+0000","last_unstale":"2024-04-25T10:30:34.196705+0000","last_undegraded":"2024-04-25T10:30:34.196705+0000","last_fullsized":"2024-04-25T10:30:34.196705+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:33.797903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.120347+0000","last_change":"2024-04-25T10:30:31.418070+0000","last_active":"2024-04-25T10:30:35.120347+0000","last_peered":"2024-04-25T10:30:35.120347+0000","last_clean":"2024-04-25T10:30:35.120347+0000","last_became_active":"2024-04-25T10:30:31.417737+0000","last_became_peered":"2024-04-25T10:30:31.417737+0000","last_unstale":"2024-04-25T10:30:35.120347+0000","last_undegraded":"2024-04-25T10:30:35.120347+0000","last_fullsized":"2024-04-25T10:30:35.120347+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:05.664855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415785+0000","last_change":"2024-04-25T10:30:32.435258+0000","last_active":"2024-04-25T10:30:33.415785+0000","last_peered":"2024-04-25T10:30:33.415785+0000","last_clean":"2024-04-25T10:30:33.415785+0000","last_became_active":"2024-04-25T10:30:32.434930+0000","last_became_peered":"2024-04-25T10:30:32.434930+0000","last_unstale":"2024-04-25T10:30:33.415785+0000","last_undegraded":"2024-04-25T10:30:33.415785+0000","last_fullsized":"2024-04-25T10:30:33.415785+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:22.727778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:37.436552+0000","last_change":"2024-04-25T10:30:31.415611+0000","last_active":"2024-04-25T10:30:37.436552+0000","last_peered":"2024-04-25T10:30:37.436552+0000","last_clean":"2024-04-25T10:30:37.436552+0000","last_became_active":"2024-04-25T10:30:31.415071+0000","last_became_peered":"2024-04-25T10:30:31.415071+0000","last_unstale":"2024-04-25T10:30:37.436552+0000","last_undegraded":"2024-04-25T10:30:37.436552+0000","last_fullsized":"2024-04-25T10:30:37.436552+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:30:54.077864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27800,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344040,"statfs":{"total":96636764160,"available":96608296960,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:30:41.991 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:30:42.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.132+0000 7fe270a61640 1 Processor -- start 2024-04-25T10:30:42.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.132+0000 7fe270a61640 1 -- start start 2024-04-25T10:30:42.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.134+0000 7fe270a61640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fe26c0c7e30 0x7fe26c0c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:42.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.134+0000 7fe270a61640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe26c156ef0 0x7fe26c0b4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:42.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.134+0000 7fe270a61640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe26c0ce630 con 0x7fe26c06d000 2024-04-25T10:30:42.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.134+0000 7fe270a61640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fe26c0c7c60 con 0x7fe26c156ef0 2024-04-25T10:30:42.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.134+0000 7fe270a61640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fe26c0c7980 con 0x7fe26c0c7e30 2024-04-25T10:30:42.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.134+0000 7fe269d9b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe26c156ef0 0x7fe26c0b4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:42.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.134+0000 7fe269d9b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe26c156ef0 0x7fe26c0b4ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48006/0 (socket says 172.21.15.106:48006) 2024-04-25T10:30:42.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.134+0000 7fe269d9b640 1 -- 172.21.15.106:0/738466552 learned_addr learned my addr 172.21.15.106:0/738466552 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:42.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.134+0000 7fe26a59c640 1 --2- 172.21.15.106:0/738466552 >> v2:172.21.15.134:3300/0 conn(0x7fe26c0c7e30 0x7fe26c0c8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:42.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.134+0000 7fe269d9b640 1 -- 172.21.15.106:0/738466552 >> v2:172.21.15.134:3300/0 conn(0x7fe26c0c7e30 msgr2=0x7fe26c0c8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:42.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.134+0000 7fe269d9b640 1 --2- 172.21.15.106:0/738466552 >> v2:172.21.15.134:3300/0 conn(0x7fe26c0c7e30 0x7fe26c0c8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:42.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.134+0000 7fe269d9b640 1 -- 172.21.15.106:0/738466552 >> v1:172.21.15.134:6789/0 conn(0x7fe26c06d000 legacy=0x7fe26c15ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:42.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.134+0000 7fe269d9b640 1 -- 172.21.15.106:0/738466552 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe26c0c7010 con 0x7fe26c156ef0 2024-04-25T10:30:42.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.135+0000 7fe269d9b640 1 --2- 172.21.15.106:0/738466552 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe26c156ef0 0x7fe26c0b4ce0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe26c0c7de0 tx=0x7fe25402a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4b6971a0c1cb1a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:42.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.135+0000 7fe26959a640 1 -- 172.21.15.106:0/738466552 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe254004030 con 0x7fe26c156ef0 2024-04-25T10:30:42.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.135+0000 7fe26959a640 1 -- 172.21.15.106:0/738466552 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2540041d0 con 0x7fe26c156ef0 2024-04-25T10:30:42.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.135+0000 7fe26959a640 1 -- 172.21.15.106:0/738466552 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe254003610 con 0x7fe26c156ef0 2024-04-25T10:30:42.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.136+0000 7fe270a61640 1 -- 172.21.15.106:0/738466552 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe26c156ef0 msgr2=0x7fe26c0b4ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:42.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.136+0000 7fe270a61640 1 --2- 172.21.15.106:0/738466552 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe26c156ef0 0x7fe26c0b4ce0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe26c0c7de0 tx=0x7fe25402a160 comp rx=0 tx=0).stop 2024-04-25T10:30:42.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.136+0000 7fe270a61640 1 -- 172.21.15.106:0/738466552 shutdown_connections 2024-04-25T10:30:42.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.136+0000 7fe270a61640 1 --2- 172.21.15.106:0/738466552 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe26c156ef0 0x7fe26c0b4ce0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:42.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.136+0000 7fe270a61640 1 --2- 172.21.15.106:0/738466552 >> v2:172.21.15.134:3300/0 conn(0x7fe26c0c7e30 0x7fe26c0c8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:42.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.136+0000 7fe270a61640 1 -- 172.21.15.106:0/738466552 >> 172.21.15.106:0/738466552 conn(0x7fe26c068b10 msgr2=0x7fe26c0d1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:42.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.136+0000 7fe270a61640 1 -- 172.21.15.106:0/738466552 shutdown_connections 2024-04-25T10:30:42.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.136+0000 7fe270a61640 1 -- 172.21.15.106:0/738466552 wait complete. 2024-04-25T10:30:42.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.138+0000 7fe270a61640 1 Processor -- start 2024-04-25T10:30:42.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.138+0000 7fe270a61640 1 -- start start 2024-04-25T10:30:42.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.139+0000 7fe270a61640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe26c06d000 0x7fe26c15b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:42.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.139+0000 7fe270a61640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe26c0c7e30 0x7fe26c153440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:42.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.139+0000 7fe270a61640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe26c156ef0 0x7fe26c143890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:42.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.139+0000 7fe270a61640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe26c15d6c0 con 0x7fe26c06d000 2024-04-25T10:30:42.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.139+0000 7fe270a61640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe26c15d540 con 0x7fe26c156ef0 2024-04-25T10:30:42.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.139+0000 7fe270a61640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe26c15bf50 con 0x7fe26c0c7e30 2024-04-25T10:30:42.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.139+0000 7fe26a59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe26c06d000 0x7fe26c15b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:42.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.139+0000 7fe26a59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe26c06d000 0x7fe26c15b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48008/0 (socket says 172.21.15.106:48008) 2024-04-25T10:30:42.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.139+0000 7fe26a59c640 1 -- 172.21.15.106:0/3050679476 learned_addr learned my addr 172.21.15.106:0/3050679476 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:42.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.139+0000 7fe269d9b640 1 --2- 172.21.15.106:0/3050679476 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe26c0c7e30 0x7fe26c153440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:42.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.139+0000 7fe26ad9d640 1 --2- 172.21.15.106:0/3050679476 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe26c156ef0 0x7fe26c143890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:42.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.139+0000 7fe26a59c640 1 -- 172.21.15.106:0/3050679476 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe26c0c7e30 msgr2=0x7fe26c153440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:42.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.139+0000 7fe26a59c640 1 --2- 172.21.15.106:0/3050679476 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe26c0c7e30 0x7fe26c153440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:42.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.139+0000 7fe26a59c640 1 -- 172.21.15.106:0/3050679476 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe26c156ef0 msgr2=0x7fe26c143890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:42.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.139+0000 7fe26a59c640 1 --2- 172.21.15.106:0/3050679476 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe26c156ef0 0x7fe26c143890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:42.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.139+0000 7fe26a59c640 1 -- 172.21.15.106:0/3050679476 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe26c15ba80 con 0x7fe26c06d000 2024-04-25T10:30:42.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.139+0000 7fe26a59c640 1 --2- 172.21.15.106:0/3050679476 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe26c06d000 0x7fe26c15b540 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe264007e70 tx=0x7fe26400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:42.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.140+0000 7fe262ffd640 1 -- 172.21.15.106:0/3050679476 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe264014620 con 0x7fe26c06d000 2024-04-25T10:30:42.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.140+0000 7fe262ffd640 1 -- 172.21.15.106:0/3050679476 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe264019070 con 0x7fe26c06d000 2024-04-25T10:30:42.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.140+0000 7fe262ffd640 1 -- 172.21.15.106:0/3050679476 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe264014a00 con 0x7fe26c06d000 2024-04-25T10:30:42.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.140+0000 7fe270a61640 1 -- 172.21.15.106:0/3050679476 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe26c0d1ca0 con 0x7fe26c06d000 2024-04-25T10:30:42.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.140+0000 7fe270a61640 1 -- 172.21.15.106:0/3050679476 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe26c150eb0 con 0x7fe26c06d000 2024-04-25T10:30:42.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.141+0000 7fe270a61640 1 -- 172.21.15.106:0/3050679476 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe26c0cdde0 con 0x7fe26c06d000 2024-04-25T10:30:42.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.141+0000 7fe262ffd640 1 -- 172.21.15.106:0/3050679476 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe264014ba0 con 0x7fe26c06d000 2024-04-25T10:30:42.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.142+0000 7fe262ffd640 1 --2- 172.21.15.106:0/3050679476 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fe24c0744b0 0x7fe24c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:42.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.142+0000 7fe262ffd640 1 -- 172.21.15.106:0/3050679476 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe264095930 con 0x7fe26c06d000 2024-04-25T10:30:42.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.142+0000 7fe269d9b640 1 --2- 172.21.15.106:0/3050679476 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fe24c0744b0 0x7fe24c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:42.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.142+0000 7fe269d9b640 1 --2- 172.21.15.106:0/3050679476 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fe24c0744b0 0x7fe24c0769c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe254033000 tx=0x7fe25402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:42.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.146+0000 7fe262ffd640 1 -- 172.21.15.106:0/3050679476 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe26405fb00 con 0x7fe26c06d000 2024-04-25T10:30:42.320 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.318+0000 7fe270a61640 1 -- 172.21.15.106:0/3050679476 --> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe26c06a3d0 con 0x7fe24c0744b0 2024-04-25T10:30:42.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.324+0000 7fe262ffd640 1 -- 172.21.15.106:0/3050679476 <== mgr.4107 v2:172.21.15.106:6832/1744210167 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251824 (secure 0 0 0) 0x7fe26c06a3d0 con 0x7fe24c0744b0 2024-04-25T10:30:42.329 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:30:42.330 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-25T10:30:42.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.327+0000 7fe270a61640 1 -- 172.21.15.106:0/3050679476 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fe24c0744b0 msgr2=0x7fe24c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:42.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.327+0000 7fe270a61640 1 --2- 172.21.15.106:0/3050679476 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fe24c0744b0 0x7fe24c0769c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe254033000 tx=0x7fe25402e000 comp rx=0 tx=0).stop 2024-04-25T10:30:42.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.327+0000 7fe270a61640 1 -- 172.21.15.106:0/3050679476 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe26c06d000 msgr2=0x7fe26c15b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:42.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.327+0000 7fe270a61640 1 --2- 172.21.15.106:0/3050679476 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe26c06d000 0x7fe26c15b540 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe264007e70 tx=0x7fe26400e4b0 comp rx=0 tx=0).stop 2024-04-25T10:30:42.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.327+0000 7fe270a61640 1 -- 172.21.15.106:0/3050679476 shutdown_connections 2024-04-25T10:30:42.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.327+0000 7fe270a61640 1 --2- 172.21.15.106:0/3050679476 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fe24c0744b0 0x7fe24c0769c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:42.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.327+0000 7fe270a61640 1 --2- 172.21.15.106:0/3050679476 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe26c156ef0 0x7fe26c143890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:42.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.327+0000 7fe270a61640 1 --2- 172.21.15.106:0/3050679476 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe26c0c7e30 0x7fe26c153440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:42.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.327+0000 7fe270a61640 1 --2- 172.21.15.106:0/3050679476 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe26c06d000 0x7fe26c15b540 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:42.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.327+0000 7fe270a61640 1 -- 172.21.15.106:0/3050679476 >> 172.21.15.106:0/3050679476 conn(0x7fe26c068b10 msgr2=0x7fe26c0d1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:42.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.327+0000 7fe270a61640 1 -- 172.21.15.106:0/3050679476 shutdown_connections 2024-04-25T10:30:42.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.327+0000 7fe270a61640 1 -- 172.21.15.106:0/3050679476 wait complete. 2024-04-25T10:30:42.341 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-04-25T10:30:42.098395+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218952,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755768,"statfs":{"total":773094113280,"available":772869906432,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.688638"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795275+0000","last_change":"2024-04-25T10:30:32.440983+0000","last_active":"2024-04-25T10:30:34.795275+0000","last_peered":"2024-04-25T10:30:34.795275+0000","last_clean":"2024-04-25T10:30:34.795275+0000","last_became_active":"2024-04-25T10:30:32.440552+0000","last_became_peered":"2024-04-25T10:30:32.440552+0000","last_unstale":"2024-04-25T10:30:34.795275+0000","last_undegraded":"2024-04-25T10:30:34.795275+0000","last_fullsized":"2024-04-25T10:30:34.795275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:47.963061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.412686+0000","last_change":"2024-04-25T10:30:32.425695+0000","last_active":"2024-04-25T10:30:33.412686+0000","last_peered":"2024-04-25T10:30:33.412686+0000","last_clean":"2024-04-25T10:30:33.412686+0000","last_became_active":"2024-04-25T10:30:32.425368+0000","last_became_peered":"2024-04-25T10:30:32.425368+0000","last_unstale":"2024-04-25T10:30:33.412686+0000","last_undegraded":"2024-04-25T10:30:33.412686+0000","last_fullsized":"2024-04-25T10:30:33.412686+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:39.120522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.196620+0000","last_change":"2024-04-25T10:30:32.430144+0000","last_active":"2024-04-25T10:30:34.196620+0000","last_peered":"2024-04-25T10:30:34.196620+0000","last_clean":"2024-04-25T10:30:34.196620+0000","last_became_active":"2024-04-25T10:30:32.429754+0000","last_became_peered":"2024-04-25T10:30:32.429754+0000","last_unstale":"2024-04-25T10:30:34.196620+0000","last_undegraded":"2024-04-25T10:30:34.196620+0000","last_fullsized":"2024-04-25T10:30:34.196620+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:11.448907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794948+0000","last_change":"2024-04-25T10:30:32.442478+0000","last_active":"2024-04-25T10:30:34.794948+0000","last_peered":"2024-04-25T10:30:34.794948+0000","last_clean":"2024-04-25T10:30:34.794948+0000","last_became_active":"2024-04-25T10:30:32.442182+0000","last_became_peered":"2024-04-25T10:30:32.442182+0000","last_unstale":"2024-04-25T10:30:34.794948+0000","last_undegraded":"2024-04-25T10:30:34.794948+0000","last_fullsized":"2024-04-25T10:30:34.794948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:22.185978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794683+0000","last_change":"2024-04-25T10:30:32.438231+0000","last_active":"2024-04-25T10:30:34.794683+0000","last_peered":"2024-04-25T10:30:34.794683+0000","last_clean":"2024-04-25T10:30:34.794683+0000","last_became_active":"2024-04-25T10:30:32.437656+0000","last_became_peered":"2024-04-25T10:30:32.437656+0000","last_unstale":"2024-04-25T10:30:34.794683+0000","last_undegraded":"2024-04-25T10:30:34.794683+0000","last_fullsized":"2024-04-25T10:30:34.794683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:56.915999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795645+0000","last_change":"2024-04-25T10:30:32.439221+0000","last_active":"2024-04-25T10:30:34.795645+0000","last_peered":"2024-04-25T10:30:34.795645+0000","last_clean":"2024-04-25T10:30:34.795645+0000","last_became_active":"2024-04-25T10:30:32.438923+0000","last_became_peered":"2024-04-25T10:30:32.438923+0000","last_unstale":"2024-04-25T10:30:34.795645+0000","last_undegraded":"2024-04-25T10:30:34.795645+0000","last_fullsized":"2024-04-25T10:30:34.795645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:40.469409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794157+0000","last_change":"2024-04-25T10:30:32.429453+0000","last_active":"2024-04-25T10:30:34.794157+0000","last_peered":"2024-04-25T10:30:34.794157+0000","last_clean":"2024-04-25T10:30:34.794157+0000","last_became_active":"2024-04-25T10:30:32.429131+0000","last_became_peered":"2024-04-25T10:30:32.429131+0000","last_unstale":"2024-04-25T10:30:34.794157+0000","last_undegraded":"2024-04-25T10:30:34.794157+0000","last_fullsized":"2024-04-25T10:30:34.794157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:26.785922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415260+0000","last_change":"2024-04-25T10:30:32.438860+0000","last_active":"2024-04-25T10:30:33.415260+0000","last_peered":"2024-04-25T10:30:33.415260+0000","last_clean":"2024-04-25T10:30:33.415260+0000","last_became_active":"2024-04-25T10:30:32.438491+0000","last_became_peered":"2024-04-25T10:30:32.438491+0000","last_unstale":"2024-04-25T10:30:33.415260+0000","last_undegraded":"2024-04-25T10:30:33.415260+0000","last_fullsized":"2024-04-25T10:30:33.415260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:08.610707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795467+0000","last_change":"2024-04-25T10:30:32.440331+0000","last_active":"2024-04-25T10:30:34.795467+0000","last_peered":"2024-04-25T10:30:34.795467+0000","last_clean":"2024-04-25T10:30:34.795467+0000","last_became_active":"2024-04-25T10:30:32.440043+0000","last_became_peered":"2024-04-25T10:30:32.440043+0000","last_unstale":"2024-04-25T10:30:34.795467+0000","last_undegraded":"2024-04-25T10:30:34.795467+0000","last_fullsized":"2024-04-25T10:30:34.795467+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:17.684685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794636+0000","last_change":"2024-04-25T10:30:32.435388+0000","last_active":"2024-04-25T10:30:34.794636+0000","last_peered":"2024-04-25T10:30:34.794636+0000","last_clean":"2024-04-25T10:30:34.794636+0000","last_became_active":"2024-04-25T10:30:32.434983+0000","last_became_peered":"2024-04-25T10:30:32.434983+0000","last_unstale":"2024-04-25T10:30:34.794636+0000","last_undegraded":"2024-04-25T10:30:34.794636+0000","last_fullsized":"2024-04-25T10:30:34.794636+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:39.256604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.793928+0000","last_change":"2024-04-25T10:30:32.435884+0000","last_active":"2024-04-25T10:30:34.793928+0000","last_peered":"2024-04-25T10:30:34.793928+0000","last_clean":"2024-04-25T10:30:34.793928+0000","last_became_active":"2024-04-25T10:30:32.434645+0000","last_became_peered":"2024-04-25T10:30:32.434645+0000","last_unstale":"2024-04-25T10:30:34.793928+0000","last_undegraded":"2024-04-25T10:30:34.793928+0000","last_fullsized":"2024-04-25T10:30:34.793928+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:07.778437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795720+0000","last_change":"2024-04-25T10:30:32.436403+0000","last_active":"2024-04-25T10:30:34.795720+0000","last_peered":"2024-04-25T10:30:34.795720+0000","last_clean":"2024-04-25T10:30:34.795720+0000","last_became_active":"2024-04-25T10:30:32.435908+0000","last_became_peered":"2024-04-25T10:30:32.435908+0000","last_unstale":"2024-04-25T10:30:34.795720+0000","last_undegraded":"2024-04-25T10:30:34.795720+0000","last_fullsized":"2024-04-25T10:30:34.795720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:03.339316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415872+0000","last_change":"2024-04-25T10:30:32.432110+0000","last_active":"2024-04-25T10:30:33.415872+0000","last_peered":"2024-04-25T10:30:33.415872+0000","last_clean":"2024-04-25T10:30:33.415872+0000","last_became_active":"2024-04-25T10:30:32.431792+0000","last_became_peered":"2024-04-25T10:30:32.431792+0000","last_unstale":"2024-04-25T10:30:33.415872+0000","last_undegraded":"2024-04-25T10:30:33.415872+0000","last_fullsized":"2024-04-25T10:30:33.415872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:34.185989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.196476+0000","last_change":"2024-04-25T10:30:32.433676+0000","last_active":"2024-04-25T10:30:34.196476+0000","last_peered":"2024-04-25T10:30:34.196476+0000","last_clean":"2024-04-25T10:30:34.196476+0000","last_became_active":"2024-04-25T10:30:32.433329+0000","last_became_peered":"2024-04-25T10:30:32.433329+0000","last_unstale":"2024-04-25T10:30:34.196476+0000","last_undegraded":"2024-04-25T10:30:34.196476+0000","last_fullsized":"2024-04-25T10:30:34.196476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:59.942368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.412754+0000","last_change":"2024-04-25T10:30:32.423582+0000","last_active":"2024-04-25T10:30:33.412754+0000","last_peered":"2024-04-25T10:30:33.412754+0000","last_clean":"2024-04-25T10:30:33.412754+0000","last_became_active":"2024-04-25T10:30:32.422991+0000","last_became_peered":"2024-04-25T10:30:32.422991+0000","last_unstale":"2024-04-25T10:30:33.412754+0000","last_undegraded":"2024-04-25T10:30:33.412754+0000","last_fullsized":"2024-04-25T10:30:33.412754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:11:47.132928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794545+0000","last_change":"2024-04-25T10:30:32.440235+0000","last_active":"2024-04-25T10:30:34.794545+0000","last_peered":"2024-04-25T10:30:34.794545+0000","last_clean":"2024-04-25T10:30:34.794545+0000","last_became_active":"2024-04-25T10:30:32.439191+0000","last_became_peered":"2024-04-25T10:30:32.439191+0000","last_unstale":"2024-04-25T10:30:34.794545+0000","last_undegraded":"2024-04-25T10:30:34.794545+0000","last_fullsized":"2024-04-25T10:30:34.794545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:51.423086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795170+0000","last_change":"2024-04-25T10:30:32.442418+0000","last_active":"2024-04-25T10:30:34.795170+0000","last_peered":"2024-04-25T10:30:34.795170+0000","last_clean":"2024-04-25T10:30:34.795170+0000","last_became_active":"2024-04-25T10:30:32.442125+0000","last_became_peered":"2024-04-25T10:30:32.442125+0000","last_unstale":"2024-04-25T10:30:34.795170+0000","last_undegraded":"2024-04-25T10:30:34.795170+0000","last_fullsized":"2024-04-25T10:30:34.795170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:14.213809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415344+0000","last_change":"2024-04-25T10:30:32.434758+0000","last_active":"2024-04-25T10:30:33.415344+0000","last_peered":"2024-04-25T10:30:33.415344+0000","last_clean":"2024-04-25T10:30:33.415344+0000","last_became_active":"2024-04-25T10:30:32.434492+0000","last_became_peered":"2024-04-25T10:30:32.434492+0000","last_unstale":"2024-04-25T10:30:33.415344+0000","last_undegraded":"2024-04-25T10:30:33.415344+0000","last_fullsized":"2024-04-25T10:30:33.415344+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:41.454407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.796066+0000","last_change":"2024-04-25T10:30:32.440399+0000","last_active":"2024-04-25T10:30:34.796066+0000","last_peered":"2024-04-25T10:30:34.796066+0000","last_clean":"2024-04-25T10:30:34.796066+0000","last_became_active":"2024-04-25T10:30:32.438227+0000","last_became_peered":"2024-04-25T10:30:32.438227+0000","last_unstale":"2024-04-25T10:30:34.796066+0000","last_undegraded":"2024-04-25T10:30:34.796066+0000","last_fullsized":"2024-04-25T10:30:34.796066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:49.571671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794149+0000","last_change":"2024-04-25T10:30:32.431609+0000","last_active":"2024-04-25T10:30:34.794149+0000","last_peered":"2024-04-25T10:30:34.794149+0000","last_clean":"2024-04-25T10:30:34.794149+0000","last_became_active":"2024-04-25T10:30:32.431314+0000","last_became_peered":"2024-04-25T10:30:32.431314+0000","last_unstale":"2024-04-25T10:30:34.794149+0000","last_undegraded":"2024-04-25T10:30:34.794149+0000","last_fullsized":"2024-04-25T10:30:34.794149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:33.421989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.196668+0000","last_change":"2024-04-25T10:30:32.432612+0000","last_active":"2024-04-25T10:30:34.196668+0000","last_peered":"2024-04-25T10:30:34.196668+0000","last_clean":"2024-04-25T10:30:34.196668+0000","last_became_active":"2024-04-25T10:30:32.432271+0000","last_became_peered":"2024-04-25T10:30:32.432271+0000","last_unstale":"2024-04-25T10:30:34.196668+0000","last_undegraded":"2024-04-25T10:30:34.196668+0000","last_fullsized":"2024-04-25T10:30:34.196668+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:14.281493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795827+0000","last_change":"2024-04-25T10:30:32.444618+0000","last_active":"2024-04-25T10:30:34.795827+0000","last_peered":"2024-04-25T10:30:34.795827+0000","last_clean":"2024-04-25T10:30:34.795827+0000","last_became_active":"2024-04-25T10:30:32.444273+0000","last_became_peered":"2024-04-25T10:30:32.444273+0000","last_unstale":"2024-04-25T10:30:34.795827+0000","last_undegraded":"2024-04-25T10:30:34.795827+0000","last_fullsized":"2024-04-25T10:30:34.795827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:09.295251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415348+0000","last_change":"2024-04-25T10:30:32.426578+0000","last_active":"2024-04-25T10:30:33.415348+0000","last_peered":"2024-04-25T10:30:33.415348+0000","last_clean":"2024-04-25T10:30:33.415348+0000","last_became_active":"2024-04-25T10:30:32.426223+0000","last_became_peered":"2024-04-25T10:30:32.426223+0000","last_unstale":"2024-04-25T10:30:33.415348+0000","last_undegraded":"2024-04-25T10:30:33.415348+0000","last_fullsized":"2024-04-25T10:30:33.415348+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:53.700585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794721+0000","last_change":"2024-04-25T10:30:32.439767+0000","last_active":"2024-04-25T10:30:34.794721+0000","last_peered":"2024-04-25T10:30:34.794721+0000","last_clean":"2024-04-25T10:30:34.794721+0000","last_became_active":"2024-04-25T10:30:32.438801+0000","last_became_peered":"2024-04-25T10:30:32.438801+0000","last_unstale":"2024-04-25T10:30:34.794721+0000","last_undegraded":"2024-04-25T10:30:34.794721+0000","last_fullsized":"2024-04-25T10:30:34.794721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:58.231512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415544+0000","last_change":"2024-04-25T10:30:32.427424+0000","last_active":"2024-04-25T10:30:33.415544+0000","last_peered":"2024-04-25T10:30:33.415544+0000","last_clean":"2024-04-25T10:30:33.415544+0000","last_became_active":"2024-04-25T10:30:32.427081+0000","last_became_peered":"2024-04-25T10:30:32.427081+0000","last_unstale":"2024-04-25T10:30:33.415544+0000","last_undegraded":"2024-04-25T10:30:33.415544+0000","last_fullsized":"2024-04-25T10:30:33.415544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:47.372451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794727+0000","last_change":"2024-04-25T10:30:32.432472+0000","last_active":"2024-04-25T10:30:34.794727+0000","last_peered":"2024-04-25T10:30:34.794727+0000","last_clean":"2024-04-25T10:30:34.794727+0000","last_became_active":"2024-04-25T10:30:32.432121+0000","last_became_peered":"2024-04-25T10:30:32.432121+0000","last_unstale":"2024-04-25T10:30:34.794727+0000","last_undegraded":"2024-04-25T10:30:34.794727+0000","last_fullsized":"2024-04-25T10:30:34.794727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:34.191339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.412571+0000","last_change":"2024-04-25T10:30:32.425108+0000","last_active":"2024-04-25T10:30:33.412571+0000","last_peered":"2024-04-25T10:30:33.412571+0000","last_clean":"2024-04-25T10:30:33.412571+0000","last_became_active":"2024-04-25T10:30:32.424794+0000","last_became_peered":"2024-04-25T10:30:32.424794+0000","last_unstale":"2024-04-25T10:30:33.412571+0000","last_undegraded":"2024-04-25T10:30:33.412571+0000","last_fullsized":"2024-04-25T10:30:33.412571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:01.282758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415734+0000","last_change":"2024-04-25T10:30:32.426672+0000","last_active":"2024-04-25T10:30:33.415734+0000","last_peered":"2024-04-25T10:30:33.415734+0000","last_clean":"2024-04-25T10:30:33.415734+0000","last_became_active":"2024-04-25T10:30:32.426348+0000","last_became_peered":"2024-04-25T10:30:32.426348+0000","last_unstale":"2024-04-25T10:30:33.415734+0000","last_undegraded":"2024-04-25T10:30:33.415734+0000","last_fullsized":"2024-04-25T10:30:33.415734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:57.645772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.196609+0000","last_change":"2024-04-25T10:30:32.427989+0000","last_active":"2024-04-25T10:30:34.196609+0000","last_peered":"2024-04-25T10:30:34.196609+0000","last_clean":"2024-04-25T10:30:34.196609+0000","last_became_active":"2024-04-25T10:30:32.427676+0000","last_became_peered":"2024-04-25T10:30:32.427676+0000","last_unstale":"2024-04-25T10:30:34.196609+0000","last_undegraded":"2024-04-25T10:30:34.196609+0000","last_fullsized":"2024-04-25T10:30:34.196609+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:15:02.332186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795935+0000","last_change":"2024-04-25T10:30:32.441112+0000","last_active":"2024-04-25T10:30:34.795935+0000","last_peered":"2024-04-25T10:30:34.795935+0000","last_clean":"2024-04-25T10:30:34.795935+0000","last_became_active":"2024-04-25T10:30:32.440815+0000","last_became_peered":"2024-04-25T10:30:32.440815+0000","last_unstale":"2024-04-25T10:30:34.795935+0000","last_undegraded":"2024-04-25T10:30:34.795935+0000","last_fullsized":"2024-04-25T10:30:34.795935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:02.956759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415178+0000","last_change":"2024-04-25T10:30:32.430902+0000","last_active":"2024-04-25T10:30:33.415178+0000","last_peered":"2024-04-25T10:30:33.415178+0000","last_clean":"2024-04-25T10:30:33.415178+0000","last_became_active":"2024-04-25T10:30:32.430615+0000","last_became_peered":"2024-04-25T10:30:32.430615+0000","last_unstale":"2024-04-25T10:30:33.415178+0000","last_undegraded":"2024-04-25T10:30:33.415178+0000","last_fullsized":"2024-04-25T10:30:33.415178+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:12.747595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795247+0000","last_change":"2024-04-25T10:30:32.435379+0000","last_active":"2024-04-25T10:30:34.795247+0000","last_peered":"2024-04-25T10:30:34.795247+0000","last_clean":"2024-04-25T10:30:34.795247+0000","last_became_active":"2024-04-25T10:30:32.435055+0000","last_became_peered":"2024-04-25T10:30:32.435055+0000","last_unstale":"2024-04-25T10:30:34.795247+0000","last_undegraded":"2024-04-25T10:30:34.795247+0000","last_fullsized":"2024-04-25T10:30:34.795247+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:13.623762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794490+0000","last_change":"2024-04-25T10:30:32.422102+0000","last_active":"2024-04-25T10:30:34.794490+0000","last_peered":"2024-04-25T10:30:34.794490+0000","last_clean":"2024-04-25T10:30:34.794490+0000","last_became_active":"2024-04-25T10:30:32.421766+0000","last_became_peered":"2024-04-25T10:30:32.421766+0000","last_unstale":"2024-04-25T10:30:34.794490+0000","last_undegraded":"2024-04-25T10:30:34.794490+0000","last_fullsized":"2024-04-25T10:30:34.794490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:30.300572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.120320+0000","last_change":"2024-04-25T10:30:31.418065+0000","last_active":"2024-04-25T10:30:35.120320+0000","last_peered":"2024-04-25T10:30:35.120320+0000","last_clean":"2024-04-25T10:30:35.120320+0000","last_became_active":"2024-04-25T10:30:31.417451+0000","last_became_peered":"2024-04-25T10:30:31.417451+0000","last_unstale":"2024-04-25T10:30:35.120320+0000","last_undegraded":"2024-04-25T10:30:35.120320+0000","last_fullsized":"2024-04-25T10:30:35.120320+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:17.262861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.196777+0000","last_change":"2024-04-25T10:30:32.426913+0000","last_active":"2024-04-25T10:30:34.196777+0000","last_peered":"2024-04-25T10:30:34.196777+0000","last_clean":"2024-04-25T10:30:34.196777+0000","last_became_active":"2024-04-25T10:30:32.426616+0000","last_became_peered":"2024-04-25T10:30:32.426616+0000","last_unstale":"2024-04-25T10:30:34.196777+0000","last_undegraded":"2024-04-25T10:30:34.196777+0000","last_fullsized":"2024-04-25T10:30:34.196777+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:50.614932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.117929+0000","last_change":"2024-04-25T10:30:31.416432+0000","last_active":"2024-04-25T10:30:35.117929+0000","last_peered":"2024-04-25T10:30:35.117929+0000","last_clean":"2024-04-25T10:30:35.117929+0000","last_became_active":"2024-04-25T10:30:31.415112+0000","last_became_peered":"2024-04-25T10:30:31.415112+0000","last_unstale":"2024-04-25T10:30:35.117929+0000","last_undegraded":"2024-04-25T10:30:35.117929+0000","last_fullsized":"2024-04-25T10:30:35.117929+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:22.674365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415646+0000","last_change":"2024-04-25T10:30:31.418059+0000","last_active":"2024-04-25T10:30:33.415646+0000","last_peered":"2024-04-25T10:30:33.415646+0000","last_clean":"2024-04-25T10:30:33.415646+0000","last_became_active":"2024-04-25T10:30:31.417716+0000","last_became_peered":"2024-04-25T10:30:31.417716+0000","last_unstale":"2024-04-25T10:30:33.415646+0000","last_undegraded":"2024-04-25T10:30:33.415646+0000","last_fullsized":"2024-04-25T10:30:33.415646+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:57.752218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415428+0000","last_change":"2024-04-25T10:30:32.428078+0000","last_active":"2024-04-25T10:30:33.415428+0000","last_peered":"2024-04-25T10:30:33.415428+0000","last_clean":"2024-04-25T10:30:33.415428+0000","last_became_active":"2024-04-25T10:30:32.427700+0000","last_became_peered":"2024-04-25T10:30:32.427700+0000","last_unstale":"2024-04-25T10:30:33.415428+0000","last_undegraded":"2024-04-25T10:30:33.415428+0000","last_fullsized":"2024-04-25T10:30:33.415428+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:26.931791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795722+0000","last_change":"2024-04-25T10:30:31.415030+0000","last_active":"2024-04-25T10:30:34.795722+0000","last_peered":"2024-04-25T10:30:34.795722+0000","last_clean":"2024-04-25T10:30:34.795722+0000","last_became_active":"2024-04-25T10:30:31.414542+0000","last_became_peered":"2024-04-25T10:30:31.414542+0000","last_unstale":"2024-04-25T10:30:34.795722+0000","last_undegraded":"2024-04-25T10:30:34.795722+0000","last_fullsized":"2024-04-25T10:30:34.795722+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:20.082420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.412471+0000","last_change":"2024-04-25T10:30:32.425207+0000","last_active":"2024-04-25T10:30:33.412471+0000","last_peered":"2024-04-25T10:30:33.412471+0000","last_clean":"2024-04-25T10:30:33.412471+0000","last_became_active":"2024-04-25T10:30:32.424849+0000","last_became_peered":"2024-04-25T10:30:32.424849+0000","last_unstale":"2024-04-25T10:30:33.412471+0000","last_undegraded":"2024-04-25T10:30:33.412471+0000","last_fullsized":"2024-04-25T10:30:33.412471+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:11.533686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794249+0000","last_change":"2024-04-25T10:30:31.407700+0000","last_active":"2024-04-25T10:30:34.794249+0000","last_peered":"2024-04-25T10:30:34.794249+0000","last_clean":"2024-04-25T10:30:34.794249+0000","last_became_active":"2024-04-25T10:30:31.407376+0000","last_became_peered":"2024-04-25T10:30:31.407376+0000","last_unstale":"2024-04-25T10:30:34.794249+0000","last_undegraded":"2024-04-25T10:30:34.794249+0000","last_fullsized":"2024-04-25T10:30:34.794249+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:50.112742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794157+0000","last_change":"2024-04-25T10:30:32.444599+0000","last_active":"2024-04-25T10:30:34.794157+0000","last_peered":"2024-04-25T10:30:34.794157+0000","last_clean":"2024-04-25T10:30:34.794157+0000","last_became_active":"2024-04-25T10:30:32.444250+0000","last_became_peered":"2024-04-25T10:30:32.444250+0000","last_unstale":"2024-04-25T10:30:34.794157+0000","last_undegraded":"2024-04-25T10:30:34.794157+0000","last_fullsized":"2024-04-25T10:30:34.794157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:55.711525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794280+0000","last_change":"2024-04-25T10:30:31.405051+0000","last_active":"2024-04-25T10:30:34.794280+0000","last_peered":"2024-04-25T10:30:34.794280+0000","last_clean":"2024-04-25T10:30:34.794280+0000","last_became_active":"2024-04-25T10:30:31.404669+0000","last_became_peered":"2024-04-25T10:30:31.404669+0000","last_unstale":"2024-04-25T10:30:34.794280+0000","last_undegraded":"2024-04-25T10:30:34.794280+0000","last_fullsized":"2024-04-25T10:30:34.794280+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:45.260847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.196763+0000","last_change":"2024-04-25T10:30:32.435804+0000","last_active":"2024-04-25T10:30:34.196763+0000","last_peered":"2024-04-25T10:30:34.196763+0000","last_clean":"2024-04-25T10:30:34.196763+0000","last_became_active":"2024-04-25T10:30:32.435471+0000","last_became_peered":"2024-04-25T10:30:32.435471+0000","last_unstale":"2024-04-25T10:30:34.196763+0000","last_undegraded":"2024-04-25T10:30:34.196763+0000","last_fullsized":"2024-04-25T10:30:34.196763+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:48.213846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:37.436569+0000","last_change":"2024-04-25T10:30:31.403560+0000","last_active":"2024-04-25T10:30:37.436569+0000","last_peered":"2024-04-25T10:30:37.436569+0000","last_clean":"2024-04-25T10:30:37.436569+0000","last_became_active":"2024-04-25T10:30:31.403167+0000","last_became_peered":"2024-04-25T10:30:31.403167+0000","last_unstale":"2024-04-25T10:30:37.436569+0000","last_undegraded":"2024-04-25T10:30:37.436569+0000","last_fullsized":"2024-04-25T10:30:37.436569+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:54.592832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795908+0000","last_change":"2024-04-25T10:30:32.424101+0000","last_active":"2024-04-25T10:30:34.795908+0000","last_peered":"2024-04-25T10:30:34.795908+0000","last_clean":"2024-04-25T10:30:34.795908+0000","last_became_active":"2024-04-25T10:30:32.423780+0000","last_became_peered":"2024-04-25T10:30:32.423780+0000","last_unstale":"2024-04-25T10:30:34.795908+0000","last_undegraded":"2024-04-25T10:30:34.795908+0000","last_fullsized":"2024-04-25T10:30:34.795908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:29.175348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:37.435669+0000","last_change":"2024-04-25T10:30:31.418201+0000","last_active":"2024-04-25T10:30:37.435669+0000","last_peered":"2024-04-25T10:30:37.435669+0000","last_clean":"2024-04-25T10:30:37.435669+0000","last_became_active":"2024-04-25T10:30:31.417824+0000","last_became_peered":"2024-04-25T10:30:31.417824+0000","last_unstale":"2024-04-25T10:30:37.435669+0000","last_undegraded":"2024-04-25T10:30:37.435669+0000","last_fullsized":"2024-04-25T10:30:37.435669+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:38.584225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795830+0000","last_change":"2024-04-25T10:30:32.431139+0000","last_active":"2024-04-25T10:30:34.795830+0000","last_peered":"2024-04-25T10:30:34.795830+0000","last_clean":"2024-04-25T10:30:34.795830+0000","last_became_active":"2024-04-25T10:30:32.429622+0000","last_became_peered":"2024-04-25T10:30:32.429622+0000","last_unstale":"2024-04-25T10:30:34.795830+0000","last_undegraded":"2024-04-25T10:30:34.795830+0000","last_fullsized":"2024-04-25T10:30:34.795830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:39.993647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:37.434076+0000","last_change":"2024-04-25T10:30:31.414775+0000","last_active":"2024-04-25T10:30:37.434076+0000","last_peered":"2024-04-25T10:30:37.434076+0000","last_clean":"2024-04-25T10:30:37.434076+0000","last_became_active":"2024-04-25T10:30:31.414422+0000","last_became_peered":"2024-04-25T10:30:31.414422+0000","last_unstale":"2024-04-25T10:30:37.434076+0000","last_undegraded":"2024-04-25T10:30:37.434076+0000","last_fullsized":"2024-04-25T10:30:37.434076+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:37.803505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.196648+0000","last_change":"2024-04-25T10:30:32.432884+0000","last_active":"2024-04-25T10:30:34.196648+0000","last_peered":"2024-04-25T10:30:34.196648+0000","last_clean":"2024-04-25T10:30:34.196648+0000","last_became_active":"2024-04-25T10:30:32.431743+0000","last_became_peered":"2024-04-25T10:30:32.431743+0000","last_unstale":"2024-04-25T10:30:34.196648+0000","last_undegraded":"2024-04-25T10:30:34.196648+0000","last_fullsized":"2024-04-25T10:30:34.196648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:15:06.737631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795999+0000","last_change":"2024-04-25T10:30:31.419497+0000","last_active":"2024-04-25T10:30:34.795999+0000","last_peered":"2024-04-25T10:30:34.795999+0000","last_clean":"2024-04-25T10:30:34.795999+0000","last_became_active":"2024-04-25T10:30:31.419135+0000","last_became_peered":"2024-04-25T10:30:31.419135+0000","last_unstale":"2024-04-25T10:30:34.795999+0000","last_undegraded":"2024-04-25T10:30:34.795999+0000","last_fullsized":"2024-04-25T10:30:34.795999+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:59.403927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794867+0000","last_change":"2024-04-25T10:30:32.434197+0000","last_active":"2024-04-25T10:30:34.794867+0000","last_peered":"2024-04-25T10:30:34.794867+0000","last_clean":"2024-04-25T10:30:34.794867+0000","last_became_active":"2024-04-25T10:30:32.433623+0000","last_became_peered":"2024-04-25T10:30:32.433623+0000","last_unstale":"2024-04-25T10:30:34.794867+0000","last_undegraded":"2024-04-25T10:30:34.794867+0000","last_fullsized":"2024-04-25T10:30:34.794867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:08.027472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.793813+0000","last_change":"2024-04-25T10:30:31.416901+0000","last_active":"2024-04-25T10:30:34.793813+0000","last_peered":"2024-04-25T10:30:34.793813+0000","last_clean":"2024-04-25T10:30:34.793813+0000","last_became_active":"2024-04-25T10:30:31.415706+0000","last_became_peered":"2024-04-25T10:30:31.415706+0000","last_unstale":"2024-04-25T10:30:34.793813+0000","last_undegraded":"2024-04-25T10:30:34.793813+0000","last_fullsized":"2024-04-25T10:30:34.793813+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:23.243039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795344+0000","last_change":"2024-04-25T10:30:32.435089+0000","last_active":"2024-04-25T10:30:34.795344+0000","last_peered":"2024-04-25T10:30:34.795344+0000","last_clean":"2024-04-25T10:30:34.795344+0000","last_became_active":"2024-04-25T10:30:32.434737+0000","last_became_peered":"2024-04-25T10:30:32.434737+0000","last_unstale":"2024-04-25T10:30:34.795344+0000","last_undegraded":"2024-04-25T10:30:34.795344+0000","last_fullsized":"2024-04-25T10:30:34.795344+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:19.738850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:37.436320+0000","last_change":"2024-04-25T10:30:31.405163+0000","last_active":"2024-04-25T10:30:37.436320+0000","last_peered":"2024-04-25T10:30:37.436320+0000","last_clean":"2024-04-25T10:30:37.436320+0000","last_became_active":"2024-04-25T10:30:31.404775+0000","last_became_peered":"2024-04-25T10:30:31.404775+0000","last_unstale":"2024-04-25T10:30:37.436320+0000","last_undegraded":"2024-04-25T10:30:37.436320+0000","last_fullsized":"2024-04-25T10:30:37.436320+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:10:21.595653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794446+0000","last_change":"2024-04-25T10:30:32.432061+0000","last_active":"2024-04-25T10:30:34.794446+0000","last_peered":"2024-04-25T10:30:34.794446+0000","last_clean":"2024-04-25T10:30:34.794446+0000","last_became_active":"2024-04-25T10:30:32.431651+0000","last_became_peered":"2024-04-25T10:30:32.431651+0000","last_unstale":"2024-04-25T10:30:34.794446+0000","last_undegraded":"2024-04-25T10:30:34.794446+0000","last_fullsized":"2024-04-25T10:30:34.794446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:39.062490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.122181+0000","last_change":"2024-04-25T10:30:31.420258+0000","last_active":"2024-04-25T10:30:35.122181+0000","last_peered":"2024-04-25T10:30:35.122181+0000","last_clean":"2024-04-25T10:30:35.122181+0000","last_became_active":"2024-04-25T10:30:31.419902+0000","last_became_peered":"2024-04-25T10:30:31.419902+0000","last_unstale":"2024-04-25T10:30:35.122181+0000","last_undegraded":"2024-04-25T10:30:35.122181+0000","last_fullsized":"2024-04-25T10:30:35.122181+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:43.919867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795443+0000","last_change":"2024-04-25T10:30:32.433568+0000","last_active":"2024-04-25T10:30:34.795443+0000","last_peered":"2024-04-25T10:30:34.795443+0000","last_clean":"2024-04-25T10:30:34.795443+0000","last_became_active":"2024-04-25T10:30:32.432572+0000","last_became_peered":"2024-04-25T10:30:32.432572+0000","last_unstale":"2024-04-25T10:30:34.795443+0000","last_undegraded":"2024-04-25T10:30:34.795443+0000","last_fullsized":"2024-04-25T10:30:34.795443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:31.771919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:37.435148+0000","last_change":"2024-04-25T10:30:31.419308+0000","last_active":"2024-04-25T10:30:37.435148+0000","last_peered":"2024-04-25T10:30:37.435148+0000","last_clean":"2024-04-25T10:30:37.435148+0000","last_became_active":"2024-04-25T10:30:31.418978+0000","last_became_peered":"2024-04-25T10:30:31.418978+0000","last_unstale":"2024-04-25T10:30:37.435148+0000","last_undegraded":"2024-04-25T10:30:37.435148+0000","last_fullsized":"2024-04-25T10:30:37.435148+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:33.784804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795670+0000","last_change":"2024-04-25T10:30:32.436109+0000","last_active":"2024-04-25T10:30:34.795670+0000","last_peered":"2024-04-25T10:30:34.795670+0000","last_clean":"2024-04-25T10:30:34.795670+0000","last_became_active":"2024-04-25T10:30:32.434701+0000","last_became_peered":"2024-04-25T10:30:32.434701+0000","last_unstale":"2024-04-25T10:30:34.795670+0000","last_undegraded":"2024-04-25T10:30:34.795670+0000","last_fullsized":"2024-04-25T10:30:34.795670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:19.471867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.119322+0000","last_change":"2024-04-25T10:30:31.416044+0000","last_active":"2024-04-25T10:30:35.119322+0000","last_peered":"2024-04-25T10:30:35.119322+0000","last_clean":"2024-04-25T10:30:35.119322+0000","last_became_active":"2024-04-25T10:30:31.415707+0000","last_became_peered":"2024-04-25T10:30:31.415707+0000","last_unstale":"2024-04-25T10:30:35.119322+0000","last_undegraded":"2024-04-25T10:30:35.119322+0000","last_fullsized":"2024-04-25T10:30:35.119322+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:31.424135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795321+0000","last_change":"2024-04-25T10:30:32.442971+0000","last_active":"2024-04-25T10:30:34.795321+0000","last_peered":"2024-04-25T10:30:34.795321+0000","last_clean":"2024-04-25T10:30:34.795321+0000","last_became_active":"2024-04-25T10:30:32.442656+0000","last_became_peered":"2024-04-25T10:30:32.442656+0000","last_unstale":"2024-04-25T10:30:34.795321+0000","last_undegraded":"2024-04-25T10:30:34.795321+0000","last_fullsized":"2024-04-25T10:30:34.795321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:29.951142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415622+0000","last_change":"2024-04-25T10:30:31.404793+0000","last_active":"2024-04-25T10:30:33.415622+0000","last_peered":"2024-04-25T10:30:33.415622+0000","last_clean":"2024-04-25T10:30:33.415622+0000","last_became_active":"2024-04-25T10:30:31.404392+0000","last_became_peered":"2024-04-25T10:30:31.404392+0000","last_unstale":"2024-04-25T10:30:33.415622+0000","last_undegraded":"2024-04-25T10:30:33.415622+0000","last_fullsized":"2024-04-25T10:30:33.415622+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:00.965450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795922+0000","last_change":"2024-04-25T10:30:32.441438+0000","last_active":"2024-04-25T10:30:34.795922+0000","last_peered":"2024-04-25T10:30:34.795922+0000","last_clean":"2024-04-25T10:30:34.795922+0000","last_became_active":"2024-04-25T10:30:32.440011+0000","last_became_peered":"2024-04-25T10:30:32.440011+0000","last_unstale":"2024-04-25T10:30:34.795922+0000","last_undegraded":"2024-04-25T10:30:34.795922+0000","last_fullsized":"2024-04-25T10:30:34.795922+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:09.127517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":110,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.793865+0000","last_change":"2024-04-25T10:30:21.429098+0000","last_active":"2024-04-25T10:30:34.793865+0000","last_peered":"2024-04-25T10:30:34.793865+0000","last_clean":"2024-04-25T10:30:34.793865+0000","last_became_active":"2024-04-25T10:30:21.427958+0000","last_became_peered":"2024-04-25T10:30:21.427958+0000","last_unstale":"2024-04-25T10:30:34.793865+0000","last_undegraded":"2024-04-25T10:30:34.793865+0000","last_fullsized":"2024-04-25T10:30:34.793865+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:20.186206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:20.186206+0000","last_clean_scrub_stamp":"2024-04-25T10:30:20.186206+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:46.046488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.793946+0000","last_change":"2024-04-25T10:30:31.404974+0000","last_active":"2024-04-25T10:30:34.793946+0000","last_peered":"2024-04-25T10:30:34.793946+0000","last_clean":"2024-04-25T10:30:34.793946+0000","last_became_active":"2024-04-25T10:30:31.404549+0000","last_became_peered":"2024-04-25T10:30:31.404549+0000","last_unstale":"2024-04-25T10:30:34.793946+0000","last_undegraded":"2024-04-25T10:30:34.793946+0000","last_fullsized":"2024-04-25T10:30:34.793946+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:06.191321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415412+0000","last_change":"2024-04-25T10:30:32.434409+0000","last_active":"2024-04-25T10:30:33.415412+0000","last_peered":"2024-04-25T10:30:33.415412+0000","last_clean":"2024-04-25T10:30:33.415412+0000","last_became_active":"2024-04-25T10:30:32.433830+0000","last_became_peered":"2024-04-25T10:30:32.433830+0000","last_unstale":"2024-04-25T10:30:33.415412+0000","last_undegraded":"2024-04-25T10:30:33.415412+0000","last_fullsized":"2024-04-25T10:30:33.415412+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:57.864627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.793793+0000","last_change":"2024-04-25T10:30:31.416025+0000","last_active":"2024-04-25T10:30:34.793793+0000","last_peered":"2024-04-25T10:30:34.793793+0000","last_clean":"2024-04-25T10:30:34.793793+0000","last_became_active":"2024-04-25T10:30:31.415675+0000","last_became_peered":"2024-04-25T10:30:31.415675+0000","last_unstale":"2024-04-25T10:30:34.793793+0000","last_undegraded":"2024-04-25T10:30:34.793793+0000","last_fullsized":"2024-04-25T10:30:34.793793+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:28:48.185715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.196505+0000","last_change":"2024-04-25T10:30:32.424630+0000","last_active":"2024-04-25T10:30:34.196505+0000","last_peered":"2024-04-25T10:30:34.196505+0000","last_clean":"2024-04-25T10:30:34.196505+0000","last_became_active":"2024-04-25T10:30:32.424303+0000","last_became_peered":"2024-04-25T10:30:32.424303+0000","last_unstale":"2024-04-25T10:30:34.196505+0000","last_undegraded":"2024-04-25T10:30:34.196505+0000","last_fullsized":"2024-04-25T10:30:34.196505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:55.535319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415284+0000","last_change":"2024-04-25T10:30:31.419145+0000","last_active":"2024-04-25T10:30:33.415284+0000","last_peered":"2024-04-25T10:30:33.415284+0000","last_clean":"2024-04-25T10:30:33.415284+0000","last_became_active":"2024-04-25T10:30:31.418817+0000","last_became_peered":"2024-04-25T10:30:31.418817+0000","last_unstale":"2024-04-25T10:30:33.415284+0000","last_undegraded":"2024-04-25T10:30:33.415284+0000","last_fullsized":"2024-04-25T10:30:33.415284+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:01.608205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.412646+0000","last_change":"2024-04-25T10:30:32.419911+0000","last_active":"2024-04-25T10:30:33.412646+0000","last_peered":"2024-04-25T10:30:33.412646+0000","last_clean":"2024-04-25T10:30:33.412646+0000","last_became_active":"2024-04-25T10:30:32.419486+0000","last_became_peered":"2024-04-25T10:30:32.419486+0000","last_unstale":"2024-04-25T10:30:33.412646+0000","last_undegraded":"2024-04-25T10:30:33.412646+0000","last_fullsized":"2024-04-25T10:30:33.412646+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:54.967873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.796060+0000","last_change":"2024-04-25T10:30:31.405646+0000","last_active":"2024-04-25T10:30:34.796060+0000","last_peered":"2024-04-25T10:30:34.796060+0000","last_clean":"2024-04-25T10:30:34.796060+0000","last_became_active":"2024-04-25T10:30:31.405238+0000","last_became_peered":"2024-04-25T10:30:31.405238+0000","last_unstale":"2024-04-25T10:30:34.796060+0000","last_undegraded":"2024-04-25T10:30:34.796060+0000","last_fullsized":"2024-04-25T10:30:34.796060+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:13.159787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795730+0000","last_change":"2024-04-25T10:30:32.426130+0000","last_active":"2024-04-25T10:30:34.795730+0000","last_peered":"2024-04-25T10:30:34.795730+0000","last_clean":"2024-04-25T10:30:34.795730+0000","last_became_active":"2024-04-25T10:30:32.425783+0000","last_became_peered":"2024-04-25T10:30:32.425783+0000","last_unstale":"2024-04-25T10:30:34.795730+0000","last_undegraded":"2024-04-25T10:30:34.795730+0000","last_fullsized":"2024-04-25T10:30:34.795730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:30.844640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.412747+0000","last_change":"2024-04-25T10:30:31.412736+0000","last_active":"2024-04-25T10:30:33.412747+0000","last_peered":"2024-04-25T10:30:33.412747+0000","last_clean":"2024-04-25T10:30:33.412747+0000","last_became_active":"2024-04-25T10:30:31.412413+0000","last_became_peered":"2024-04-25T10:30:31.412413+0000","last_unstale":"2024-04-25T10:30:33.412747+0000","last_undegraded":"2024-04-25T10:30:33.412747+0000","last_fullsized":"2024-04-25T10:30:33.412747+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:19:56.236327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795688+0000","last_change":"2024-04-25T10:30:32.440858+0000","last_active":"2024-04-25T10:30:34.795688+0000","last_peered":"2024-04-25T10:30:34.795688+0000","last_clean":"2024-04-25T10:30:34.795688+0000","last_became_active":"2024-04-25T10:30:32.440519+0000","last_became_peered":"2024-04-25T10:30:32.440519+0000","last_unstale":"2024-04-25T10:30:34.795688+0000","last_undegraded":"2024-04-25T10:30:34.795688+0000","last_fullsized":"2024-04-25T10:30:34.795688+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:21.288748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.796138+0000","last_change":"2024-04-25T10:30:31.417958+0000","last_active":"2024-04-25T10:30:34.796138+0000","last_peered":"2024-04-25T10:30:34.796138+0000","last_clean":"2024-04-25T10:30:34.796138+0000","last_became_active":"2024-04-25T10:30:31.417541+0000","last_became_peered":"2024-04-25T10:30:31.417541+0000","last_unstale":"2024-04-25T10:30:34.796138+0000","last_undegraded":"2024-04-25T10:30:34.796138+0000","last_fullsized":"2024-04-25T10:30:34.796138+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:35.373351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.412553+0000","last_change":"2024-04-25T10:30:32.420451+0000","last_active":"2024-04-25T10:30:33.412553+0000","last_peered":"2024-04-25T10:30:33.412553+0000","last_clean":"2024-04-25T10:30:33.412553+0000","last_became_active":"2024-04-25T10:30:32.420109+0000","last_became_peered":"2024-04-25T10:30:32.420109+0000","last_unstale":"2024-04-25T10:30:33.412553+0000","last_undegraded":"2024-04-25T10:30:33.412553+0000","last_fullsized":"2024-04-25T10:30:33.412553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:22.067828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.412601+0000","last_change":"2024-04-25T10:30:31.406005+0000","last_active":"2024-04-25T10:30:33.412601+0000","last_peered":"2024-04-25T10:30:33.412601+0000","last_clean":"2024-04-25T10:30:33.412601+0000","last_became_active":"2024-04-25T10:30:31.404739+0000","last_became_peered":"2024-04-25T10:30:31.404739+0000","last_unstale":"2024-04-25T10:30:33.412601+0000","last_undegraded":"2024-04-25T10:30:33.412601+0000","last_fullsized":"2024-04-25T10:30:33.412601+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:31.771123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415711+0000","last_change":"2024-04-25T10:30:32.437513+0000","last_active":"2024-04-25T10:30:33.415711+0000","last_peered":"2024-04-25T10:30:33.415711+0000","last_clean":"2024-04-25T10:30:33.415711+0000","last_became_active":"2024-04-25T10:30:32.437171+0000","last_became_peered":"2024-04-25T10:30:32.437171+0000","last_unstale":"2024-04-25T10:30:33.415711+0000","last_undegraded":"2024-04-25T10:30:33.415711+0000","last_fullsized":"2024-04-25T10:30:33.415711+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:50.924843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:37.436706+0000","last_change":"2024-04-25T10:30:31.413140+0000","last_active":"2024-04-25T10:30:37.436706+0000","last_peered":"2024-04-25T10:30:37.436706+0000","last_clean":"2024-04-25T10:30:37.436706+0000","last_became_active":"2024-04-25T10:30:31.412821+0000","last_became_peered":"2024-04-25T10:30:31.412821+0000","last_unstale":"2024-04-25T10:30:37.436706+0000","last_undegraded":"2024-04-25T10:30:37.436706+0000","last_fullsized":"2024-04-25T10:30:37.436706+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:37.177254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.196799+0000","last_change":"2024-04-25T10:30:32.434333+0000","last_active":"2024-04-25T10:30:34.196799+0000","last_peered":"2024-04-25T10:30:34.196799+0000","last_clean":"2024-04-25T10:30:34.196799+0000","last_became_active":"2024-04-25T10:30:32.434000+0000","last_became_peered":"2024-04-25T10:30:32.434000+0000","last_unstale":"2024-04-25T10:30:34.196799+0000","last_undegraded":"2024-04-25T10:30:34.196799+0000","last_fullsized":"2024-04-25T10:30:34.196799+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:54.057457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794573+0000","last_change":"2024-04-25T10:30:31.419475+0000","last_active":"2024-04-25T10:30:34.794573+0000","last_peered":"2024-04-25T10:30:34.794573+0000","last_clean":"2024-04-25T10:30:34.794573+0000","last_became_active":"2024-04-25T10:30:31.417605+0000","last_became_peered":"2024-04-25T10:30:31.417605+0000","last_unstale":"2024-04-25T10:30:34.794573+0000","last_undegraded":"2024-04-25T10:30:34.794573+0000","last_fullsized":"2024-04-25T10:30:34.794573+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:00.948269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794414+0000","last_change":"2024-04-25T10:30:32.419693+0000","last_active":"2024-04-25T10:30:34.794414+0000","last_peered":"2024-04-25T10:30:34.794414+0000","last_clean":"2024-04-25T10:30:34.794414+0000","last_became_active":"2024-04-25T10:30:32.419370+0000","last_became_peered":"2024-04-25T10:30:32.419370+0000","last_unstale":"2024-04-25T10:30:34.794414+0000","last_undegraded":"2024-04-25T10:30:34.794414+0000","last_fullsized":"2024-04-25T10:30:34.794414+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:31.199065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.412581+0000","last_change":"2024-04-25T10:30:31.410767+0000","last_active":"2024-04-25T10:30:33.412581+0000","last_peered":"2024-04-25T10:30:33.412581+0000","last_clean":"2024-04-25T10:30:33.412581+0000","last_became_active":"2024-04-25T10:30:31.410404+0000","last_became_peered":"2024-04-25T10:30:31.410404+0000","last_unstale":"2024-04-25T10:30:33.412581+0000","last_undegraded":"2024-04-25T10:30:33.412581+0000","last_fullsized":"2024-04-25T10:30:33.412581+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:37.431532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415585+0000","last_change":"2024-04-25T10:30:32.433687+0000","last_active":"2024-04-25T10:30:33.415585+0000","last_peered":"2024-04-25T10:30:33.415585+0000","last_clean":"2024-04-25T10:30:33.415585+0000","last_became_active":"2024-04-25T10:30:32.433389+0000","last_became_peered":"2024-04-25T10:30:32.433389+0000","last_unstale":"2024-04-25T10:30:33.415585+0000","last_undegraded":"2024-04-25T10:30:33.415585+0000","last_fullsized":"2024-04-25T10:30:33.415585+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:39.155317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.412506+0000","last_change":"2024-04-25T10:30:31.419118+0000","last_active":"2024-04-25T10:30:33.412506+0000","last_peered":"2024-04-25T10:30:33.412506+0000","last_clean":"2024-04-25T10:30:33.412506+0000","last_became_active":"2024-04-25T10:30:31.418822+0000","last_became_peered":"2024-04-25T10:30:31.418822+0000","last_unstale":"2024-04-25T10:30:33.412506+0000","last_undegraded":"2024-04-25T10:30:33.412506+0000","last_fullsized":"2024-04-25T10:30:33.412506+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:22.755042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415890+0000","last_change":"2024-04-25T10:30:32.427894+0000","last_active":"2024-04-25T10:30:33.415890+0000","last_peered":"2024-04-25T10:30:33.415890+0000","last_clean":"2024-04-25T10:30:33.415890+0000","last_became_active":"2024-04-25T10:30:32.427504+0000","last_became_peered":"2024-04-25T10:30:32.427504+0000","last_unstale":"2024-04-25T10:30:33.415890+0000","last_undegraded":"2024-04-25T10:30:33.415890+0000","last_fullsized":"2024-04-25T10:30:33.415890+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:53.877484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:37.436274+0000","last_change":"2024-04-25T10:30:31.417115+0000","last_active":"2024-04-25T10:30:37.436274+0000","last_peered":"2024-04-25T10:30:37.436274+0000","last_clean":"2024-04-25T10:30:37.436274+0000","last_became_active":"2024-04-25T10:30:31.416766+0000","last_became_peered":"2024-04-25T10:30:31.416766+0000","last_unstale":"2024-04-25T10:30:37.436274+0000","last_undegraded":"2024-04-25T10:30:37.436274+0000","last_fullsized":"2024-04-25T10:30:37.436274+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:06.021270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795407+0000","last_change":"2024-04-25T10:30:32.431674+0000","last_active":"2024-04-25T10:30:34.795407+0000","last_peered":"2024-04-25T10:30:34.795407+0000","last_clean":"2024-04-25T10:30:34.795407+0000","last_became_active":"2024-04-25T10:30:32.431395+0000","last_became_peered":"2024-04-25T10:30:32.431395+0000","last_unstale":"2024-04-25T10:30:34.795407+0000","last_undegraded":"2024-04-25T10:30:34.795407+0000","last_fullsized":"2024-04-25T10:30:34.795407+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:49.726323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795508+0000","last_change":"2024-04-25T10:30:32.437700+0000","last_active":"2024-04-25T10:30:34.795508+0000","last_peered":"2024-04-25T10:30:34.795508+0000","last_clean":"2024-04-25T10:30:34.795508+0000","last_became_active":"2024-04-25T10:30:32.437242+0000","last_became_peered":"2024-04-25T10:30:32.437242+0000","last_unstale":"2024-04-25T10:30:34.795508+0000","last_undegraded":"2024-04-25T10:30:34.795508+0000","last_fullsized":"2024-04-25T10:30:34.795508+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:06.499414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:37.435895+0000","last_change":"2024-04-25T10:30:31.419678+0000","last_active":"2024-04-25T10:30:37.435895+0000","last_peered":"2024-04-25T10:30:37.435895+0000","last_clean":"2024-04-25T10:30:37.435895+0000","last_became_active":"2024-04-25T10:30:31.419269+0000","last_became_peered":"2024-04-25T10:30:31.419269+0000","last_unstale":"2024-04-25T10:30:37.435895+0000","last_undegraded":"2024-04-25T10:30:37.435895+0000","last_fullsized":"2024-04-25T10:30:37.435895+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:33.034260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795138+0000","last_change":"2024-04-25T10:30:32.434410+0000","last_active":"2024-04-25T10:30:34.795138+0000","last_peered":"2024-04-25T10:30:34.795138+0000","last_clean":"2024-04-25T10:30:34.795138+0000","last_became_active":"2024-04-25T10:30:32.434093+0000","last_became_peered":"2024-04-25T10:30:32.434093+0000","last_unstale":"2024-04-25T10:30:34.795138+0000","last_undegraded":"2024-04-25T10:30:34.795138+0000","last_fullsized":"2024-04-25T10:30:34.795138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:46.964704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:37.434829+0000","last_change":"2024-04-25T10:30:31.418360+0000","last_active":"2024-04-25T10:30:37.434829+0000","last_peered":"2024-04-25T10:30:37.434829+0000","last_clean":"2024-04-25T10:30:37.434829+0000","last_became_active":"2024-04-25T10:30:31.418005+0000","last_became_peered":"2024-04-25T10:30:31.418005+0000","last_unstale":"2024-04-25T10:30:37.434829+0000","last_undegraded":"2024-04-25T10:30:37.434829+0000","last_fullsized":"2024-04-25T10:30:37.434829+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:15.556859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.196705+0000","last_change":"2024-04-25T10:30:32.439515+0000","last_active":"2024-04-25T10:30:34.196705+0000","last_peered":"2024-04-25T10:30:34.196705+0000","last_clean":"2024-04-25T10:30:34.196705+0000","last_became_active":"2024-04-25T10:30:32.439101+0000","last_became_peered":"2024-04-25T10:30:32.439101+0000","last_unstale":"2024-04-25T10:30:34.196705+0000","last_undegraded":"2024-04-25T10:30:34.196705+0000","last_fullsized":"2024-04-25T10:30:34.196705+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:33.797903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.120347+0000","last_change":"2024-04-25T10:30:31.418070+0000","last_active":"2024-04-25T10:30:35.120347+0000","last_peered":"2024-04-25T10:30:35.120347+0000","last_clean":"2024-04-25T10:30:35.120347+0000","last_became_active":"2024-04-25T10:30:31.417737+0000","last_became_peered":"2024-04-25T10:30:31.417737+0000","last_unstale":"2024-04-25T10:30:35.120347+0000","last_undegraded":"2024-04-25T10:30:35.120347+0000","last_fullsized":"2024-04-25T10:30:35.120347+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:05.664855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415785+0000","last_change":"2024-04-25T10:30:32.435258+0000","last_active":"2024-04-25T10:30:33.415785+0000","last_peered":"2024-04-25T10:30:33.415785+0000","last_clean":"2024-04-25T10:30:33.415785+0000","last_became_active":"2024-04-25T10:30:32.434930+0000","last_became_peered":"2024-04-25T10:30:32.434930+0000","last_unstale":"2024-04-25T10:30:33.415785+0000","last_undegraded":"2024-04-25T10:30:33.415785+0000","last_fullsized":"2024-04-25T10:30:33.415785+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:22.727778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:37.436552+0000","last_change":"2024-04-25T10:30:31.415611+0000","last_active":"2024-04-25T10:30:37.436552+0000","last_peered":"2024-04-25T10:30:37.436552+0000","last_clean":"2024-04-25T10:30:37.436552+0000","last_became_active":"2024-04-25T10:30:31.415071+0000","last_became_peered":"2024-04-25T10:30:31.415071+0000","last_unstale":"2024-04-25T10:30:37.436552+0000","last_undegraded":"2024-04-25T10:30:37.436552+0000","last_fullsized":"2024-04-25T10:30:37.436552+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:30:54.077864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27800,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344040,"statfs":{"total":96636764160,"available":96608296960,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:30:42.347 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T10:30:42.347 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T10:30:42.347 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-25T10:30:42.347 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T10:30:42.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.486+0000 7f9fd45b2640 1 Processor -- start 2024-04-25T10:30:42.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.486+0000 7f9fd45b2640 1 -- start start 2024-04-25T10:30:42.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.488+0000 7f9fd45b2640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9fcc0c8d90 0x7f9fcc156ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:42.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.488+0000 7f9fd45b2640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f9fcc0b4ce0 0x7f9fcc15ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:42.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.488+0000 7f9fd45b2640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fcc0ce630 con 0x7f9fcc0c7e30 2024-04-25T10:30:42.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.488+0000 7f9fd45b2640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f9fcc0b50f0 con 0x7f9fcc0b4ce0 2024-04-25T10:30:42.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.488+0000 7f9fd45b2640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f9fcc0c7c60 con 0x7f9fcc0c8d90 2024-04-25T10:30:42.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.488+0000 7f9fd2b4f640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f9fcc0b4ce0 0x7f9fcc15ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:42.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.488+0000 7f9fd1b4d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9fcc0c8d90 0x7f9fcc156ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:42.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.488+0000 7f9fd2b4f640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f9fcc0b4ce0 0x7f9fcc15ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.106:48180/0 (socket says 172.21.15.106:48180) 2024-04-25T10:30:42.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.488+0000 7f9fd2b4f640 1 -- 172.21.15.106:0/3396762480 learned_addr learned my addr 172.21.15.106:0/3396762480 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:42.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.488+0000 7f9fd234e640 1 --1- 172.21.15.106:0/3396762480 >> v1:172.21.15.134:6789/0 conn(0x7f9fcc0c7e30 0x7f9fcc06d000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.106:41260/0 (socket says 172.21.15.106:41260) 2024-04-25T10:30:42.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.488+0000 7f9fd134c640 1 -- 172.21.15.106:0/3396762480 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2936760432 0 0) 0x7f9fcc0ce630 con 0x7f9fcc0c7e30 2024-04-25T10:30:42.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.488+0000 7f9fd134c640 1 -- 172.21.15.106:0/3396762480 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9fb8003520 con 0x7f9fcc0c7e30 2024-04-25T10:30:42.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.488+0000 7f9fd1b4d640 1 -- 172.21.15.106:0/3396762480 >> v2:172.21.15.134:3300/0 conn(0x7f9fcc0b4ce0 msgr2=0x7f9fcc15ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:42.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.488+0000 7f9fd1b4d640 1 --2- 172.21.15.106:0/3396762480 >> v2:172.21.15.134:3300/0 conn(0x7f9fcc0b4ce0 0x7f9fcc15ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:42.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.488+0000 7f9fd1b4d640 1 -- 172.21.15.106:0/3396762480 >> v1:172.21.15.134:6789/0 conn(0x7f9fcc0c7e30 legacy=0x7f9fcc06d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:42.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.488+0000 7f9fd1b4d640 1 -- 172.21.15.106:0/3396762480 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fcc0c7980 con 0x7f9fcc0c8d90 2024-04-25T10:30:42.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.488+0000 7f9fd2b4f640 1 --2- 172.21.15.106:0/3396762480 >> v2:172.21.15.134:3300/0 conn(0x7f9fcc0b4ce0 0x7f9fcc15ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:42.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.489+0000 7f9fd1b4d640 1 --2- 172.21.15.106:0/3396762480 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9fcc0c8d90 0x7f9fcc156ef0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc0c7de0 tx=0x7f9fbc02a0d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b6919e194a133a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:42.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.489+0000 7f9fd134c640 1 -- 172.21.15.106:0/3396762480 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fbc037070 con 0x7f9fcc0c8d90 2024-04-25T10:30:42.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.489+0000 7f9fd134c640 1 -- 172.21.15.106:0/3396762480 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fbc02ae20 con 0x7f9fcc0c8d90 2024-04-25T10:30:42.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.489+0000 7f9fd134c640 1 -- 172.21.15.106:0/3396762480 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fbc032660 con 0x7f9fcc0c8d90 2024-04-25T10:30:42.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.490+0000 7f9fd45b2640 1 -- 172.21.15.106:0/3396762480 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9fcc0c8d90 msgr2=0x7f9fcc156ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:42.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.490+0000 7f9fd45b2640 1 --2- 172.21.15.106:0/3396762480 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9fcc0c8d90 0x7f9fcc156ef0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc0c7de0 tx=0x7f9fbc02a0d0 comp rx=0 tx=0).stop 2024-04-25T10:30:42.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.490+0000 7f9fd45b2640 1 -- 172.21.15.106:0/3396762480 shutdown_connections 2024-04-25T10:30:42.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.490+0000 7f9fd45b2640 1 --2- 172.21.15.106:0/3396762480 >> v2:172.21.15.134:3300/0 conn(0x7f9fcc0b4ce0 0x7f9fcc15ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:42.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.490+0000 7f9fd45b2640 1 --2- 172.21.15.106:0/3396762480 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9fcc0c8d90 0x7f9fcc156ef0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:42.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.490+0000 7f9fd45b2640 1 -- 172.21.15.106:0/3396762480 >> 172.21.15.106:0/3396762480 conn(0x7f9fcc068b10 msgr2=0x7f9fcc0d1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:42.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.490+0000 7f9fd45b2640 1 -- 172.21.15.106:0/3396762480 shutdown_connections 2024-04-25T10:30:42.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.490+0000 7f9fd45b2640 1 -- 172.21.15.106:0/3396762480 wait complete. 2024-04-25T10:30:42.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.492+0000 7f9fd45b2640 1 Processor -- start 2024-04-25T10:30:42.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.492+0000 7f9fd45b2640 1 -- start start 2024-04-25T10:30:42.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.493+0000 7f9fd45b2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9fcc0b4ce0 0x7f9fcc1514d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:42.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.493+0000 7f9fd45b2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9fcc0c7e30 0x7f9fcc160a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:42.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.493+0000 7f9fd45b2640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9fcc0c8d90 0x7f9fcc1506d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:42.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.493+0000 7f9fd45b2640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9fcc06d710 con 0x7f9fcc0b4ce0 2024-04-25T10:30:42.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.493+0000 7f9fd45b2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9fcc0c72f0 con 0x7f9fcc0c7e30 2024-04-25T10:30:42.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.493+0000 7f9fd45b2640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f9fcc06d890 con 0x7f9fcc0c8d90 2024-04-25T10:30:42.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.493+0000 7f9fd234e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9fcc0b4ce0 0x7f9fcc1514d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:42.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.493+0000 7f9fd234e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9fcc0b4ce0 0x7f9fcc1514d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48010/0 (socket says 172.21.15.106:48010) 2024-04-25T10:30:42.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.493+0000 7f9fd234e640 1 -- 172.21.15.106:0/3486225273 learned_addr learned my addr 172.21.15.106:0/3486225273 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:42.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.493+0000 7f9fd1b4d640 1 --2- 172.21.15.106:0/3486225273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9fcc0c7e30 0x7f9fcc160a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:42.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.493+0000 7f9fd2b4f640 1 --2- 172.21.15.106:0/3486225273 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9fcc0c8d90 0x7f9fcc1506d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:42.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.493+0000 7f9fd234e640 1 -- 172.21.15.106:0/3486225273 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9fcc0c8d90 msgr2=0x7f9fcc1506d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:42.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.493+0000 7f9fd234e640 1 --2- 172.21.15.106:0/3486225273 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9fcc0c8d90 0x7f9fcc1506d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:42.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.493+0000 7f9fd234e640 1 -- 172.21.15.106:0/3486225273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9fcc0c7e30 msgr2=0x7f9fcc160a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:42.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.493+0000 7f9fd234e640 1 --2- 172.21.15.106:0/3486225273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9fcc0c7e30 0x7f9fcc160a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:42.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.493+0000 7f9fd234e640 1 -- 172.21.15.106:0/3486225273 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fcc152a20 con 0x7f9fcc0b4ce0 2024-04-25T10:30:42.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.494+0000 7f9fd234e640 1 --2- 172.21.15.106:0/3486225273 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9fcc0b4ce0 0x7f9fcc1514d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9fc800c700 tx=0x7f9fc800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:42.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.494+0000 7f9fc2ffd640 1 -- 172.21.15.106:0/3486225273 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc8017070 con 0x7f9fcc0b4ce0 2024-04-25T10:30:42.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.494+0000 7f9fc2ffd640 1 -- 172.21.15.106:0/3486225273 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fc800ce10 con 0x7f9fcc0b4ce0 2024-04-25T10:30:42.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.494+0000 7f9fd45b2640 1 -- 172.21.15.106:0/3486225273 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fcc152cc0 con 0x7f9fcc0b4ce0 2024-04-25T10:30:42.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.494+0000 7f9fc2ffd640 1 -- 172.21.15.106:0/3486225273 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc800a860 con 0x7f9fcc0b4ce0 2024-04-25T10:30:42.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.494+0000 7f9fd45b2640 1 -- 172.21.15.106:0/3486225273 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fcc153280 con 0x7f9fcc0b4ce0 2024-04-25T10:30:42.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.495+0000 7f9fd45b2640 1 -- 172.21.15.106:0/3486225273 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fcc06a3d0 con 0x7f9fcc0b4ce0 2024-04-25T10:30:42.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.495+0000 7f9fc2ffd640 1 -- 172.21.15.106:0/3486225273 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9fc801a070 con 0x7f9fcc0b4ce0 2024-04-25T10:30:42.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.496+0000 7f9fc2ffd640 1 --2- 172.21.15.106:0/3486225273 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f9fb00744e0 0x7f9fb00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:42.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.496+0000 7f9fc2ffd640 1 -- 172.21.15.106:0/3486225273 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9fc80947d0 con 0x7f9fcc0b4ce0 2024-04-25T10:30:42.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.496+0000 7f9fd1b4d640 1 --2- 172.21.15.106:0/3486225273 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f9fb00744e0 0x7f9fb00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:42.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.496+0000 7f9fd1b4d640 1 --2- 172.21.15.106:0/3486225273 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f9fb00744e0 0x7f9fb00769f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc034000 tx=0x7f9fbc02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:42.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.500+0000 7f9fc2ffd640 1 -- 172.21.15.106:0/3486225273 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9fc805e910 con 0x7f9fcc0b4ce0 2024-04-25T10:30:42.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.711+0000 7f9fd45b2640 1 -- 172.21.15.106:0/3486225273 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f9fcc15a5a0 con 0x7f9fcc0b4ce0 2024-04-25T10:30:42.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.712+0000 7f9fc2ffd640 1 -- 172.21.15.106:0/3486225273 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+196 (secure 0 0 0) 0x7f9fc8062c10 con 0x7f9fcc0b4ce0 2024-04-25T10:30:42.715 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:30:42.715 INFO:teuthology.orchestra.run.smithi106.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2024-04-25T10:30:42.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.714+0000 7f9fd45b2640 1 -- 172.21.15.106:0/3486225273 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f9fb00744e0 msgr2=0x7f9fb00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:42.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.714+0000 7f9fd45b2640 1 --2- 172.21.15.106:0/3486225273 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f9fb00744e0 0x7f9fb00769f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc034000 tx=0x7f9fbc02e000 comp rx=0 tx=0).stop 2024-04-25T10:30:42.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.714+0000 7f9fd45b2640 1 -- 172.21.15.106:0/3486225273 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9fcc0b4ce0 msgr2=0x7f9fcc1514d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:42.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.714+0000 7f9fd45b2640 1 --2- 172.21.15.106:0/3486225273 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9fcc0b4ce0 0x7f9fcc1514d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9fc800c700 tx=0x7f9fc800cbc0 comp rx=0 tx=0).stop 2024-04-25T10:30:42.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.714+0000 7f9fd45b2640 1 -- 172.21.15.106:0/3486225273 shutdown_connections 2024-04-25T10:30:42.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.714+0000 7f9fd45b2640 1 --2- 172.21.15.106:0/3486225273 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f9fb00744e0 0x7f9fb00769f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:42.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.714+0000 7f9fd45b2640 1 --2- 172.21.15.106:0/3486225273 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9fcc0c8d90 0x7f9fcc1506d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:42.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.714+0000 7f9fd45b2640 1 --2- 172.21.15.106:0/3486225273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9fcc0c7e30 0x7f9fcc160a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:42.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.714+0000 7f9fd45b2640 1 --2- 172.21.15.106:0/3486225273 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9fcc0b4ce0 0x7f9fcc1514d0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:42.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.714+0000 7f9fd45b2640 1 -- 172.21.15.106:0/3486225273 >> 172.21.15.106:0/3486225273 conn(0x7f9fcc068b10 msgr2=0x7f9fcc15c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:42.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.714+0000 7f9fd45b2640 1 -- 172.21.15.106:0/3486225273 shutdown_connections 2024-04-25T10:30:42.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:42.714+0000 7f9fd45b2640 1 -- 172.21.15.106:0/3486225273 wait complete. 2024-04-25T10:30:45.728 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T10:30:45.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:45.844+0000 7f2739634640 1 Processor -- start 2024-04-25T10:30:45.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:45.844+0000 7f2739634640 1 -- start start 2024-04-25T10:30:45.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:45.845+0000 7f2739634640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f2734154c60 0x7f2734070ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:45.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:45.845+0000 7f2739634640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2734071220 0x7f27340c5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:45.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:45.845+0000 7f2739634640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f27340c5940 0x7f273415aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:45.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:45.845+0000 7f2739634640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f27340cb880 con 0x7f27340c5940 2024-04-25T10:30:45.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:45.845+0000 7f2739634640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f27340caa00 con 0x7f2734154c60 2024-04-25T10:30:45.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:45.845+0000 7f2739634640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f27340cac70 con 0x7f2734071220 2024-04-25T10:30:45.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:45.845+0000 7f27337fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f27340c5940 0x7f273415aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:45.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:45.845+0000 7f27337fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f27340c5940 0x7f273415aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55442/0 (socket says 172.21.15.106:55442) 2024-04-25T10:30:45.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:45.845+0000 7f27337fe640 1 -- 172.21.15.106:0/4188640847 learned_addr learned my addr 172.21.15.106:0/4188640847 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:45.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:45.845+0000 7f2732ffd640 1 --2- 172.21.15.106:0/4188640847 >> v2:172.21.15.134:3300/0 conn(0x7f2734154c60 0x7f2734070ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:45.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:45.845+0000 7f272bfff640 1 --2- 172.21.15.106:0/4188640847 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2734071220 0x7f27340c5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:45.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:45.846+0000 7f27337fe640 1 -- 172.21.15.106:0/4188640847 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2734071220 msgr2=0x7f27340c5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:45.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:45.846+0000 7f27337fe640 1 --2- 172.21.15.106:0/4188640847 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2734071220 0x7f27340c5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:45.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:45.846+0000 7f27337fe640 1 -- 172.21.15.106:0/4188640847 >> v2:172.21.15.134:3300/0 conn(0x7f2734154c60 msgr2=0x7f2734070ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:45.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:45.846+0000 7f27337fe640 1 --2- 172.21.15.106:0/4188640847 >> v2:172.21.15.134:3300/0 conn(0x7f2734154c60 0x7f2734070ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:45.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:45.846+0000 7f27337fe640 1 -- 172.21.15.106:0/4188640847 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27340caf50 con 0x7f27340c5940 2024-04-25T10:30:45.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:45.846+0000 7f27337fe640 1 --2- 172.21.15.106:0/4188640847 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f27340c5940 0x7f273415aaa0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f272c009e50 tx=0x7f272c02a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c408d538aface567 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:45.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:45.846+0000 7f27327fc640 1 -- 172.21.15.106:0/4188640847 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f272c036070 con 0x7f27340c5940 2024-04-25T10:30:45.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:45.847+0000 7f27327fc640 1 -- 172.21.15.106:0/4188640847 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f272c031600 con 0x7f27340c5940 2024-04-25T10:30:45.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:45.847+0000 7f27327fc640 1 -- 172.21.15.106:0/4188640847 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f272c0319d0 con 0x7f27340c5940 2024-04-25T10:30:45.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:45.847+0000 7f2739634640 1 -- 172.21.15.106:0/4188640847 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f27340c5940 msgr2=0x7f273415aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:45.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:45.847+0000 7f2739634640 1 --2- 172.21.15.106:0/4188640847 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f27340c5940 0x7f273415aaa0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f272c009e50 tx=0x7f272c02a750 comp rx=0 tx=0).stop 2024-04-25T10:30:45.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:45.847+0000 7f2739634640 1 -- 172.21.15.106:0/4188640847 shutdown_connections 2024-04-25T10:30:45.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:45.847+0000 7f2739634640 1 --2- 172.21.15.106:0/4188640847 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f27340c5940 0x7f273415aaa0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:45.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:45.847+0000 7f2739634640 1 --2- 172.21.15.106:0/4188640847 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2734071220 0x7f27340c5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:45.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:45.847+0000 7f2739634640 1 --2- 172.21.15.106:0/4188640847 >> v2:172.21.15.134:3300/0 conn(0x7f2734154c60 0x7f2734070ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:45.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:45.847+0000 7f2739634640 1 -- 172.21.15.106:0/4188640847 >> 172.21.15.106:0/4188640847 conn(0x7f27340cff60 msgr2=0x7f27340d0370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:45.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:45.847+0000 7f2739634640 1 -- 172.21.15.106:0/4188640847 shutdown_connections 2024-04-25T10:30:45.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:45.847+0000 7f2739634640 1 -- 172.21.15.106:0/4188640847 wait complete. 2024-04-25T10:30:45.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:45.849+0000 7f2739634640 1 Processor -- start 2024-04-25T10:30:45.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:45.849+0000 7f2739634640 1 -- start start 2024-04-25T10:30:45.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:45.850+0000 7f2739634640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2734071220 0x7f27341f3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:45.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:45.850+0000 7f2739634640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27340c5940 0x7f27340cf150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:45.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:45.850+0000 7f2739634640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2734154c60 0x7f273415e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:45.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:45.850+0000 7f2739634640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2734155070 con 0x7f2734071220 2024-04-25T10:30:45.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:45.850+0000 7f2739634640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f27340cb510 con 0x7f2734154c60 2024-04-25T10:30:45.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:45.850+0000 7f2739634640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f27341551f0 con 0x7f27340c5940 2024-04-25T10:30:45.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:45.850+0000 7f2732ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2734071220 0x7f27341f3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:45.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:45.850+0000 7f2732ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2734071220 0x7f27341f3400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55446/0 (socket says 172.21.15.106:55446) 2024-04-25T10:30:45.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:45.850+0000 7f2732ffd640 1 -- 172.21.15.106:0/3676211332 learned_addr learned my addr 172.21.15.106:0/3676211332 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:45.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:45.850+0000 7f27337fe640 1 --2- 172.21.15.106:0/3676211332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2734154c60 0x7f273415e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:45.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:45.850+0000 7f272bfff640 1 --2- 172.21.15.106:0/3676211332 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27340c5940 0x7f27340cf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:45.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:45.851+0000 7f2732ffd640 1 -- 172.21.15.106:0/3676211332 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27340c5940 msgr2=0x7f27340cf150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:45.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:45.851+0000 7f2732ffd640 1 --2- 172.21.15.106:0/3676211332 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27340c5940 0x7f27340cf150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:45.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:45.851+0000 7f2732ffd640 1 -- 172.21.15.106:0/3676211332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2734154c60 msgr2=0x7f273415e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:45.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:45.851+0000 7f2732ffd640 1 --2- 172.21.15.106:0/3676211332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2734154c60 0x7f273415e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:45.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:45.851+0000 7f2732ffd640 1 -- 172.21.15.106:0/3676211332 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27341f3940 con 0x7f2734071220 2024-04-25T10:30:45.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:45.851+0000 7f2732ffd640 1 --2- 172.21.15.106:0/3676211332 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2734071220 0x7f27341f3400 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f272400e8a0 tx=0x7f272400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:45.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:45.851+0000 7f272b7fe640 1 -- 172.21.15.106:0/3676211332 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2724017070 con 0x7f2734071220 2024-04-25T10:30:45.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:45.851+0000 7f272b7fe640 1 -- 172.21.15.106:0/3676211332 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f272400c940 con 0x7f2734071220 2024-04-25T10:30:45.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:45.851+0000 7f2739634640 1 -- 172.21.15.106:0/3676211332 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f273414b4d0 con 0x7f2734071220 2024-04-25T10:30:45.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:45.851+0000 7f272b7fe640 1 -- 172.21.15.106:0/3676211332 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f272400cd30 con 0x7f2734071220 2024-04-25T10:30:45.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:45.851+0000 7f2739634640 1 -- 172.21.15.106:0/3676211332 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f273414ba90 con 0x7f2734071220 2024-04-25T10:30:45.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:45.852+0000 7f2739634640 1 -- 172.21.15.106:0/3676211332 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27341f3b10 con 0x7f2734071220 2024-04-25T10:30:45.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:45.853+0000 7f272b7fe640 1 -- 172.21.15.106:0/3676211332 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f272401a070 con 0x7f2734071220 2024-04-25T10:30:45.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:45.853+0000 7f272b7fe640 1 --2- 172.21.15.106:0/3676211332 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f27140744e0 0x7f27140769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:45.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:45.853+0000 7f272bfff640 1 --2- 172.21.15.106:0/3676211332 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f27140744e0 0x7f27140769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:45.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:45.854+0000 7f272b7fe640 1 -- 172.21.15.106:0/3676211332 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2724094fc0 con 0x7f2734071220 2024-04-25T10:30:45.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:45.854+0000 7f272bfff640 1 --2- 172.21.15.106:0/3676211332 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f27140744e0 0x7f27140769f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f273415be00 tx=0x7f271c009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:45.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:45.858+0000 7f272b7fe640 1 -- 172.21.15.106:0/3676211332 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f272405f100 con 0x7f2734071220 2024-04-25T10:30:46.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:46.068+0000 7f2739634640 1 -- 172.21.15.106:0/3676211332 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f273414bc60 con 0x7f2734071220 2024-04-25T10:30:46.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:46.069+0000 7f272b7fe640 1 -- 172.21.15.106:0/3676211332 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+196 (secure 0 0 0) 0x7f2724063400 con 0x7f2734071220 2024-04-25T10:30:46.071 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:30:46.071 INFO:teuthology.orchestra.run.smithi106.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2024-04-25T10:30:46.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:46.071+0000 7f2739634640 1 -- 172.21.15.106:0/3676211332 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f27140744e0 msgr2=0x7f27140769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:46.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:46.071+0000 7f2739634640 1 --2- 172.21.15.106:0/3676211332 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f27140744e0 0x7f27140769f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f273415be00 tx=0x7f271c009190 comp rx=0 tx=0).stop 2024-04-25T10:30:46.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:46.071+0000 7f2739634640 1 -- 172.21.15.106:0/3676211332 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2734071220 msgr2=0x7f27341f3400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:46.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:46.071+0000 7f2739634640 1 --2- 172.21.15.106:0/3676211332 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2734071220 0x7f27341f3400 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f272400e8a0 tx=0x7f272400ed60 comp rx=0 tx=0).stop 2024-04-25T10:30:46.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:46.071+0000 7f2739634640 1 -- 172.21.15.106:0/3676211332 shutdown_connections 2024-04-25T10:30:46.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:46.071+0000 7f2739634640 1 --2- 172.21.15.106:0/3676211332 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f27140744e0 0x7f27140769f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:46.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:46.071+0000 7f2739634640 1 --2- 172.21.15.106:0/3676211332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2734154c60 0x7f273415e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:46.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:46.071+0000 7f2739634640 1 --2- 172.21.15.106:0/3676211332 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27340c5940 0x7f27340cf150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:46.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:46.071+0000 7f2739634640 1 --2- 172.21.15.106:0/3676211332 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2734071220 0x7f27341f3400 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:46.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:46.071+0000 7f2739634640 1 -- 172.21.15.106:0/3676211332 >> 172.21.15.106:0/3676211332 conn(0x7f27340cff60 msgr2=0x7f2734153990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:46.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:46.071+0000 7f2739634640 1 -- 172.21.15.106:0/3676211332 shutdown_connections 2024-04-25T10:30:46.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:46.071+0000 7f2739634640 1 -- 172.21.15.106:0/3676211332 wait complete. 2024-04-25T10:30:46.817 INFO:tasks.ceph.mds.d.smithi134.stderr:2024-04-25T10:30:46.815+0000 7f3ef08bf640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:30:46.817 INFO:tasks.ceph.mds.d.smithi134.stderr:2024-04-25T10:30:46.815+0000 7f3ef08bf640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-25T10:30:49.085 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T10:30:49.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.202+0000 7f00b203c640 1 Processor -- start 2024-04-25T10:30:49.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.202+0000 7f00b203c640 1 -- start start 2024-04-25T10:30:49.207 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.204+0000 7f00b203c640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f00ac0cc600 0x7f00ac0c5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:49.207 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.204+0000 7f00b203c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f00ac0cb2b0 0x7f00ac165340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:49.207 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.204+0000 7f00b203c640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00ac15b8c0 con 0x7f00ac0c58d0 2024-04-25T10:30:49.207 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.204+0000 7f00b203c640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f00ac15eea0 con 0x7f00ac0cc600 2024-04-25T10:30:49.207 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.204+0000 7f00b203c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f00ac15b5e0 con 0x7f00ac0cb2b0 2024-04-25T10:30:49.207 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.204+0000 7f00ab7fe640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f00ac0cc600 0x7f00ac0c5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:49.207 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.204+0000 7f00abfff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f00ac0cb2b0 0x7f00ac165340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:49.207 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.204+0000 7f00ab7fe640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f00ac0cc600 0x7f00ac0c5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.106:59588/0 (socket says 172.21.15.106:59588) 2024-04-25T10:30:49.207 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.204+0000 7f00ab7fe640 1 -- 172.21.15.106:0/317382948 learned_addr learned my addr 172.21.15.106:0/317382948 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:49.207 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.204+0000 7f00aaffd640 1 --1- 172.21.15.106:0/317382948 >> v1:172.21.15.134:6789/0 conn(0x7f00ac0c58d0 0x7f00ac0cce90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.106:44278/0 (socket says 172.21.15.106:44278) 2024-04-25T10:30:49.207 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.204+0000 7f00ab7fe640 1 -- 172.21.15.106:0/317382948 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f00ac0cb2b0 msgr2=0x7f00ac165340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:49.207 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.204+0000 7f00ab7fe640 1 --2- 172.21.15.106:0/317382948 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f00ac0cb2b0 0x7f00ac165340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:49.207 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.204+0000 7f00ab7fe640 1 -- 172.21.15.106:0/317382948 >> v1:172.21.15.134:6789/0 conn(0x7f00ac0c58d0 legacy=0x7f00ac0cce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:49.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.204+0000 7f00ab7fe640 1 -- 172.21.15.106:0/317382948 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00ac15e0e0 con 0x7f00ac0cc600 2024-04-25T10:30:49.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.204+0000 7f00aa7fc640 1 -- 172.21.15.106:0/317382948 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4226189866 0 0) 0x7f00ac15b8c0 con 0x7f00ac0c58d0 2024-04-25T10:30:49.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.204+0000 7f00abfff640 1 --2- 172.21.15.106:0/317382948 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f00ac0cb2b0 0x7f00ac165340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:49.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.205+0000 7f00ab7fe640 1 --2- 172.21.15.106:0/317382948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00ac0cc600 0x7f00ac0c5390 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f009c009870 tx=0x7f009c02a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f63c7d8121b91900 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:49.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.205+0000 7f00aa7fc640 1 -- 172.21.15.106:0/317382948 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f009c037070 con 0x7f00ac0cc600 2024-04-25T10:30:49.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.205+0000 7f00aa7fc640 1 -- 172.21.15.106:0/317382948 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f009c032440 con 0x7f00ac0cc600 2024-04-25T10:30:49.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.205+0000 7f00aa7fc640 1 -- 172.21.15.106:0/317382948 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f009c032810 con 0x7f00ac0cc600 2024-04-25T10:30:49.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.206+0000 7f00b203c640 1 -- 172.21.15.106:0/317382948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00ac0cc600 msgr2=0x7f00ac0c5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:49.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.206+0000 7f00b203c640 1 --2- 172.21.15.106:0/317382948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00ac0cc600 0x7f00ac0c5390 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f009c009870 tx=0x7f009c02a270 comp rx=0 tx=0).stop 2024-04-25T10:30:49.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.206+0000 7f00b203c640 1 -- 172.21.15.106:0/317382948 shutdown_connections 2024-04-25T10:30:49.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.206+0000 7f00b203c640 1 --2- 172.21.15.106:0/317382948 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f00ac0cb2b0 0x7f00ac165340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:49.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.206+0000 7f00b203c640 1 --2- 172.21.15.106:0/317382948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00ac0cc600 0x7f00ac0c5390 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:49.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.206+0000 7f00b203c640 1 -- 172.21.15.106:0/317382948 >> 172.21.15.106:0/317382948 conn(0x7f00ac0727d0 msgr2=0x7f00ac072c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:49.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.206+0000 7f00b203c640 1 -- 172.21.15.106:0/317382948 shutdown_connections 2024-04-25T10:30:49.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.206+0000 7f00b203c640 1 -- 172.21.15.106:0/317382948 wait complete. 2024-04-25T10:30:49.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.208+0000 7f00b203c640 1 Processor -- start 2024-04-25T10:30:49.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.208+0000 7f00b203c640 1 -- start start 2024-04-25T10:30:49.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.209+0000 7f00b203c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f00ac0c58d0 0x7f00ac1f6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:49.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.209+0000 7f00b203c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00ac0cb2b0 0x7f00ac0c7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:49.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.209+0000 7f00b203c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00ac0cc600 0x7f00ac14d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:49.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.209+0000 7f00b203c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f00ac15e980 con 0x7f00ac0cb2b0 2024-04-25T10:30:49.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.209+0000 7f00b203c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f00ac15e6a0 con 0x7f00ac0cc600 2024-04-25T10:30:49.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.209+0000 7f00b203c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f00ac0cbff0 con 0x7f00ac0c58d0 2024-04-25T10:30:49.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.209+0000 7f00ab7fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f00ac0c58d0 0x7f00ac1f6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:49.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.209+0000 7f00aaffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00ac0cb2b0 0x7f00ac0c7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:49.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.209+0000 7f00abfff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00ac0cc600 0x7f00ac14d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:49.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.209+0000 7f00ab7fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f00ac0c58d0 0x7f00ac1f6ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.106:35042/0 (socket says 172.21.15.106:35042) 2024-04-25T10:30:49.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.209+0000 7f00ab7fe640 1 -- 172.21.15.106:0/2515860495 learned_addr learned my addr 172.21.15.106:0/2515860495 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:49.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.210+0000 7f00ab7fe640 1 -- 172.21.15.106:0/2515860495 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00ac0cc600 msgr2=0x7f00ac14d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:49.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.210+0000 7f00ab7fe640 1 --2- 172.21.15.106:0/2515860495 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00ac0cc600 0x7f00ac14d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:49.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.210+0000 7f00ab7fe640 1 -- 172.21.15.106:0/2515860495 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00ac0cb2b0 msgr2=0x7f00ac0c7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:49.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.210+0000 7f00ab7fe640 1 --2- 172.21.15.106:0/2515860495 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00ac0cb2b0 0x7f00ac0c7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:49.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.210+0000 7f00ab7fe640 1 -- 172.21.15.106:0/2515860495 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00ac06d6d0 con 0x7f00ac0c58d0 2024-04-25T10:30:49.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.210+0000 7f00abfff640 1 --2- 172.21.15.106:0/2515860495 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00ac0cc600 0x7f00ac14d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:49.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.210+0000 7f00aaffd640 1 --2- 172.21.15.106:0/2515860495 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00ac0cb2b0 0x7f00ac0c7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:49.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.210+0000 7f00ab7fe640 1 --2- 172.21.15.106:0/2515860495 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f00ac0c58d0 0x7f00ac1f6ea0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f009c009870 tx=0x7f009c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:49.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.210+0000 7f0093fff640 1 -- 172.21.15.106:0/2515860495 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f009c041070 con 0x7f00ac0c58d0 2024-04-25T10:30:49.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.210+0000 7f0093fff640 1 -- 172.21.15.106:0/2515860495 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f009c03c890 con 0x7f00ac0c58d0 2024-04-25T10:30:49.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.210+0000 7f00b203c640 1 -- 172.21.15.106:0/2515860495 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00ac151d70 con 0x7f00ac0c58d0 2024-04-25T10:30:49.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.210+0000 7f0093fff640 1 -- 172.21.15.106:0/2515860495 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f009c037040 con 0x7f00ac0c58d0 2024-04-25T10:30:49.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.210+0000 7f00b203c640 1 -- 172.21.15.106:0/2515860495 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f00ac1522b0 con 0x7f00ac0c58d0 2024-04-25T10:30:49.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.211+0000 7f00b203c640 1 -- 172.21.15.106:0/2515860495 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00ac06a3d0 con 0x7f00ac0c58d0 2024-04-25T10:30:49.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.212+0000 7f0093fff640 1 -- 172.21.15.106:0/2515860495 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f009c044070 con 0x7f00ac0c58d0 2024-04-25T10:30:49.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.212+0000 7f0093fff640 1 --2- 172.21.15.106:0/2515860495 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f00980744e0 0x7f00980769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:49.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.212+0000 7f0093fff640 1 -- 172.21.15.106:0/2515860495 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f009c0b5a90 con 0x7f00ac0c58d0 2024-04-25T10:30:49.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.212+0000 7f00aaffd640 1 --2- 172.21.15.106:0/2515860495 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f00980744e0 0x7f00980769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:49.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.212+0000 7f00aaffd640 1 --2- 172.21.15.106:0/2515860495 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f00980744e0 0x7f00980769f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f00a0007fd0 tx=0x7f00a0007de0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:49.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.217+0000 7f0093fff640 1 -- 172.21.15.106:0/2515860495 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f009c07fc60 con 0x7f00ac0c58d0 2024-04-25T10:30:49.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.426+0000 7f00b203c640 1 -- 172.21.15.106:0/2515860495 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f00ac0b33f0 con 0x7f00ac0c58d0 2024-04-25T10:30:49.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.427+0000 7f0093fff640 1 -- 172.21.15.106:0/2515860495 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+196 (secure 0 0 0) 0x7f009c083f60 con 0x7f00ac0c58d0 2024-04-25T10:30:49.429 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:30:49.430 INFO:teuthology.orchestra.run.smithi106.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2024-04-25T10:30:49.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.429+0000 7f00b203c640 1 -- 172.21.15.106:0/2515860495 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f00980744e0 msgr2=0x7f00980769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:49.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.429+0000 7f00b203c640 1 --2- 172.21.15.106:0/2515860495 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f00980744e0 0x7f00980769f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f00a0007fd0 tx=0x7f00a0007de0 comp rx=0 tx=0).stop 2024-04-25T10:30:49.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.429+0000 7f00b203c640 1 -- 172.21.15.106:0/2515860495 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f00ac0c58d0 msgr2=0x7f00ac1f6ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:49.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.429+0000 7f00b203c640 1 --2- 172.21.15.106:0/2515860495 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f00ac0c58d0 0x7f00ac1f6ea0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f009c009870 tx=0x7f009c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:30:49.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.429+0000 7f00b203c640 1 -- 172.21.15.106:0/2515860495 shutdown_connections 2024-04-25T10:30:49.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.429+0000 7f00b203c640 1 --2- 172.21.15.106:0/2515860495 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f00980744e0 0x7f00980769f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:49.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.429+0000 7f00b203c640 1 --2- 172.21.15.106:0/2515860495 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00ac0cc600 0x7f00ac14d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:49.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.429+0000 7f00b203c640 1 --2- 172.21.15.106:0/2515860495 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00ac0cb2b0 0x7f00ac0c7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:49.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.429+0000 7f00b203c640 1 --2- 172.21.15.106:0/2515860495 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f00ac0c58d0 0x7f00ac1f6ea0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:49.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.429+0000 7f00b203c640 1 -- 172.21.15.106:0/2515860495 >> 172.21.15.106:0/2515860495 conn(0x7f00ac0727d0 msgr2=0x7f00ac1506f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:49.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.429+0000 7f00b203c640 1 -- 172.21.15.106:0/2515860495 shutdown_connections 2024-04-25T10:30:49.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:49.429+0000 7f00b203c640 1 -- 172.21.15.106:0/2515860495 wait complete. 2024-04-25T10:30:51.818 INFO:tasks.ceph.mds.d.smithi134.stderr:2024-04-25T10:30:51.814+0000 7f3ef08bf640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-04-25T10:30:52.444 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T10:30:52.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.560+0000 7f8032ff3640 1 Processor -- start 2024-04-25T10:30:52.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.561+0000 7f8032ff3640 1 -- start start 2024-04-25T10:30:52.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.562+0000 7f8032ff3640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f802c072870 0x7f802c15a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:52.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.562+0000 7f8032ff3640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f802c0c5280 0x7f802c15d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:52.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.562+0000 7f8032ff3640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f802c0cae80 0x7f802c153710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:52.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.562+0000 7f8032ff3640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f802c0c5690 con 0x7f802c0c5280 2024-04-25T10:30:52.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.562+0000 7f8032ff3640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f802c0c7c40 con 0x7f802c072870 2024-04-25T10:30:52.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.562+0000 7f8032ff3640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f802c0c7f20 con 0x7f802c0cae80 2024-04-25T10:30:52.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.562+0000 7f802bfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f802c0c5280 0x7f802c15d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:52.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.562+0000 7f802bfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f802c0c5280 0x7f802c15d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55460/0 (socket says 172.21.15.106:55460) 2024-04-25T10:30:52.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.562+0000 7f802bfff640 1 -- 172.21.15.106:0/3479329389 learned_addr learned my addr 172.21.15.106:0/3479329389 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:52.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.563+0000 7f8031590640 1 --2- 172.21.15.106:0/3479329389 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f802c0cae80 0x7f802c153710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:52.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.563+0000 7f8030d8f640 1 --2- 172.21.15.106:0/3479329389 >> v2:172.21.15.134:3300/0 conn(0x7f802c072870 0x7f802c15a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:52.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.563+0000 7f802bfff640 1 -- 172.21.15.106:0/3479329389 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f802c0cae80 msgr2=0x7f802c153710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:52.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.563+0000 7f802bfff640 1 --2- 172.21.15.106:0/3479329389 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f802c0cae80 0x7f802c153710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:52.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.563+0000 7f802bfff640 1 -- 172.21.15.106:0/3479329389 >> v2:172.21.15.134:3300/0 conn(0x7f802c072870 msgr2=0x7f802c15a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:52.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.563+0000 7f802bfff640 1 --2- 172.21.15.106:0/3479329389 >> v2:172.21.15.134:3300/0 conn(0x7f802c072870 0x7f802c15a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:52.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.563+0000 7f802bfff640 1 -- 172.21.15.106:0/3479329389 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f802c0c8200 con 0x7f802c0c5280 2024-04-25T10:30:52.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.563+0000 7f802bfff640 1 --2- 172.21.15.106:0/3479329389 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f802c0c5280 0x7f802c15d9a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f802c0712e0 tx=0x7f802002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e723a1540c038c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:52.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.564+0000 7f802b7fe640 1 -- 172.21.15.106:0/3479329389 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8020037070 con 0x7f802c0c5280 2024-04-25T10:30:52.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.564+0000 7f802b7fe640 1 -- 172.21.15.106:0/3479329389 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8020032440 con 0x7f802c0c5280 2024-04-25T10:30:52.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.564+0000 7f802b7fe640 1 -- 172.21.15.106:0/3479329389 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8020032810 con 0x7f802c0c5280 2024-04-25T10:30:52.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.565+0000 7f8032ff3640 1 -- 172.21.15.106:0/3479329389 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f802c0c5280 msgr2=0x7f802c15d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:52.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.565+0000 7f8032ff3640 1 --2- 172.21.15.106:0/3479329389 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f802c0c5280 0x7f802c15d9a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f802c0712e0 tx=0x7f802002a140 comp rx=0 tx=0).stop 2024-04-25T10:30:52.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.565+0000 7f8032ff3640 1 -- 172.21.15.106:0/3479329389 shutdown_connections 2024-04-25T10:30:52.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.565+0000 7f8032ff3640 1 --2- 172.21.15.106:0/3479329389 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f802c0cae80 0x7f802c153710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:52.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.565+0000 7f8032ff3640 1 --2- 172.21.15.106:0/3479329389 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f802c0c5280 0x7f802c15d9a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:52.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.565+0000 7f8032ff3640 1 --2- 172.21.15.106:0/3479329389 >> v2:172.21.15.134:3300/0 conn(0x7f802c072870 0x7f802c15a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:52.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.565+0000 7f8032ff3640 1 -- 172.21.15.106:0/3479329389 >> 172.21.15.106:0/3479329389 conn(0x7f802c06c9b0 msgr2=0x7f802c06cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:52.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.565+0000 7f8032ff3640 1 -- 172.21.15.106:0/3479329389 shutdown_connections 2024-04-25T10:30:52.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.565+0000 7f8032ff3640 1 -- 172.21.15.106:0/3479329389 wait complete. 2024-04-25T10:30:52.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.567+0000 7f8032ff3640 1 Processor -- start 2024-04-25T10:30:52.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.567+0000 7f8032ff3640 1 -- start start 2024-04-25T10:30:52.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.568+0000 7f8032ff3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f802c072870 0x7f802c14c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:52.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.568+0000 7f8032ff3640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f802c0cae80 0x7f802c1eee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:52.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.568+0000 7f8032ff3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f802c15b470 0x7f802c146420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:52.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.568+0000 7f8032ff3640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f802c15a070 con 0x7f802c072870 2024-04-25T10:30:52.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.568+0000 7f8032ff3640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f802c159d90 con 0x7f802c15b470 2024-04-25T10:30:52.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.568+0000 7f8032ff3640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f802c0c7920 con 0x7f802c0cae80 2024-04-25T10:30:52.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.568+0000 7f8030d8f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f802c072870 0x7f802c14c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:52.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.568+0000 7f8030d8f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f802c072870 0x7f802c14c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55470/0 (socket says 172.21.15.106:55470) 2024-04-25T10:30:52.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.568+0000 7f8030d8f640 1 -- 172.21.15.106:0/3135272861 learned_addr learned my addr 172.21.15.106:0/3135272861 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:52.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.568+0000 7f8031590640 1 --2- 172.21.15.106:0/3135272861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f802c15b470 0x7f802c146420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:52.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.568+0000 7f802bfff640 1 --2- 172.21.15.106:0/3135272861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f802c0cae80 0x7f802c1eee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:52.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.568+0000 7f8030d8f640 1 -- 172.21.15.106:0/3135272861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f802c0cae80 msgr2=0x7f802c1eee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:52.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.568+0000 7f8030d8f640 1 --2- 172.21.15.106:0/3135272861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f802c0cae80 0x7f802c1eee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:52.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.568+0000 7f8030d8f640 1 -- 172.21.15.106:0/3135272861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f802c15b470 msgr2=0x7f802c146420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:52.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.568+0000 7f8030d8f640 1 --2- 172.21.15.106:0/3135272861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f802c15b470 0x7f802c146420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:52.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.568+0000 7f8030d8f640 1 -- 172.21.15.106:0/3135272861 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f802c1ef3b0 con 0x7f802c072870 2024-04-25T10:30:52.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.569+0000 7f8030d8f640 1 --2- 172.21.15.106:0/3135272861 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f802c072870 0x7f802c14c010 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f801c010ed0 tx=0x7f801c00bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:52.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.569+0000 7f80297fa640 1 -- 172.21.15.106:0/3135272861 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f801c01a020 con 0x7f802c072870 2024-04-25T10:30:52.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.569+0000 7f80297fa640 1 -- 172.21.15.106:0/3135272861 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f801c00daa0 con 0x7f802c072870 2024-04-25T10:30:52.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.569+0000 7f8032ff3640 1 -- 172.21.15.106:0/3135272861 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f802c14a9b0 con 0x7f802c072870 2024-04-25T10:30:52.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.569+0000 7f80297fa640 1 -- 172.21.15.106:0/3135272861 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f801c016400 con 0x7f802c072870 2024-04-25T10:30:52.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.569+0000 7f8032ff3640 1 -- 172.21.15.106:0/3135272861 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f802c150470 con 0x7f802c072870 2024-04-25T10:30:52.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.570+0000 7f8032ff3640 1 -- 172.21.15.106:0/3135272861 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f802c06a3d0 con 0x7f802c072870 2024-04-25T10:30:52.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.571+0000 7f80297fa640 1 -- 172.21.15.106:0/3135272861 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f801c01d020 con 0x7f802c072870 2024-04-25T10:30:52.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.571+0000 7f80297fa640 1 --2- 172.21.15.106:0/3135272861 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f80180744b0 0x7f80180769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:52.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.571+0000 7f80297fa640 1 -- 172.21.15.106:0/3135272861 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f801c097110 con 0x7f802c072870 2024-04-25T10:30:52.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.571+0000 7f802bfff640 1 --2- 172.21.15.106:0/3135272861 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f80180744b0 0x7f80180769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:52.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.571+0000 7f802bfff640 1 --2- 172.21.15.106:0/3135272861 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f80180744b0 0x7f80180769c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8020034000 tx=0x7f802002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:52.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.576+0000 7f80297fa640 1 -- 172.21.15.106:0/3135272861 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f801c024070 con 0x7f802c072870 2024-04-25T10:30:52.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.787+0000 7f8032ff3640 1 -- 172.21.15.106:0/3135272861 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f802c14af10 con 0x7f802c072870 2024-04-25T10:30:52.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.787+0000 7f80297fa640 1 -- 172.21.15.106:0/3135272861 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+196 (secure 0 0 0) 0x7f801c061250 con 0x7f802c072870 2024-04-25T10:30:52.789 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:30:52.789 INFO:teuthology.orchestra.run.smithi106.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2024-04-25T10:30:52.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.789+0000 7f8032ff3640 1 -- 172.21.15.106:0/3135272861 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f80180744b0 msgr2=0x7f80180769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:52.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.789+0000 7f8032ff3640 1 --2- 172.21.15.106:0/3135272861 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f80180744b0 0x7f80180769c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8020034000 tx=0x7f802002e000 comp rx=0 tx=0).stop 2024-04-25T10:30:52.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.789+0000 7f8032ff3640 1 -- 172.21.15.106:0/3135272861 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f802c072870 msgr2=0x7f802c14c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:52.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.789+0000 7f8032ff3640 1 --2- 172.21.15.106:0/3135272861 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f802c072870 0x7f802c14c010 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f801c010ed0 tx=0x7f801c00bce0 comp rx=0 tx=0).stop 2024-04-25T10:30:52.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.789+0000 7f8032ff3640 1 -- 172.21.15.106:0/3135272861 shutdown_connections 2024-04-25T10:30:52.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.789+0000 7f8032ff3640 1 --2- 172.21.15.106:0/3135272861 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f80180744b0 0x7f80180769c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:52.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.789+0000 7f8032ff3640 1 --2- 172.21.15.106:0/3135272861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f802c15b470 0x7f802c146420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:52.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.789+0000 7f8032ff3640 1 --2- 172.21.15.106:0/3135272861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f802c0cae80 0x7f802c1eee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:52.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.789+0000 7f8032ff3640 1 --2- 172.21.15.106:0/3135272861 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f802c072870 0x7f802c14c010 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:52.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.789+0000 7f8032ff3640 1 -- 172.21.15.106:0/3135272861 >> 172.21.15.106:0/3135272861 conn(0x7f802c06c9b0 msgr2=0x7f802c157710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:52.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.789+0000 7f8032ff3640 1 -- 172.21.15.106:0/3135272861 shutdown_connections 2024-04-25T10:30:52.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:52.789+0000 7f8032ff3640 1 -- 172.21.15.106:0/3135272861 wait complete. 2024-04-25T10:30:55.804 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T10:30:55.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:55.926+0000 7fec52315640 1 Processor -- start 2024-04-25T10:30:55.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:55.926+0000 7fec52315640 1 -- start start 2024-04-25T10:30:55.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:55.927+0000 7fec52315640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fec4c0c7e30 0x7fec4c0c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:55.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:55.927+0000 7fec52315640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fec4c156ef0 0x7fec4c0b4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:55.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:55.927+0000 7fec52315640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec4c06d000 0x7fec4c06d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:55.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:55.927+0000 7fec52315640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fec4c0c7c60 con 0x7fec4c0c7e30 2024-04-25T10:30:55.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:55.927+0000 7fec52315640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fec4c0ce630 con 0x7fec4c156ef0 2024-04-25T10:30:55.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:55.927+0000 7fec52315640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fec4c0c7980 con 0x7fec4c06d000 2024-04-25T10:30:55.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:55.927+0000 7fec4bfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fec4c0c7e30 0x7fec4c0c8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:55.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:55.927+0000 7fec4bfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fec4c0c7e30 0x7fec4c0c8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34992/0 (socket says 172.21.15.106:34992) 2024-04-25T10:30:55.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:55.927+0000 7fec4bfff640 1 -- 172.21.15.106:0/2722001695 learned_addr learned my addr 172.21.15.106:0/2722001695 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:55.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:55.927+0000 7fec4b7fe640 1 --2- 172.21.15.106:0/2722001695 >> v2:172.21.15.134:3300/0 conn(0x7fec4c156ef0 0x7fec4c0b4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:55.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:55.927+0000 7fec508b2640 1 --2- 172.21.15.106:0/2722001695 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec4c06d000 0x7fec4c06d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:55.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:55.927+0000 7fec4bfff640 1 -- 172.21.15.106:0/2722001695 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec4c06d000 msgr2=0x7fec4c06d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:55.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:55.927+0000 7fec4bfff640 1 --2- 172.21.15.106:0/2722001695 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec4c06d000 0x7fec4c06d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:55.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:55.927+0000 7fec4bfff640 1 -- 172.21.15.106:0/2722001695 >> v2:172.21.15.134:3300/0 conn(0x7fec4c156ef0 msgr2=0x7fec4c0b4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:55.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:55.927+0000 7fec4bfff640 1 --2- 172.21.15.106:0/2722001695 >> v2:172.21.15.134:3300/0 conn(0x7fec4c156ef0 0x7fec4c0b4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:55.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:55.927+0000 7fec4bfff640 1 -- 172.21.15.106:0/2722001695 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec4c0c7010 con 0x7fec4c0c7e30 2024-04-25T10:30:55.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:55.928+0000 7fec4bfff640 1 --2- 172.21.15.106:0/2722001695 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec4c0c7e30 0x7fec4c0c8d90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fec4c06ef00 tx=0x7fec3c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f07058affddf7ba4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:55.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:55.928+0000 7fec4affd640 1 -- 172.21.15.106:0/2722001695 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3c037070 con 0x7fec4c0c7e30 2024-04-25T10:30:55.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:55.928+0000 7fec4affd640 1 -- 172.21.15.106:0/2722001695 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec3c032440 con 0x7fec4c0c7e30 2024-04-25T10:30:55.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:55.928+0000 7fec4affd640 1 -- 172.21.15.106:0/2722001695 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3c032810 con 0x7fec4c0c7e30 2024-04-25T10:30:55.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:55.929+0000 7fec52315640 1 -- 172.21.15.106:0/2722001695 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec4c0c7e30 msgr2=0x7fec4c0c8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:55.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:55.929+0000 7fec52315640 1 --2- 172.21.15.106:0/2722001695 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec4c0c7e30 0x7fec4c0c8d90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fec4c06ef00 tx=0x7fec3c02a140 comp rx=0 tx=0).stop 2024-04-25T10:30:55.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:55.929+0000 7fec52315640 1 -- 172.21.15.106:0/2722001695 shutdown_connections 2024-04-25T10:30:55.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:55.929+0000 7fec52315640 1 --2- 172.21.15.106:0/2722001695 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec4c06d000 0x7fec4c06d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:55.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:55.929+0000 7fec52315640 1 --2- 172.21.15.106:0/2722001695 >> v2:172.21.15.134:3300/0 conn(0x7fec4c156ef0 0x7fec4c0b4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:55.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:55.929+0000 7fec52315640 1 --2- 172.21.15.106:0/2722001695 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec4c0c7e30 0x7fec4c0c8d90 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:55.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:55.929+0000 7fec52315640 1 -- 172.21.15.106:0/2722001695 >> 172.21.15.106:0/2722001695 conn(0x7fec4c068b10 msgr2=0x7fec4c0d1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:55.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:55.929+0000 7fec52315640 1 -- 172.21.15.106:0/2722001695 shutdown_connections 2024-04-25T10:30:55.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:55.929+0000 7fec52315640 1 -- 172.21.15.106:0/2722001695 wait complete. 2024-04-25T10:30:55.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:55.931+0000 7fec52315640 1 Processor -- start 2024-04-25T10:30:55.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:55.931+0000 7fec52315640 1 -- start start 2024-04-25T10:30:55.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:55.932+0000 7fec52315640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec4c06d000 0x7fec4c1511e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:55.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:55.932+0000 7fec52315640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec4c0c7e30 0x7fec4c0d1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:55.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:55.932+0000 7fec52315640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec4c156ef0 0x7fec4c1e8780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:55.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:55.932+0000 7fec52315640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fec4c15cfb0 con 0x7fec4c0c7e30 2024-04-25T10:30:55.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:55.932+0000 7fec52315640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fec4c15ce30 con 0x7fec4c156ef0 2024-04-25T10:30:55.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:55.932+0000 7fec52315640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fec4c15d130 con 0x7fec4c06d000 2024-04-25T10:30:55.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:55.933+0000 7fec4b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec4c0c7e30 0x7fec4c0d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:55.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:55.933+0000 7fec4b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec4c0c7e30 0x7fec4c0d1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35000/0 (socket says 172.21.15.106:35000) 2024-04-25T10:30:55.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:55.933+0000 7fec4b7fe640 1 -- 172.21.15.106:0/3122963120 learned_addr learned my addr 172.21.15.106:0/3122963120 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:55.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:55.933+0000 7fec4bfff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec4c06d000 0x7fec4c1511e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:55.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:55.933+0000 7fec508b2640 1 --2- 172.21.15.106:0/3122963120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec4c156ef0 0x7fec4c1e8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:55.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:55.933+0000 7fec4bfff640 1 -- 172.21.15.106:0/3122963120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec4c156ef0 msgr2=0x7fec4c1e8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:55.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:55.933+0000 7fec4bfff640 1 --2- 172.21.15.106:0/3122963120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec4c156ef0 0x7fec4c1e8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:55.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:55.933+0000 7fec4bfff640 1 -- 172.21.15.106:0/3122963120 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec4c0c7e30 msgr2=0x7fec4c0d1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:55.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:55.933+0000 7fec4bfff640 1 --2- 172.21.15.106:0/3122963120 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec4c0c7e30 0x7fec4c0d1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:55.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:55.933+0000 7fec4bfff640 1 -- 172.21.15.106:0/3122963120 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec4c1539d0 con 0x7fec4c06d000 2024-04-25T10:30:55.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:55.933+0000 7fec4b7fe640 1 --2- 172.21.15.106:0/3122963120 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec4c0c7e30 0x7fec4c0d1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:55.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:55.933+0000 7fec4bfff640 1 --2- 172.21.15.106:0/3122963120 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec4c06d000 0x7fec4c1511e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fec3c031000 tx=0x7fec3c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:55.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:55.933+0000 7fec48ff9640 1 -- 172.21.15.106:0/3122963120 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3c041020 con 0x7fec4c06d000 2024-04-25T10:30:55.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:55.933+0000 7fec48ff9640 1 -- 172.21.15.106:0/3122963120 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec3c03d660 con 0x7fec4c06d000 2024-04-25T10:30:55.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:55.933+0000 7fec48ff9640 1 -- 172.21.15.106:0/3122963120 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3c037040 con 0x7fec4c06d000 2024-04-25T10:30:55.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:55.933+0000 7fec52315640 1 -- 172.21.15.106:0/3122963120 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec4c151720 con 0x7fec4c06d000 2024-04-25T10:30:55.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:55.933+0000 7fec52315640 1 -- 172.21.15.106:0/3122963120 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fec4c153410 con 0x7fec4c06d000 2024-04-25T10:30:55.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:55.934+0000 7fec52315640 1 -- 172.21.15.106:0/3122963120 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec4c06a3d0 con 0x7fec4c06d000 2024-04-25T10:30:55.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:55.935+0000 7fec48ff9640 1 -- 172.21.15.106:0/3122963120 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fec3c044020 con 0x7fec4c06d000 2024-04-25T10:30:55.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:55.935+0000 7fec48ff9640 1 --2- 172.21.15.106:0/3122963120 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fec380744b0 0x7fec380769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:55.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:55.935+0000 7fec48ff9640 1 -- 172.21.15.106:0/3122963120 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fec3c0b57c0 con 0x7fec4c06d000 2024-04-25T10:30:55.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:55.935+0000 7fec4b7fe640 1 --2- 172.21.15.106:0/3122963120 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fec380744b0 0x7fec380769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:55.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:55.936+0000 7fec4b7fe640 1 --2- 172.21.15.106:0/3122963120 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fec380744b0 0x7fec380769c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fec4c0d1a00 tx=0x7fec40009110 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:55.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:55.940+0000 7fec48ff9640 1 -- 172.21.15.106:0/3122963120 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fec3c04b030 con 0x7fec4c06d000 2024-04-25T10:30:56.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.155+0000 7fec52315640 1 -- 172.21.15.106:0/3122963120 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fec4c15a5a0 con 0x7fec4c06d000 2024-04-25T10:30:56.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.155+0000 7fec48ff9640 1 -- 172.21.15.106:0/3122963120 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fec3c07f900 con 0x7fec4c06d000 2024-04-25T10:30:56.157 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:30:56.157 INFO:teuthology.orchestra.run.smithi106.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T10:30:56.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.157+0000 7fec52315640 1 -- 172.21.15.106:0/3122963120 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fec380744b0 msgr2=0x7fec380769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:56.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.157+0000 7fec52315640 1 --2- 172.21.15.106:0/3122963120 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fec380744b0 0x7fec380769c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fec4c0d1a00 tx=0x7fec40009110 comp rx=0 tx=0).stop 2024-04-25T10:30:56.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.157+0000 7fec52315640 1 -- 172.21.15.106:0/3122963120 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec4c06d000 msgr2=0x7fec4c1511e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:56.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.157+0000 7fec52315640 1 --2- 172.21.15.106:0/3122963120 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec4c06d000 0x7fec4c1511e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fec3c031000 tx=0x7fec3c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:30:56.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.157+0000 7fec52315640 1 -- 172.21.15.106:0/3122963120 shutdown_connections 2024-04-25T10:30:56.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.157+0000 7fec52315640 1 --2- 172.21.15.106:0/3122963120 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fec380744b0 0x7fec380769c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:56.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.157+0000 7fec52315640 1 --2- 172.21.15.106:0/3122963120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec4c156ef0 0x7fec4c1e8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:56.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.158+0000 7fec52315640 1 --2- 172.21.15.106:0/3122963120 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec4c0c7e30 0x7fec4c0d1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:56.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.158+0000 7fec52315640 1 --2- 172.21.15.106:0/3122963120 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec4c06d000 0x7fec4c1511e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:56.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.158+0000 7fec52315640 1 -- 172.21.15.106:0/3122963120 >> 172.21.15.106:0/3122963120 conn(0x7fec4c068b10 msgr2=0x7fec4c0cc630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:56.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.158+0000 7fec52315640 1 -- 172.21.15.106:0/3122963120 shutdown_connections 2024-04-25T10:30:56.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.158+0000 7fec52315640 1 -- 172.21.15.106:0/3122963120 wait complete. 2024-04-25T10:30:56.170 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-25T10:30:56.171 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:30:56.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.276+0000 7f2cf134d640 1 Processor -- start 2024-04-25T10:30:56.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.277+0000 7f2cf134d640 1 -- start start 2024-04-25T10:30:56.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.278+0000 7f2cf134d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2cec0cc6b0 0x7f2cec153c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:56.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.278+0000 7f2cf134d640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2cec0c9d40 con 0x7f2cec0c6a80 2024-04-25T10:30:56.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.278+0000 7f2cf134d640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2cec0c64c0 con 0x7f2cec0cc250 2024-04-25T10:30:56.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.278+0000 7f2cf134d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f2cec0c67a0 con 0x7f2cec0cc6b0 2024-04-25T10:30:56.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.278+0000 7f2ceb7fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f2cec0c6a80 0x7f2cec0c8a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:53502/0 (socket says 172.21.15.106:53502) 2024-04-25T10:30:56.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.278+0000 7f2ceb7fe640 1 -- 172.21.15.106:0/3899650494 learned_addr learned my addr 172.21.15.106:0/3899650494 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:30:56.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.278+0000 7f2cea7fc640 1 --2- 172.21.15.106:0/3899650494 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2cec0cc6b0 0x7f2cec153c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:56.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.278+0000 7f2ce9ffb640 1 -- 172.21.15.106:0/3899650494 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1452592794 0 0) 0x7f2cec0c9d40 con 0x7f2cec0c6a80 2024-04-25T10:30:56.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.278+0000 7f2ce9ffb640 1 -- 172.21.15.106:0/3899650494 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2cd4003520 con 0x7f2cec0c6a80 2024-04-25T10:30:56.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.278+0000 7f2ce9ffb640 1 -- 172.21.15.106:0/3899650494 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2847084733 0 0) 0x7f2cec0c64c0 con 0x7f2cec0cc250 2024-04-25T10:30:56.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.278+0000 7f2cea7fc640 1 -- 172.21.15.106:0/3899650494 >> v1:172.21.15.134:6789/0 conn(0x7f2cec0cc250 legacy=0x7f2cec159a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:56.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.278+0000 7f2cea7fc640 1 -- 172.21.15.106:0/3899650494 >> v1:172.21.15.106:6789/0 conn(0x7f2cec0c6a80 legacy=0x7f2cec0c8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:56.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.278+0000 7f2cea7fc640 1 -- 172.21.15.106:0/3899650494 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cec0cba50 con 0x7f2cec0cc6b0 2024-04-25T10:30:56.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.279+0000 7f2cea7fc640 1 --2- 172.21.15.106:0/3899650494 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2cec0cc6b0 0x7f2cec153c20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2ce00097d0 tx=0x7f2ce002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=951fc043da92c42a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:56.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.279+0000 7f2ce9ffb640 1 -- 172.21.15.106:0/3899650494 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce0037070 con 0x7f2cec0cc6b0 2024-04-25T10:30:56.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.279+0000 7f2ce9ffb640 1 -- 172.21.15.106:0/3899650494 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ce0032440 con 0x7f2cec0cc6b0 2024-04-25T10:30:56.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.279+0000 7f2ce9ffb640 1 -- 172.21.15.106:0/3899650494 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce0032810 con 0x7f2cec0cc6b0 2024-04-25T10:30:56.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.280+0000 7f2cf134d640 1 -- 172.21.15.106:0/3899650494 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2cec0cc6b0 msgr2=0x7f2cec153c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:56.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.280+0000 7f2cf134d640 1 --2- 172.21.15.106:0/3899650494 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2cec0cc6b0 0x7f2cec153c20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2ce00097d0 tx=0x7f2ce002a210 comp rx=0 tx=0).stop 2024-04-25T10:30:56.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.280+0000 7f2cf134d640 1 -- 172.21.15.106:0/3899650494 shutdown_connections 2024-04-25T10:30:56.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.280+0000 7f2cf134d640 1 --2- 172.21.15.106:0/3899650494 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2cec0cc6b0 0x7f2cec153c20 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:56.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.280+0000 7f2cf134d640 1 -- 172.21.15.106:0/3899650494 >> 172.21.15.106:0/3899650494 conn(0x7f2cec06d6d0 msgr2=0x7f2cec06dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:56.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.280+0000 7f2cf134d640 1 -- 172.21.15.106:0/3899650494 shutdown_connections 2024-04-25T10:30:56.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.280+0000 7f2cf134d640 1 -- 172.21.15.106:0/3899650494 wait complete. 2024-04-25T10:30:56.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.282+0000 7f2cf134d640 1 Processor -- start 2024-04-25T10:30:56.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.282+0000 7f2cf134d640 1 -- start start 2024-04-25T10:30:56.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.283+0000 7f2cf134d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2cec0c6a80 0x7f2cec06c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:56.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.283+0000 7f2cf134d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2cec0cc6b0 0x7f2cec14e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:56.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.283+0000 7f2cf134d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2cec15d510 0x7f2cec14ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:56.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.283+0000 7f2cf134d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2cec0c9180 con 0x7f2cec0c6a80 2024-04-25T10:30:56.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.283+0000 7f2cf134d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2cec0c7450 con 0x7f2cec15d510 2024-04-25T10:30:56.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.283+0000 7f2cf134d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f2cec0c9300 con 0x7f2cec0cc6b0 2024-04-25T10:30:56.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.283+0000 7f2ceaffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2cec0c6a80 0x7f2cec06c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:56.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.283+0000 7f2ceaffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2cec0c6a80 0x7f2cec06c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35004/0 (socket says 172.21.15.106:35004) 2024-04-25T10:30:56.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.283+0000 7f2ceaffd640 1 -- 172.21.15.106:0/2028232164 learned_addr learned my addr 172.21.15.106:0/2028232164 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:56.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.283+0000 7f2cea7fc640 1 --2- 172.21.15.106:0/2028232164 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2cec0cc6b0 0x7f2cec14e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:56.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.283+0000 7f2ceb7fe640 1 --2- 172.21.15.106:0/2028232164 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2cec15d510 0x7f2cec14ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:56.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.283+0000 7f2ceaffd640 1 -- 172.21.15.106:0/2028232164 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2cec0cc6b0 msgr2=0x7f2cec14e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:56.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.283+0000 7f2ceaffd640 1 --2- 172.21.15.106:0/2028232164 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2cec0cc6b0 0x7f2cec14e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:56.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.283+0000 7f2ceaffd640 1 -- 172.21.15.106:0/2028232164 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2cec15d510 msgr2=0x7f2cec14ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:56.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.283+0000 7f2ceaffd640 1 --2- 172.21.15.106:0/2028232164 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2cec15d510 0x7f2cec14ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:56.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.283+0000 7f2ceaffd640 1 -- 172.21.15.106:0/2028232164 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cec0ca5c0 con 0x7f2cec0c6a80 2024-04-25T10:30:56.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.283+0000 7f2ceaffd640 1 --2- 172.21.15.106:0/2028232164 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2cec0c6a80 0x7f2cec06c7a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc00c820 tx=0x7f2cdc00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:56.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.284+0000 7f2cd37fe640 1 -- 172.21.15.106:0/2028232164 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cdc017070 con 0x7f2cec0c6a80 2024-04-25T10:30:56.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.284+0000 7f2cd37fe640 1 -- 172.21.15.106:0/2028232164 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cdc003da0 con 0x7f2cec0c6a80 2024-04-25T10:30:56.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.284+0000 7f2cd37fe640 1 -- 172.21.15.106:0/2028232164 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cdc005290 con 0x7f2cec0c6a80 2024-04-25T10:30:56.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.284+0000 7f2cf134d640 1 -- 172.21.15.106:0/2028232164 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cec074110 con 0x7f2cec0c6a80 2024-04-25T10:30:56.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.284+0000 7f2cf134d640 1 -- 172.21.15.106:0/2028232164 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cec0743d0 con 0x7f2cec0c6a80 2024-04-25T10:30:56.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.285+0000 7f2cf134d640 1 -- 172.21.15.106:0/2028232164 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cec06a3d0 con 0x7f2cec0c6a80 2024-04-25T10:30:56.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.285+0000 7f2cd37fe640 1 -- 172.21.15.106:0/2028232164 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2cdc01a070 con 0x7f2cec0c6a80 2024-04-25T10:30:56.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.286+0000 7f2cd37fe640 1 --2- 172.21.15.106:0/2028232164 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f2cd80744b0 0x7f2cd80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:56.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.286+0000 7f2cea7fc640 1 --2- 172.21.15.106:0/2028232164 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f2cd80744b0 0x7f2cd80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:56.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.286+0000 7f2cd37fe640 1 -- 172.21.15.106:0/2028232164 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2cdc0947b0 con 0x7f2cec0c6a80 2024-04-25T10:30:56.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.286+0000 7f2cea7fc640 1 --2- 172.21.15.106:0/2028232164 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f2cd80744b0 0x7f2cd80769c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2ce0034000 tx=0x7f2ce002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:56.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.290+0000 7f2cd37fe640 1 -- 172.21.15.106:0/2028232164 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2cdc05e980 con 0x7f2cec0c6a80 2024-04-25T10:30:56.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.469+0000 7f2cf134d640 1 -- 172.21.15.106:0/2028232164 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2cec0ca000 con 0x7f2cec0c6a80 2024-04-25T10:30:56.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.471+0000 7f2cd37fe640 1 -- 172.21.15.106:0/2028232164 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16641 (secure 0 0 0) 0x7f2cdc062c80 con 0x7f2cec0c6a80 2024-04-25T10:30:56.472 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:30:56.472 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":20,"fsid":"3ee77985-e8e4-41cf-81e9-7c774cf7891f","created":"2024-04-25T10:30:07.981054+0000","modified":"2024-04-25T10:30:33.409267+0000","last_up_change":"2024-04-25T10:30:19.184325+0000","last_in_change":"2024-04-25T10:30:10.217553+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:30:20.111046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:30:30.314914+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:30:30.715500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a5962cb1-dbbd-47b1-8d15-9bae114bfdf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1240717079},{"type":"v1","addr":"172.21.15.106:6801","nonce":1240717079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1240717079},{"type":"v1","addr":"172.21.15.106:6803","nonce":1240717079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1240717079},{"type":"v1","addr":"172.21.15.106:6807","nonce":1240717079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1240717079},{"type":"v1","addr":"172.21.15.106:6805","nonce":1240717079}]},"public_addr":"172.21.15.106:6801/1240717079","cluster_addr":"172.21.15.106:6803/1240717079","heartbeat_back_addr":"172.21.15.106:6807/1240717079","heartbeat_front_addr":"172.21.15.106:6805/1240717079","state":["exists","up"]},{"osd":1,"uuid":"94436e13-4769-4752-8397-b5e396a6f7ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4089653051},{"type":"v1","addr":"172.21.15.106:6825","nonce":4089653051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4089653051},{"type":"v1","addr":"172.21.15.106:6827","nonce":4089653051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4089653051},{"type":"v1","addr":"172.21.15.106:6831","nonce":4089653051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4089653051},{"type":"v1","addr":"172.21.15.106:6829","nonce":4089653051}]},"public_addr":"172.21.15.106:6825/4089653051","cluster_addr":"172.21.15.106:6827/4089653051","heartbeat_back_addr":"172.21.15.106:6831/4089653051","heartbeat_front_addr":"172.21.15.106:6829/4089653051","state":["exists","up"]},{"osd":2,"uuid":"429aae0b-70c6-4e7b-9043-6a11334c7efc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":1441018239},{"type":"v1","addr":"172.21.15.106:6809","nonce":1441018239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1441018239},{"type":"v1","addr":"172.21.15.106:6811","nonce":1441018239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1441018239},{"type":"v1","addr":"172.21.15.106:6815","nonce":1441018239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":1441018239},{"type":"v1","addr":"172.21.15.106:6813","nonce":1441018239}]},"public_addr":"172.21.15.106:6809/1441018239","cluster_addr":"172.21.15.106:6811/1441018239","heartbeat_back_addr":"172.21.15.106:6815/1441018239","heartbeat_front_addr":"172.21.15.106:6813/1441018239","state":["exists","up"]},{"osd":3,"uuid":"9593a4d1-772f-4667-ab0d-2982f841e342","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":1846968711},{"type":"v1","addr":"172.21.15.106:6817","nonce":1846968711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1846968711},{"type":"v1","addr":"172.21.15.106:6819","nonce":1846968711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":1846968711},{"type":"v1","addr":"172.21.15.106:6823","nonce":1846968711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":1846968711},{"type":"v1","addr":"172.21.15.106:6821","nonce":1846968711}]},"public_addr":"172.21.15.106:6817/1846968711","cluster_addr":"172.21.15.106:6819/1846968711","heartbeat_back_addr":"172.21.15.106:6823/1846968711","heartbeat_front_addr":"172.21.15.106:6821/1846968711","state":["exists","up"]},{"osd":4,"uuid":"1b61f16d-5c60-480a-b449-c745aa3f6b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":729224739},{"type":"v1","addr":"172.21.15.134:6825","nonce":729224739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":729224739},{"type":"v1","addr":"172.21.15.134:6827","nonce":729224739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":729224739},{"type":"v1","addr":"172.21.15.134:6831","nonce":729224739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":729224739},{"type":"v1","addr":"172.21.15.134:6829","nonce":729224739}]},"public_addr":"172.21.15.134:6825/729224739","cluster_addr":"172.21.15.134:6827/729224739","heartbeat_back_addr":"172.21.15.134:6831/729224739","heartbeat_front_addr":"172.21.15.134:6829/729224739","state":["exists","up"]},{"osd":5,"uuid":"944cdd7f-47c0-4b71-b7e0-d093542f4528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3132275652},{"type":"v1","addr":"172.21.15.134:6809","nonce":3132275652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3132275652},{"type":"v1","addr":"172.21.15.134:6811","nonce":3132275652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3132275652},{"type":"v1","addr":"172.21.15.134:6815","nonce":3132275652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3132275652},{"type":"v1","addr":"172.21.15.134:6813","nonce":3132275652}]},"public_addr":"172.21.15.134:6809/3132275652","cluster_addr":"172.21.15.134:6811/3132275652","heartbeat_back_addr":"172.21.15.134:6815/3132275652","heartbeat_front_addr":"172.21.15.134:6813/3132275652","state":["exists","up"]},{"osd":6,"uuid":"cc2446e2-1d21-4135-ac04-6af7c990058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":992702668},{"type":"v1","addr":"172.21.15.134:6801","nonce":992702668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":992702668},{"type":"v1","addr":"172.21.15.134:6803","nonce":992702668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":992702668},{"type":"v1","addr":"172.21.15.134:6807","nonce":992702668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":992702668},{"type":"v1","addr":"172.21.15.134:6805","nonce":992702668}]},"public_addr":"172.21.15.134:6801/992702668","cluster_addr":"172.21.15.134:6803/992702668","heartbeat_back_addr":"172.21.15.134:6807/992702668","heartbeat_front_addr":"172.21.15.134:6805/992702668","state":["exists","up"]},{"osd":7,"uuid":"3df1fc7a-82fc-4257-8629-98027b6e2aed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":3291062284},{"type":"v1","addr":"172.21.15.134:6817","nonce":3291062284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":3291062284},{"type":"v1","addr":"172.21.15.134:6819","nonce":3291062284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":3291062284},{"type":"v1","addr":"172.21.15.134:6823","nonce":3291062284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":3291062284},{"type":"v1","addr":"172.21.15.134:6821","nonce":3291062284}]},"public_addr":"172.21.15.134:6817/3291062284","cluster_addr":"172.21.15.134:6819/3291062284","heartbeat_back_addr":"172.21.15.134:6823/3291062284","heartbeat_front_addr":"172.21.15.134:6821/3291062284","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.403941+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.382563+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.411544+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.330444+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.405430+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.387737+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.408922+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.454450+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:30:56.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.473+0000 7f2cf134d640 1 -- 172.21.15.106:0/2028232164 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f2cd80744b0 msgr2=0x7f2cd80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:56.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.473+0000 7f2cf134d640 1 --2- 172.21.15.106:0/2028232164 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f2cd80744b0 0x7f2cd80769c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2ce0034000 tx=0x7f2ce002e000 comp rx=0 tx=0).stop 2024-04-25T10:30:56.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.473+0000 7f2cf134d640 1 -- 172.21.15.106:0/2028232164 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2cec0c6a80 msgr2=0x7f2cec06c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:56.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.473+0000 7f2cf134d640 1 --2- 172.21.15.106:0/2028232164 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2cec0c6a80 0x7f2cec06c7a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc00c820 tx=0x7f2cdc00cce0 comp rx=0 tx=0).stop 2024-04-25T10:30:56.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.473+0000 7f2cf134d640 1 -- 172.21.15.106:0/2028232164 shutdown_connections 2024-04-25T10:30:56.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.473+0000 7f2cf134d640 1 --2- 172.21.15.106:0/2028232164 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f2cd80744b0 0x7f2cd80769c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:56.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.473+0000 7f2cf134d640 1 --2- 172.21.15.106:0/2028232164 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2cec15d510 0x7f2cec14ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:56.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.473+0000 7f2cf134d640 1 --2- 172.21.15.106:0/2028232164 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2cec0cc6b0 0x7f2cec14e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:56.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.473+0000 7f2cf134d640 1 --2- 172.21.15.106:0/2028232164 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2cec0c6a80 0x7f2cec06c7a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:56.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.473+0000 7f2cf134d640 1 -- 172.21.15.106:0/2028232164 >> 172.21.15.106:0/2028232164 conn(0x7f2cec06d6d0 msgr2=0x7f2cec0cab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:56.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.473+0000 7f2cf134d640 1 -- 172.21.15.106:0/2028232164 shutdown_connections 2024-04-25T10:30:56.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.473+0000 7f2cf134d640 1 -- 172.21.15.106:0/2028232164 wait complete. 2024-04-25T10:30:56.485 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:30:20.111046+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:30:30.314914+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:30:30.715500+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:30:56.486 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:30:56.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.596+0000 7efca89ba640 1 Processor -- start 2024-04-25T10:30:56.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.596+0000 7efca89ba640 1 -- start start 2024-04-25T10:30:56.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.597+0000 7efca89ba640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7efca40c7e30 0x7efca40c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:56.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.597+0000 7efca89ba640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7efca4156ef0 0x7efca40b4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:56.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.597+0000 7efca89ba640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efca40ce630 con 0x7efca406d000 2024-04-25T10:30:56.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.597+0000 7efca89ba640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7efca40c7c60 con 0x7efca40c7e30 2024-04-25T10:30:56.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.597+0000 7efca89ba640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7efca40c7980 con 0x7efca4156ef0 2024-04-25T10:30:56.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.597+0000 7efca259c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7efca40c7e30 0x7efca40c8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:56.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.597+0000 7efca259c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7efca40c7e30 0x7efca40c8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35014/0 (socket says 172.21.15.106:35014) 2024-04-25T10:30:56.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.597+0000 7efca259c640 1 -- 172.21.15.106:0/1686688162 learned_addr learned my addr 172.21.15.106:0/1686688162 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:56.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.597+0000 7efca2d9d640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7efca406d000 0x7efca415ce30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:53508/0 (socket says 172.21.15.106:53508) 2024-04-25T10:30:56.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.598+0000 7efca1d9b640 1 --2- 172.21.15.106:0/1686688162 >> v2:172.21.15.134:3300/0 conn(0x7efca4156ef0 0x7efca40b4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:56.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.598+0000 7efca259c640 1 -- 172.21.15.106:0/1686688162 >> v2:172.21.15.134:3300/0 conn(0x7efca4156ef0 msgr2=0x7efca40b4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:56.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.598+0000 7efca259c640 1 --2- 172.21.15.106:0/1686688162 >> v2:172.21.15.134:3300/0 conn(0x7efca4156ef0 0x7efca40b4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:56.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.598+0000 7efca259c640 1 -- 172.21.15.106:0/1686688162 >> v1:172.21.15.106:6789/0 conn(0x7efca406d000 legacy=0x7efca415ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:56.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.598+0000 7efca159a640 1 -- 172.21.15.106:0/1686688162 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4152596079 0 0) 0x7efca40ce630 con 0x7efca406d000 2024-04-25T10:30:56.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.598+0000 7efca259c640 1 -- 172.21.15.106:0/1686688162 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efca40c7010 con 0x7efca40c7e30 2024-04-25T10:30:56.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.598+0000 7efca259c640 1 --2- 172.21.15.106:0/1686688162 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efca40c7e30 0x7efca40c8d90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7efca40c7de0 tx=0x7efc9402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d07242b1b92b821c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:56.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.599+0000 7efca159a640 1 -- 172.21.15.106:0/1686688162 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc94037070 con 0x7efca40c7e30 2024-04-25T10:30:56.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.599+0000 7efca159a640 1 -- 172.21.15.106:0/1686688162 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc94032440 con 0x7efca40c7e30 2024-04-25T10:30:56.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.599+0000 7efca159a640 1 -- 172.21.15.106:0/1686688162 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc94032810 con 0x7efca40c7e30 2024-04-25T10:30:56.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.599+0000 7efca89ba640 1 -- 172.21.15.106:0/1686688162 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efca40c7e30 msgr2=0x7efca40c8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:56.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.599+0000 7efca89ba640 1 --2- 172.21.15.106:0/1686688162 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efca40c7e30 0x7efca40c8d90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7efca40c7de0 tx=0x7efc9402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:30:56.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.599+0000 7efca89ba640 1 -- 172.21.15.106:0/1686688162 shutdown_connections 2024-04-25T10:30:56.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.599+0000 7efca89ba640 1 --2- 172.21.15.106:0/1686688162 >> v2:172.21.15.134:3300/0 conn(0x7efca4156ef0 0x7efca40b4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:56.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.599+0000 7efca89ba640 1 --2- 172.21.15.106:0/1686688162 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efca40c7e30 0x7efca40c8d90 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:56.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.599+0000 7efca89ba640 1 -- 172.21.15.106:0/1686688162 >> 172.21.15.106:0/1686688162 conn(0x7efca4068b10 msgr2=0x7efca40d1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:56.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.599+0000 7efca89ba640 1 -- 172.21.15.106:0/1686688162 shutdown_connections 2024-04-25T10:30:56.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.599+0000 7efca89ba640 1 -- 172.21.15.106:0/1686688162 wait complete. 2024-04-25T10:30:56.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.601+0000 7efca89ba640 1 Processor -- start 2024-04-25T10:30:56.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.601+0000 7efca89ba640 1 -- start start 2024-04-25T10:30:56.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.602+0000 7efca89ba640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efca406d000 0x7efca4152150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:56.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.602+0000 7efca89ba640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efca40c7e30 0x7efca41f65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:56.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.602+0000 7efca89ba640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efca4156ef0 0x7efca415a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:56.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.602+0000 7efca89ba640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7efca415d6c0 con 0x7efca4156ef0 2024-04-25T10:30:56.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.602+0000 7efca89ba640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efca415d540 con 0x7efca40c7e30 2024-04-25T10:30:56.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.602+0000 7efca89ba640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7efca41f6f60 con 0x7efca406d000 2024-04-25T10:30:56.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.602+0000 7efca2d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efca4156ef0 0x7efca415a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:56.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.602+0000 7efca2d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efca4156ef0 0x7efca415a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35026/0 (socket says 172.21.15.106:35026) 2024-04-25T10:30:56.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.603+0000 7efca2d9d640 1 -- 172.21.15.106:0/3326748011 learned_addr learned my addr 172.21.15.106:0/3326748011 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:56.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.603+0000 7efca1d9b640 1 --2- 172.21.15.106:0/3326748011 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efca40c7e30 0x7efca41f65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:56.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.603+0000 7efca259c640 1 --2- 172.21.15.106:0/3326748011 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efca406d000 0x7efca4152150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:56.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.603+0000 7efca2d9d640 1 -- 172.21.15.106:0/3326748011 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efca406d000 msgr2=0x7efca4152150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:56.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.603+0000 7efca2d9d640 1 --2- 172.21.15.106:0/3326748011 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efca406d000 0x7efca4152150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:56.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.603+0000 7efca2d9d640 1 -- 172.21.15.106:0/3326748011 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efca40c7e30 msgr2=0x7efca41f65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:56.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.603+0000 7efca2d9d640 1 --2- 172.21.15.106:0/3326748011 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efca40c7e30 0x7efca41f65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:56.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.603+0000 7efca2d9d640 1 -- 172.21.15.106:0/3326748011 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efca415a360 con 0x7efca4156ef0 2024-04-25T10:30:56.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.603+0000 7efca259c640 1 --2- 172.21.15.106:0/3326748011 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efca406d000 0x7efca4152150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:56.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.603+0000 7efca2d9d640 1 --2- 172.21.15.106:0/3326748011 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efca4156ef0 0x7efca415a9a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7efc9c00c730 tx=0x7efc9c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:56.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.603+0000 7efc92ffd640 1 -- 172.21.15.106:0/3326748011 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc9c018070 con 0x7efca4156ef0 2024-04-25T10:30:56.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.603+0000 7efc92ffd640 1 -- 172.21.15.106:0/3326748011 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc9c00ce40 con 0x7efca4156ef0 2024-04-25T10:30:56.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.603+0000 7efca89ba640 1 -- 172.21.15.106:0/3326748011 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efca4159da0 con 0x7efca4156ef0 2024-04-25T10:30:56.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.603+0000 7efc92ffd640 1 -- 172.21.15.106:0/3326748011 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc9c00a800 con 0x7efca4156ef0 2024-04-25T10:30:56.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.603+0000 7efca89ba640 1 -- 172.21.15.106:0/3326748011 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efca40d05c0 con 0x7efca4156ef0 2024-04-25T10:30:56.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.604+0000 7efca89ba640 1 -- 172.21.15.106:0/3326748011 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efca406a3d0 con 0x7efca4156ef0 2024-04-25T10:30:56.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.605+0000 7efc92ffd640 1 -- 172.21.15.106:0/3326748011 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7efc9c01b070 con 0x7efca4156ef0 2024-04-25T10:30:56.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.605+0000 7efc92ffd640 1 --2- 172.21.15.106:0/3326748011 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7efc840744b0 0x7efc840769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:56.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.605+0000 7efc92ffd640 1 -- 172.21.15.106:0/3326748011 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7efc9c094720 con 0x7efca4156ef0 2024-04-25T10:30:56.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.605+0000 7efca259c640 1 --2- 172.21.15.106:0/3326748011 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7efc840744b0 0x7efc840769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:56.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.606+0000 7efca259c640 1 --2- 172.21.15.106:0/3326748011 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7efc840744b0 0x7efc840769c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7efc94034000 tx=0x7efc9402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:56.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.610+0000 7efc92ffd640 1 -- 172.21.15.106:0/3326748011 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efc9c05e860 con 0x7efca4156ef0 2024-04-25T10:30:56.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.783+0000 7efca89ba640 1 -- 172.21.15.106:0/3326748011 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7efca415a5a0 con 0x7efca4156ef0 2024-04-25T10:30:56.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.784+0000 7efc92ffd640 1 -- 172.21.15.106:0/3326748011 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7efc9c062b60 con 0x7efca4156ef0 2024-04-25T10:30:56.785 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-25T10:30:56.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.786+0000 7efca89ba640 1 -- 172.21.15.106:0/3326748011 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7efc840744b0 msgr2=0x7efc840769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:56.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.786+0000 7efca89ba640 1 --2- 172.21.15.106:0/3326748011 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7efc840744b0 0x7efc840769c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7efc94034000 tx=0x7efc9402e000 comp rx=0 tx=0).stop 2024-04-25T10:30:56.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.786+0000 7efca89ba640 1 -- 172.21.15.106:0/3326748011 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efca4156ef0 msgr2=0x7efca415a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:56.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.786+0000 7efca89ba640 1 --2- 172.21.15.106:0/3326748011 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efca4156ef0 0x7efca415a9a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7efc9c00c730 tx=0x7efc9c00cbf0 comp rx=0 tx=0).stop 2024-04-25T10:30:56.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.786+0000 7efca89ba640 1 -- 172.21.15.106:0/3326748011 shutdown_connections 2024-04-25T10:30:56.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.786+0000 7efca89ba640 1 --2- 172.21.15.106:0/3326748011 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7efc840744b0 0x7efc840769c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:56.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.786+0000 7efca89ba640 1 --2- 172.21.15.106:0/3326748011 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efca4156ef0 0x7efca415a9a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:56.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.786+0000 7efca89ba640 1 --2- 172.21.15.106:0/3326748011 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efca40c7e30 0x7efca41f65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:56.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.786+0000 7efca89ba640 1 --2- 172.21.15.106:0/3326748011 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efca406d000 0x7efca4152150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:56.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.786+0000 7efca89ba640 1 -- 172.21.15.106:0/3326748011 >> 172.21.15.106:0/3326748011 conn(0x7efca4068b10 msgr2=0x7efca4155290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:56.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.786+0000 7efca89ba640 1 -- 172.21.15.106:0/3326748011 shutdown_connections 2024-04-25T10:30:56.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.786+0000 7efca89ba640 1 -- 172.21.15.106:0/3326748011 wait complete. 2024-04-25T10:30:56.798 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:30:56.817 INFO:tasks.ceph.mds.d.smithi134.stderr:2024-04-25T10:30:56.814+0000 7f3ef08bf640 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-04-25T10:30:56.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.906+0000 7f2e37dd7640 1 Processor -- start 2024-04-25T10:30:56.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.906+0000 7f2e37dd7640 1 -- start start 2024-04-25T10:30:56.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.908+0000 7f2e37dd7640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f2e3015d7c0 0x7f2e300c61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:56.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.908+0000 7f2e37dd7640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2e30156ec0 0x7f2e300ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:56.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.908+0000 7f2e37dd7640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e301572d0 con 0x7f2e3006dc00 2024-04-25T10:30:56.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.908+0000 7f2e37dd7640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f2e3015dbd0 con 0x7f2e30156ec0 2024-04-25T10:30:56.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.908+0000 7f2e37dd7640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f2e30160ae0 con 0x7f2e3015d7c0 2024-04-25T10:30:56.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.908+0000 7f2e36374640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2e30156ec0 0x7f2e300ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:56.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.908+0000 7f2e36374640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2e30156ec0 0x7f2e300ccde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35040/0 (socket says 172.21.15.106:35040) 2024-04-25T10:30:56.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.908+0000 7f2e35b73640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f2e3006dc00 0x7f2e301601c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.106:41172/0 (socket says 172.21.15.106:41172) 2024-04-25T10:30:56.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.908+0000 7f2e36374640 1 -- 172.21.15.106:0/2830046611 learned_addr learned my addr 172.21.15.106:0/2830046611 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:56.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.908+0000 7f2e35372640 1 --2- 172.21.15.106:0/2830046611 >> v2:172.21.15.134:3300/0 conn(0x7f2e3015d7c0 0x7f2e300c61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:56.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.908+0000 7f2e36374640 1 -- 172.21.15.106:0/2830046611 >> v2:172.21.15.134:3300/0 conn(0x7f2e3015d7c0 msgr2=0x7f2e300c61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:56.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.908+0000 7f2e36374640 1 --2- 172.21.15.106:0/2830046611 >> v2:172.21.15.134:3300/0 conn(0x7f2e3015d7c0 0x7f2e300c61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:56.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.908+0000 7f2e34b71640 1 -- 172.21.15.106:0/2830046611 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 82374354 0 0) 0x7f2e301572d0 con 0x7f2e3006dc00 2024-04-25T10:30:56.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.908+0000 7f2e36374640 1 -- 172.21.15.106:0/2830046611 >> v1:172.21.15.134:6789/0 conn(0x7f2e3006dc00 legacy=0x7f2e301601c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:56.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.908+0000 7f2e36374640 1 -- 172.21.15.106:0/2830046611 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e30160dc0 con 0x7f2e30156ec0 2024-04-25T10:30:56.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.909+0000 7f2e36374640 1 --2- 172.21.15.106:0/2830046611 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2e30156ec0 0x7f2e300ccde0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2e300cd980 tx=0x7f2e2802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d3ac6f2e6cd310f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:56.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.909+0000 7f2e34b71640 1 -- 172.21.15.106:0/2830046611 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e28037070 con 0x7f2e30156ec0 2024-04-25T10:30:56.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.909+0000 7f2e34b71640 1 -- 172.21.15.106:0/2830046611 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e2802ae50 con 0x7f2e30156ec0 2024-04-25T10:30:56.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.909+0000 7f2e34b71640 1 -- 172.21.15.106:0/2830046611 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e28032860 con 0x7f2e30156ec0 2024-04-25T10:30:56.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.910+0000 7f2e37dd7640 1 -- 172.21.15.106:0/2830046611 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2e30156ec0 msgr2=0x7f2e300ccde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:56.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.910+0000 7f2e37dd7640 1 --2- 172.21.15.106:0/2830046611 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2e30156ec0 0x7f2e300ccde0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2e300cd980 tx=0x7f2e2802a300 comp rx=0 tx=0).stop 2024-04-25T10:30:56.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.910+0000 7f2e37dd7640 1 -- 172.21.15.106:0/2830046611 shutdown_connections 2024-04-25T10:30:56.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.910+0000 7f2e37dd7640 1 --2- 172.21.15.106:0/2830046611 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2e30156ec0 0x7f2e300ccde0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:56.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.910+0000 7f2e37dd7640 1 --2- 172.21.15.106:0/2830046611 >> v2:172.21.15.134:3300/0 conn(0x7f2e3015d7c0 0x7f2e300c61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:56.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.910+0000 7f2e37dd7640 1 -- 172.21.15.106:0/2830046611 >> 172.21.15.106:0/2830046611 conn(0x7f2e300727d0 msgr2=0x7f2e30072c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:56.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.910+0000 7f2e37dd7640 1 -- 172.21.15.106:0/2830046611 shutdown_connections 2024-04-25T10:30:56.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.910+0000 7f2e37dd7640 1 -- 172.21.15.106:0/2830046611 wait complete. 2024-04-25T10:30:56.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.911+0000 7f2e37dd7640 1 Processor -- start 2024-04-25T10:30:56.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.911+0000 7f2e37dd7640 1 -- start start 2024-04-25T10:30:56.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.913+0000 7f2e37dd7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2e3006dc00 0x7f2e300c8da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:56.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.913+0000 7f2e37dd7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e30156ec0 0x7f2e30150830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:56.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.913+0000 7f2e37dd7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2e3015d7c0 0x7f2e30150d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:56.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.913+0000 7f2e37dd7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2e3015cfc0 con 0x7f2e3015d7c0 2024-04-25T10:30:56.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.913+0000 7f2e37dd7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2e3015cce0 con 0x7f2e30156ec0 2024-04-25T10:30:56.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.913+0000 7f2e37dd7640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f2e3015d2a0 con 0x7f2e3006dc00 2024-04-25T10:30:56.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.913+0000 7f2e36374640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2e3015d7c0 0x7f2e30150d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:56.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.913+0000 7f2e36374640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2e3015d7c0 0x7f2e30150d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35050/0 (socket says 172.21.15.106:35050) 2024-04-25T10:30:56.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.913+0000 7f2e36374640 1 -- 172.21.15.106:0/2715177400 learned_addr learned my addr 172.21.15.106:0/2715177400 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:56.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.913+0000 7f2e35b73640 1 --2- 172.21.15.106:0/2715177400 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2e3006dc00 0x7f2e300c8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:56.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.913+0000 7f2e35372640 1 --2- 172.21.15.106:0/2715177400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e30156ec0 0x7f2e30150830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:56.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.913+0000 7f2e36374640 1 -- 172.21.15.106:0/2715177400 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2e3006dc00 msgr2=0x7f2e300c8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:56.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.913+0000 7f2e36374640 1 --2- 172.21.15.106:0/2715177400 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2e3006dc00 0x7f2e300c8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:56.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.913+0000 7f2e36374640 1 -- 172.21.15.106:0/2715177400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e30156ec0 msgr2=0x7f2e30150830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:56.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.913+0000 7f2e36374640 1 --2- 172.21.15.106:0/2715177400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e30156ec0 0x7f2e30150830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:56.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.913+0000 7f2e36374640 1 -- 172.21.15.106:0/2715177400 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e300c92e0 con 0x7f2e3015d7c0 2024-04-25T10:30:56.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.913+0000 7f2e36374640 1 --2- 172.21.15.106:0/2715177400 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2e3015d7c0 0x7f2e30150d70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2e280090b0 tx=0x7f2e280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:56.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.913+0000 7f2e267fc640 1 -- 172.21.15.106:0/2715177400 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e28041070 con 0x7f2e3015d7c0 2024-04-25T10:30:56.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.913+0000 7f2e267fc640 1 -- 172.21.15.106:0/2715177400 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e2803c660 con 0x7f2e3015d7c0 2024-04-25T10:30:56.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.913+0000 7f2e267fc640 1 -- 172.21.15.106:0/2715177400 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e28037040 con 0x7f2e3015d7c0 2024-04-25T10:30:56.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.913+0000 7f2e37dd7640 1 -- 172.21.15.106:0/2715177400 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e3015ab20 con 0x7f2e3015d7c0 2024-04-25T10:30:56.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.913+0000 7f2e37dd7640 1 -- 172.21.15.106:0/2715177400 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e300c7e50 con 0x7f2e3015d7c0 2024-04-25T10:30:56.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.914+0000 7f2e37dd7640 1 -- 172.21.15.106:0/2715177400 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e3006a3d0 con 0x7f2e3015d7c0 2024-04-25T10:30:56.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.915+0000 7f2e267fc640 1 -- 172.21.15.106:0/2715177400 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2e28044070 con 0x7f2e3015d7c0 2024-04-25T10:30:56.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.915+0000 7f2e267fc640 1 --2- 172.21.15.106:0/2715177400 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f2e140744b0 0x7f2e140769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:56.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.915+0000 7f2e35b73640 1 --2- 172.21.15.106:0/2715177400 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f2e140744b0 0x7f2e140769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:56.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.915+0000 7f2e267fc640 1 -- 172.21.15.106:0/2715177400 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2e280b5910 con 0x7f2e3015d7c0 2024-04-25T10:30:56.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.916+0000 7f2e35b73640 1 --2- 172.21.15.106:0/2715177400 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f2e140744b0 0x7f2e140769c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c0012e0 tx=0x7f2e2c001630 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:56.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:56.920+0000 7f2e267fc640 1 -- 172.21.15.106:0/2715177400 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2e2807fae0 con 0x7f2e3015d7c0 2024-04-25T10:30:57.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.096+0000 7f2e37dd7640 1 -- 172.21.15.106:0/2715177400 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2e300cbdf0 con 0x7f2e3015d7c0 2024-04-25T10:30:57.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.097+0000 7f2e267fc640 1 -- 172.21.15.106:0/2715177400 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f2e28088120 con 0x7f2e3015d7c0 2024-04-25T10:30:57.098 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-25T10:30:57.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.098+0000 7f2e37dd7640 1 -- 172.21.15.106:0/2715177400 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f2e140744b0 msgr2=0x7f2e140769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:57.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.098+0000 7f2e37dd7640 1 --2- 172.21.15.106:0/2715177400 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f2e140744b0 0x7f2e140769c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c0012e0 tx=0x7f2e2c001630 comp rx=0 tx=0).stop 2024-04-25T10:30:57.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.098+0000 7f2e37dd7640 1 -- 172.21.15.106:0/2715177400 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2e3015d7c0 msgr2=0x7f2e30150d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:57.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.098+0000 7f2e37dd7640 1 --2- 172.21.15.106:0/2715177400 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2e3015d7c0 0x7f2e30150d70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2e280090b0 tx=0x7f2e280046c0 comp rx=0 tx=0).stop 2024-04-25T10:30:57.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.099+0000 7f2e37dd7640 1 -- 172.21.15.106:0/2715177400 shutdown_connections 2024-04-25T10:30:57.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.099+0000 7f2e37dd7640 1 --2- 172.21.15.106:0/2715177400 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f2e140744b0 0x7f2e140769c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:57.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.099+0000 7f2e37dd7640 1 --2- 172.21.15.106:0/2715177400 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2e3015d7c0 0x7f2e30150d70 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:57.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.099+0000 7f2e37dd7640 1 --2- 172.21.15.106:0/2715177400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e30156ec0 0x7f2e30150830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:57.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.099+0000 7f2e37dd7640 1 --2- 172.21.15.106:0/2715177400 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2e3006dc00 0x7f2e300c8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:57.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.099+0000 7f2e37dd7640 1 -- 172.21.15.106:0/2715177400 >> 172.21.15.106:0/2715177400 conn(0x7f2e300727d0 msgr2=0x7f2e301514d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:57.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.099+0000 7f2e37dd7640 1 -- 172.21.15.106:0/2715177400 shutdown_connections 2024-04-25T10:30:57.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.099+0000 7f2e37dd7640 1 -- 172.21.15.106:0/2715177400 wait complete. 2024-04-25T10:30:57.111 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:30:57.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.220+0000 7feed6cf9640 1 Processor -- start 2024-04-25T10:30:57.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.220+0000 7feed6cf9640 1 -- start start 2024-04-25T10:30:57.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.221+0000 7feed6cf9640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7feed0156f70 0x7feed015ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:57.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.221+0000 7feed6cf9640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feed0073980 con 0x7feed00ca170 2024-04-25T10:30:57.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.221+0000 7feed6cf9640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feed0160ea0 con 0x7feed015d340 2024-04-25T10:30:57.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.221+0000 7feed6cf9640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7feed0073f00 con 0x7feed0156f70 2024-04-25T10:30:57.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.222+0000 7feed4a95640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7feed00ca170 0x7feed006c5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:53512/0 (socket says 172.21.15.106:53512) 2024-04-25T10:30:57.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.222+0000 7feed4a95640 1 -- 172.21.15.106:0/3246894755 learned_addr learned my addr 172.21.15.106:0/3246894755 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:30:57.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.222+0000 7feecffff640 1 --2- 172.21.15.106:0/3246894755 >> v2:172.21.15.134:3300/0 conn(0x7feed0156f70 0x7feed015ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:57.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.222+0000 7feecf7fe640 1 -- 172.21.15.106:0/3246894755 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3294778588 0 0) 0x7feed0073980 con 0x7feed00ca170 2024-04-25T10:30:57.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.222+0000 7feecf7fe640 1 -- 172.21.15.106:0/3246894755 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feeb8003520 con 0x7feed00ca170 2024-04-25T10:30:57.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.222+0000 7feecffff640 1 -- 172.21.15.106:0/3246894755 >> v1:172.21.15.134:6789/0 conn(0x7feed015d340 legacy=0x7feed0167680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:57.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.222+0000 7feecf7fe640 1 -- 172.21.15.106:0/3246894755 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3194175322 0 0) 0x7feed0160ea0 con 0x7feed015d340 2024-04-25T10:30:57.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.222+0000 7feecffff640 1 -- 172.21.15.106:0/3246894755 >> v1:172.21.15.106:6789/0 conn(0x7feed00ca170 legacy=0x7feed006c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:57.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.222+0000 7feecffff640 1 -- 172.21.15.106:0/3246894755 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feed015d750 con 0x7feed0156f70 2024-04-25T10:30:57.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.223+0000 7feecffff640 1 --2- 172.21.15.106:0/3246894755 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feed0156f70 0x7feed015ce00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7feec40097a0 tx=0x7feec402a2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd35d3efbd06e93e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:57.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.223+0000 7feecf7fe640 1 -- 172.21.15.106:0/3246894755 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec4037070 con 0x7feed0156f70 2024-04-25T10:30:57.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.223+0000 7feecf7fe640 1 -- 172.21.15.106:0/3246894755 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feec4032640 con 0x7feed0156f70 2024-04-25T10:30:57.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.223+0000 7feecf7fe640 1 -- 172.21.15.106:0/3246894755 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec4032a10 con 0x7feed0156f70 2024-04-25T10:30:57.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.223+0000 7feed6cf9640 1 -- 172.21.15.106:0/3246894755 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feed0156f70 msgr2=0x7feed015ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:57.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.223+0000 7feed6cf9640 1 --2- 172.21.15.106:0/3246894755 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feed0156f70 0x7feed015ce00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7feec40097a0 tx=0x7feec402a2c0 comp rx=0 tx=0).stop 2024-04-25T10:30:57.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.223+0000 7feed6cf9640 1 -- 172.21.15.106:0/3246894755 shutdown_connections 2024-04-25T10:30:57.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.223+0000 7feed6cf9640 1 --2- 172.21.15.106:0/3246894755 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feed0156f70 0x7feed015ce00 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:57.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.223+0000 7feed6cf9640 1 -- 172.21.15.106:0/3246894755 >> 172.21.15.106:0/3246894755 conn(0x7feed00cef20 msgr2=0x7feed00cf330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:57.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.224+0000 7feed6cf9640 1 -- 172.21.15.106:0/3246894755 shutdown_connections 2024-04-25T10:30:57.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.224+0000 7feed6cf9640 1 -- 172.21.15.106:0/3246894755 wait complete. 2024-04-25T10:30:57.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.225+0000 7feed6cf9640 1 Processor -- start 2024-04-25T10:30:57.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.225+0000 7feed6cf9640 1 -- start start 2024-04-25T10:30:57.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.226+0000 7feed6cf9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feed00ca170 0x7feed01f2f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:57.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.227+0000 7feed6cf9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feed0156f70 0x7feed01512d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:57.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.227+0000 7feed6cf9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feed015d340 0x7feed013e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:57.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.227+0000 7feed6cf9640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7feed0073c20 con 0x7feed015d340 2024-04-25T10:30:57.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.227+0000 7feed6cf9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7feed0074990 con 0x7feed0156f70 2024-04-25T10:30:57.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.227+0000 7feed6cf9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7feed0161140 con 0x7feed00ca170 2024-04-25T10:30:57.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.227+0000 7feed5296640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feed015d340 0x7feed013e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:57.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.227+0000 7feed5296640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feed015d340 0x7feed013e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35066/0 (socket says 172.21.15.106:35066) 2024-04-25T10:30:57.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.227+0000 7feed5296640 1 -- 172.21.15.106:0/945528239 learned_addr learned my addr 172.21.15.106:0/945528239 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:57.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.227+0000 7feecffff640 1 --2- 172.21.15.106:0/945528239 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feed0156f70 0x7feed01512d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:57.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.227+0000 7feed4a95640 1 --2- 172.21.15.106:0/945528239 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feed00ca170 0x7feed01f2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:57.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.227+0000 7feed5296640 1 -- 172.21.15.106:0/945528239 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feed00ca170 msgr2=0x7feed01f2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:57.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.227+0000 7feed5296640 1 --2- 172.21.15.106:0/945528239 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feed00ca170 0x7feed01f2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:57.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.227+0000 7feed5296640 1 -- 172.21.15.106:0/945528239 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feed0156f70 msgr2=0x7feed01512d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:57.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.227+0000 7feed5296640 1 --2- 172.21.15.106:0/945528239 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feed0156f70 0x7feed01512d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:57.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.227+0000 7feed5296640 1 -- 172.21.15.106:0/945528239 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feed00ce9b0 con 0x7feed015d340 2024-04-25T10:30:57.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.227+0000 7feecffff640 1 --2- 172.21.15.106:0/945528239 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feed0156f70 0x7feed01512d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:57.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.227+0000 7feed5296640 1 --2- 172.21.15.106:0/945528239 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feed015d340 0x7feed013e300 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7feec800c740 tx=0x7feec800cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:57.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.227+0000 7feecd7fa640 1 -- 172.21.15.106:0/945528239 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec8017070 con 0x7feed015d340 2024-04-25T10:30:57.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.227+0000 7feecd7fa640 1 -- 172.21.15.106:0/945528239 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feec800ce50 con 0x7feed015d340 2024-04-25T10:30:57.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.227+0000 7feed6cf9640 1 -- 172.21.15.106:0/945528239 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feed015e060 con 0x7feed015d340 2024-04-25T10:30:57.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.227+0000 7feecd7fa640 1 -- 172.21.15.106:0/945528239 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec8005070 con 0x7feed015d340 2024-04-25T10:30:57.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.227+0000 7feed6cf9640 1 -- 172.21.15.106:0/945528239 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feed015e620 con 0x7feed015d340 2024-04-25T10:30:57.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.228+0000 7feed6cf9640 1 -- 172.21.15.106:0/945528239 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feed00c9a40 con 0x7feed015d340 2024-04-25T10:30:57.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.229+0000 7feecd7fa640 1 -- 172.21.15.106:0/945528239 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7feec801a070 con 0x7feed015d340 2024-04-25T10:30:57.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.229+0000 7feecd7fa640 1 --2- 172.21.15.106:0/945528239 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7feebc0744b0 0x7feebc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:57.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.230+0000 7feed4a95640 1 --2- 172.21.15.106:0/945528239 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7feebc0744b0 0x7feebc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:57.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.230+0000 7feecd7fa640 1 -- 172.21.15.106:0/945528239 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7feec80948d0 con 0x7feed015d340 2024-04-25T10:30:57.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.230+0000 7feed4a95640 1 --2- 172.21.15.106:0/945528239 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7feebc0744b0 0x7feebc0769c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7feec0001200 tx=0x7feec0000bc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:57.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.234+0000 7feecd7fa640 1 -- 172.21.15.106:0/945528239 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7feed00c9a40 con 0x7feed015d340 2024-04-25T10:30:57.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.410+0000 7feed6cf9640 1 -- 172.21.15.106:0/945528239 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7feed01503b0 con 0x7feed015d340 2024-04-25T10:30:57.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.410+0000 7feecd7fa640 1 -- 172.21.15.106:0/945528239 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7feed01503b0 con 0x7feed015d340 2024-04-25T10:30:57.412 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-25T10:30:57.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.412+0000 7feed6cf9640 1 -- 172.21.15.106:0/945528239 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7feebc0744b0 msgr2=0x7feebc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:57.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.412+0000 7feed6cf9640 1 --2- 172.21.15.106:0/945528239 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7feebc0744b0 0x7feebc0769c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7feec0001200 tx=0x7feec0000bc0 comp rx=0 tx=0).stop 2024-04-25T10:30:57.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.412+0000 7feed6cf9640 1 -- 172.21.15.106:0/945528239 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feed015d340 msgr2=0x7feed013e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:57.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.412+0000 7feed6cf9640 1 --2- 172.21.15.106:0/945528239 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feed015d340 0x7feed013e300 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7feec800c740 tx=0x7feec800cc00 comp rx=0 tx=0).stop 2024-04-25T10:30:57.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.412+0000 7feed6cf9640 1 -- 172.21.15.106:0/945528239 shutdown_connections 2024-04-25T10:30:57.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.412+0000 7feed6cf9640 1 --2- 172.21.15.106:0/945528239 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7feebc0744b0 0x7feebc0769c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:57.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.412+0000 7feed6cf9640 1 --2- 172.21.15.106:0/945528239 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feed015d340 0x7feed013e300 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:57.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.412+0000 7feed6cf9640 1 --2- 172.21.15.106:0/945528239 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feed0156f70 0x7feed01512d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:57.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.412+0000 7feed6cf9640 1 --2- 172.21.15.106:0/945528239 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feed00ca170 0x7feed01f2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:57.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.412+0000 7feed6cf9640 1 -- 172.21.15.106:0/945528239 >> 172.21.15.106:0/945528239 conn(0x7feed00cef20 msgr2=0x7feed00c8700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:57.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.412+0000 7feed6cf9640 1 -- 172.21.15.106:0/945528239 shutdown_connections 2024-04-25T10:30:57.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.412+0000 7feed6cf9640 1 -- 172.21.15.106:0/945528239 wait complete. 2024-04-25T10:30:57.425 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:30:57.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.530+0000 7fb4fae70640 1 Processor -- start 2024-04-25T10:30:57.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.530+0000 7fb4fae70640 1 -- start start 2024-04-25T10:30:57.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.531+0000 7fb4fae70640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb4f4068790 0x7fb4f4156f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:57.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.531+0000 7fb4fae70640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4f40c8f70 0x7fb4f40c9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:57.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.531+0000 7fb4fae70640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4f4073320 con 0x7fb4f415d4a0 2024-04-25T10:30:57.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.531+0000 7fb4fae70640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fb4f4074160 con 0x7fb4f4068790 2024-04-25T10:30:57.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.531+0000 7fb4fae70640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb4f4074420 con 0x7fb4f40c8f70 2024-04-25T10:30:57.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.531+0000 7fb4f8c0c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb4f4068790 0x7fb4f4156f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:57.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.531+0000 7fb4f940d640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fb4f415d4a0 0x7fb4f40708a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:53514/0 (socket says 172.21.15.106:53514) 2024-04-25T10:30:57.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.531+0000 7fb4f940d640 1 -- 172.21.15.106:0/812364467 learned_addr learned my addr 172.21.15.106:0/812364467 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:30:57.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.532+0000 7fb4f3fff640 1 --2- 172.21.15.106:0/812364467 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4f40c8f70 0x7fb4f40c9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:57.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.532+0000 7fb4f8c0c640 1 -- 172.21.15.106:0/812364467 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4f40c8f70 msgr2=0x7fb4f40c9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:57.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.532+0000 7fb4f8c0c640 1 --2- 172.21.15.106:0/812364467 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4f40c8f70 0x7fb4f40c9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:57.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.532+0000 7fb4f8c0c640 1 -- 172.21.15.106:0/812364467 >> v1:172.21.15.106:6789/0 conn(0x7fb4f415d4a0 legacy=0x7fb4f40708a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:57.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.532+0000 7fb4f37fe640 1 -- 172.21.15.106:0/812364467 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2860691294 0 0) 0x7fb4ec001490 con 0x7fb4f415d4a0 2024-04-25T10:30:57.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.532+0000 7fb4f8c0c640 1 -- 172.21.15.106:0/812364467 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4f4074700 con 0x7fb4f4068790 2024-04-25T10:30:57.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.532+0000 7fb4f8c0c640 1 --2- 172.21.15.106:0/812364467 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb4f4068790 0x7fb4f4156f10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb4e80097a0 tx=0x7fb4e802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8a405f24e78a0c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:57.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.533+0000 7fb4f37fe640 1 -- 172.21.15.106:0/812364467 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e8037070 con 0x7fb4f4068790 2024-04-25T10:30:57.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.533+0000 7fb4f37fe640 1 -- 172.21.15.106:0/812364467 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4e8032440 con 0x7fb4f4068790 2024-04-25T10:30:57.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.533+0000 7fb4f37fe640 1 -- 172.21.15.106:0/812364467 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e8032810 con 0x7fb4f4068790 2024-04-25T10:30:57.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.533+0000 7fb4fae70640 1 -- 172.21.15.106:0/812364467 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb4f4068790 msgr2=0x7fb4f4156f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:57.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.533+0000 7fb4fae70640 1 --2- 172.21.15.106:0/812364467 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb4f4068790 0x7fb4f4156f10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb4e80097a0 tx=0x7fb4e802a180 comp rx=0 tx=0).stop 2024-04-25T10:30:57.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.533+0000 7fb4fae70640 1 -- 172.21.15.106:0/812364467 shutdown_connections 2024-04-25T10:30:57.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.533+0000 7fb4fae70640 1 --2- 172.21.15.106:0/812364467 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4f40c8f70 0x7fb4f40c9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:57.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.533+0000 7fb4fae70640 1 --2- 172.21.15.106:0/812364467 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb4f4068790 0x7fb4f4156f10 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:57.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.533+0000 7fb4fae70640 1 -- 172.21.15.106:0/812364467 >> 172.21.15.106:0/812364467 conn(0x7fb4f40ce570 msgr2=0x7fb4f40ce980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:57.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.533+0000 7fb4fae70640 1 -- 172.21.15.106:0/812364467 shutdown_connections 2024-04-25T10:30:57.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.533+0000 7fb4fae70640 1 -- 172.21.15.106:0/812364467 wait complete. 2024-04-25T10:30:57.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.535+0000 7fb4fae70640 1 Processor -- start 2024-04-25T10:30:57.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.535+0000 7fb4fae70640 1 -- start start 2024-04-25T10:30:57.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.536+0000 7fb4fae70640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4f4068790 0x7fb4f41f3210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:57.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.536+0000 7fb4fae70640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4f40c8f70 0x7fb4f41536a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:57.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.536+0000 7fb4fae70640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb4f415d4a0 0x7fb4f4151de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:57.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.536+0000 7fb4fae70640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f4157450 con 0x7fb4f415d4a0 2024-04-25T10:30:57.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.536+0000 7fb4fae70640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f415cf80 con 0x7fb4f4068790 2024-04-25T10:30:57.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.536+0000 7fb4fae70640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb4f4070fb0 con 0x7fb4f40c8f70 2024-04-25T10:30:57.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.536+0000 7fb4f940d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb4f415d4a0 0x7fb4f4151de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:57.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.536+0000 7fb4f940d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb4f415d4a0 0x7fb4f4151de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35084/0 (socket says 172.21.15.106:35084) 2024-04-25T10:30:57.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.536+0000 7fb4f940d640 1 -- 172.21.15.106:0/3303032115 learned_addr learned my addr 172.21.15.106:0/3303032115 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:57.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.536+0000 7fb4f8c0c640 1 --2- 172.21.15.106:0/3303032115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4f4068790 0x7fb4f41f3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:57.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.537+0000 7fb4f3fff640 1 --2- 172.21.15.106:0/3303032115 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4f40c8f70 0x7fb4f41536a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:57.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.537+0000 7fb4f8c0c640 1 -- 172.21.15.106:0/3303032115 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4f40c8f70 msgr2=0x7fb4f41536a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:57.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.537+0000 7fb4f8c0c640 1 --2- 172.21.15.106:0/3303032115 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4f40c8f70 0x7fb4f41536a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:57.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.537+0000 7fb4f8c0c640 1 -- 172.21.15.106:0/3303032115 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb4f415d4a0 msgr2=0x7fb4f4151de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:57.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.537+0000 7fb4f8c0c640 1 --2- 172.21.15.106:0/3303032115 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb4f415d4a0 0x7fb4f4151de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:57.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.537+0000 7fb4f8c0c640 1 -- 172.21.15.106:0/3303032115 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4f41f2d50 con 0x7fb4f4068790 2024-04-25T10:30:57.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.537+0000 7fb4f940d640 1 --2- 172.21.15.106:0/3303032115 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb4f415d4a0 0x7fb4f4151de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:57.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.537+0000 7fb4f8c0c640 1 --2- 172.21.15.106:0/3303032115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4f4068790 0x7fb4f41f3210 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb4e8031000 tx=0x7fb4e80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:57.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.537+0000 7fb4f17fa640 1 -- 172.21.15.106:0/3303032115 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e8041020 con 0x7fb4f4068790 2024-04-25T10:30:57.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.537+0000 7fb4f17fa640 1 -- 172.21.15.106:0/3303032115 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4e8030700 con 0x7fb4f4068790 2024-04-25T10:30:57.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.537+0000 7fb4fae70640 1 -- 172.21.15.106:0/3303032115 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4f41f3750 con 0x7fb4f4068790 2024-04-25T10:30:57.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.537+0000 7fb4f17fa640 1 -- 172.21.15.106:0/3303032115 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e8037040 con 0x7fb4f4068790 2024-04-25T10:30:57.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.537+0000 7fb4fae70640 1 -- 172.21.15.106:0/3303032115 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4f41e4a10 con 0x7fb4f4068790 2024-04-25T10:30:57.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.538+0000 7fb4fae70640 1 -- 172.21.15.106:0/3303032115 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4f406a3d0 con 0x7fb4f4068790 2024-04-25T10:30:57.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.539+0000 7fb4f17fa640 1 -- 172.21.15.106:0/3303032115 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb4e8044020 con 0x7fb4f4068790 2024-04-25T10:30:57.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.539+0000 7fb4f17fa640 1 --2- 172.21.15.106:0/3303032115 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fb4d80744e0 0x7fb4d80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:57.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.539+0000 7fb4f3fff640 1 --2- 172.21.15.106:0/3303032115 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fb4d80744e0 0x7fb4d80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:57.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.539+0000 7fb4f17fa640 1 -- 172.21.15.106:0/3303032115 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb4e80b59b0 con 0x7fb4f4068790 2024-04-25T10:30:57.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.540+0000 7fb4f3fff640 1 --2- 172.21.15.106:0/3303032115 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fb4d80744e0 0x7fb4d80769f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb4f40b2110 tx=0x7fb4e0009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:57.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.544+0000 7fb4f17fa640 1 -- 172.21.15.106:0/3303032115 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb4e804b030 con 0x7fb4f4068790 2024-04-25T10:30:57.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.756+0000 7fb4fae70640 1 -- 172.21.15.106:0/3303032115 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb4f40c96a0 con 0x7fb4f4068790 2024-04-25T10:30:57.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.757+0000 7fb4f17fa640 1 -- 172.21.15.106:0/3303032115 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+6918 (secure 0 0 0) 0x7fb4f41f2d50 con 0x7fb4f4068790 2024-04-25T10:30:57.759 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:30:57.759 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/2834155852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":2834155852},{"type":"v1","addr":"172.21.15.106:6835","nonce":2834155852}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:30:31.761107+0000","modified":"2024-04-25T10:30:54.574122+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4290,"mds_1":4287,"mds_2":4284,"mds_3":4269,"mds_4":4266},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4266":{"gid":4266,"name":"c","rank":4,"incarnation":16,"state":"up:creating","state_seq":1,"addr":"172.21.15.106:6837/2632121774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":2632121774},{"type":"v1","addr":"172.21.15.106:6837","nonce":2632121774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4269":{"gid":4269,"name":"e","rank":3,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.106:6839/486339834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":486339834},{"type":"v1","addr":"172.21.15.106:6839","nonce":486339834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4284":{"gid":4284,"name":"b","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.134:6836/2781656864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6833","nonce":2781656864},{"type":"v1","addr":"172.21.15.134:6836","nonce":2781656864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4287":{"gid":4287,"name":"f","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.134:6835/1993918351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1993918351},{"type":"v1","addr":"172.21.15.134:6835","nonce":1993918351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4290":{"gid":4290,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.134:6837/2662003420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2662003420},{"type":"v1","addr":"172.21.15.134:6837","nonce":2662003420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4290,"qdb_cluster":[4290,4287,4284,4269]},"id":1}]} 2024-04-25T10:30:57.759 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 16 2024-04-25T10:30:57.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.759+0000 7fb4fae70640 1 -- 172.21.15.106:0/3303032115 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fb4d80744e0 msgr2=0x7fb4d80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:57.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.759+0000 7fb4fae70640 1 --2- 172.21.15.106:0/3303032115 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fb4d80744e0 0x7fb4d80769f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb4f40b2110 tx=0x7fb4e0009190 comp rx=0 tx=0).stop 2024-04-25T10:30:57.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.759+0000 7fb4fae70640 1 -- 172.21.15.106:0/3303032115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4f4068790 msgr2=0x7fb4f41f3210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:57.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.759+0000 7fb4fae70640 1 --2- 172.21.15.106:0/3303032115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4f4068790 0x7fb4f41f3210 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb4e8031000 tx=0x7fb4e80046c0 comp rx=0 tx=0).stop 2024-04-25T10:30:57.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.759+0000 7fb4fae70640 1 -- 172.21.15.106:0/3303032115 shutdown_connections 2024-04-25T10:30:57.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.759+0000 7fb4fae70640 1 --2- 172.21.15.106:0/3303032115 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fb4d80744e0 0x7fb4d80769f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:57.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.759+0000 7fb4fae70640 1 --2- 172.21.15.106:0/3303032115 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb4f415d4a0 0x7fb4f4151de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:57.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.759+0000 7fb4fae70640 1 --2- 172.21.15.106:0/3303032115 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4f40c8f70 0x7fb4f41536a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:57.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.759+0000 7fb4fae70640 1 --2- 172.21.15.106:0/3303032115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4f4068790 0x7fb4f41f3210 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:57.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.759+0000 7fb4fae70640 1 -- 172.21.15.106:0/3303032115 >> 172.21.15.106:0/3303032115 conn(0x7fb4f40ce570 msgr2=0x7fb4f4071290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:57.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.759+0000 7fb4fae70640 1 -- 172.21.15.106:0/3303032115 shutdown_connections 2024-04-25T10:30:57.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.759+0000 7fb4fae70640 1 -- 172.21.15.106:0/3303032115 wait complete. 2024-04-25T10:30:57.772 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:30:57.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.877+0000 7f1f41a1a640 1 Processor -- start 2024-04-25T10:30:57.879 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.877+0000 7f1f41a1a640 1 -- start start 2024-04-25T10:30:57.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.878+0000 7f1f41a1a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1f3c0c8d90 0x7f1f3c156ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:57.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.878+0000 7f1f41a1a640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f1f3c0b4ce0 0x7f1f3c15ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:57.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.879+0000 7f1f41a1a640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f3c0ce630 con 0x7f1f3c0c7e30 2024-04-25T10:30:57.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.879+0000 7f1f41a1a640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f1f3c0b50f0 con 0x7f1f3c0b4ce0 2024-04-25T10:30:57.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.879+0000 7f1f41a1a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f1f3c0c7c60 con 0x7f1f3c0c8d90 2024-04-25T10:30:57.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.879+0000 7f1f3b7fe640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f1f3c0b4ce0 0x7f1f3c15ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:57.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.879+0000 7f1f3a7fc640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1f3c0c8d90 0x7f1f3c156ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:57.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.879+0000 7f1f3b7fe640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f1f3c0b4ce0 0x7f1f3c15ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.106:55452/0 (socket says 172.21.15.106:55452) 2024-04-25T10:30:57.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.879+0000 7f1f3affd640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f1f3c0c7e30 0x7f1f3c06d000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.106:41182/0 (socket says 172.21.15.106:41182) 2024-04-25T10:30:57.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.879+0000 7f1f3b7fe640 1 -- 172.21.15.106:0/3391873225 learned_addr learned my addr 172.21.15.106:0/3391873225 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:57.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.879+0000 7f1f3a7fc640 1 --2- 172.21.15.106:0/3391873225 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1f3c0c8d90 0x7f1f3c156ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.106:54586/0 (socket says 172.21.15.106:54586) 2024-04-25T10:30:57.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.879+0000 7f1f3b7fe640 1 -- 172.21.15.106:0/3391873225 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1f3c0c8d90 msgr2=0x7f1f3c156ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:57.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.879+0000 7f1f3b7fe640 1 --2- 172.21.15.106:0/3391873225 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1f3c0c8d90 0x7f1f3c156ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:57.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.879+0000 7f1f3b7fe640 1 -- 172.21.15.106:0/3391873225 >> v1:172.21.15.134:6789/0 conn(0x7f1f3c0c7e30 legacy=0x7f1f3c06d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:57.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.879+0000 7f1f3b7fe640 1 -- 172.21.15.106:0/3391873225 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f3c0c7980 con 0x7f1f3c0b4ce0 2024-04-25T10:30:57.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.879+0000 7f1f39ffb640 1 -- 172.21.15.106:0/3391873225 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 913879744 0 0) 0x7f1f3c0ce630 con 0x7f1f3c0c7e30 2024-04-25T10:30:57.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.879+0000 7f1f3a7fc640 1 --2- 172.21.15.106:0/3391873225 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1f3c0c8d90 0x7f1f3c156ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:57.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.880+0000 7f1f3b7fe640 1 --2- 172.21.15.106:0/3391873225 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f3c0b4ce0 0x7f1f3c15ce30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c0c7de0 tx=0x7f1f3402a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f415fcb5ceaffb1e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:57.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.880+0000 7f1f39ffb640 1 -- 172.21.15.106:0/3391873225 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f34037070 con 0x7f1f3c0b4ce0 2024-04-25T10:30:57.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.880+0000 7f1f39ffb640 1 -- 172.21.15.106:0/3391873225 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f34032440 con 0x7f1f3c0b4ce0 2024-04-25T10:30:57.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.880+0000 7f1f39ffb640 1 -- 172.21.15.106:0/3391873225 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f34032810 con 0x7f1f3c0b4ce0 2024-04-25T10:30:57.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.881+0000 7f1f41a1a640 1 -- 172.21.15.106:0/3391873225 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f3c0b4ce0 msgr2=0x7f1f3c15ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:57.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.881+0000 7f1f41a1a640 1 --2- 172.21.15.106:0/3391873225 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f3c0b4ce0 0x7f1f3c15ce30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c0c7de0 tx=0x7f1f3402a1c0 comp rx=0 tx=0).stop 2024-04-25T10:30:57.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.881+0000 7f1f41a1a640 1 -- 172.21.15.106:0/3391873225 shutdown_connections 2024-04-25T10:30:57.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.881+0000 7f1f41a1a640 1 --2- 172.21.15.106:0/3391873225 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f3c0b4ce0 0x7f1f3c15ce30 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:57.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.881+0000 7f1f41a1a640 1 --2- 172.21.15.106:0/3391873225 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1f3c0c8d90 0x7f1f3c156ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:57.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.881+0000 7f1f41a1a640 1 -- 172.21.15.106:0/3391873225 >> 172.21.15.106:0/3391873225 conn(0x7f1f3c068b10 msgr2=0x7f1f3c0d1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:57.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.881+0000 7f1f41a1a640 1 -- 172.21.15.106:0/3391873225 shutdown_connections 2024-04-25T10:30:57.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.881+0000 7f1f41a1a640 1 -- 172.21.15.106:0/3391873225 wait complete. 2024-04-25T10:30:57.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.882+0000 7f1f41a1a640 1 Processor -- start 2024-04-25T10:30:57.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.883+0000 7f1f41a1a640 1 -- start start 2024-04-25T10:30:57.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.884+0000 7f1f41a1a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1f3c0b4ce0 0x7f1f3c1e8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:57.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.884+0000 7f1f41a1a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f3c0c7e30 0x7f1f3c1f6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:57.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.884+0000 7f1f41a1a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f3c0c8d90 0x7f1f3c0d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:57.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.884+0000 7f1f41a1a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1f3c06d710 con 0x7f1f3c0c8d90 2024-04-25T10:30:57.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.884+0000 7f1f41a1a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1f3c0c72f0 con 0x7f1f3c0c7e30 2024-04-25T10:30:57.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.884+0000 7f1f41a1a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f1f3c06d890 con 0x7f1f3c0b4ce0 2024-04-25T10:30:57.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.884+0000 7f1f3b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f3c0c8d90 0x7f1f3c0d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:57.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.884+0000 7f1f3affd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1f3c0b4ce0 0x7f1f3c1e8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:57.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.884+0000 7f1f3b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f3c0c8d90 0x7f1f3c0d0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35090/0 (socket says 172.21.15.106:35090) 2024-04-25T10:30:57.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.884+0000 7f1f3b7fe640 1 -- 172.21.15.106:0/4147105055 learned_addr learned my addr 172.21.15.106:0/4147105055 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:57.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.884+0000 7f1f3a7fc640 1 --2- 172.21.15.106:0/4147105055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f3c0c7e30 0x7f1f3c1f6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:57.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.884+0000 7f1f3affd640 1 -- 172.21.15.106:0/4147105055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f3c0c7e30 msgr2=0x7f1f3c1f6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:57.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.884+0000 7f1f3affd640 1 --2- 172.21.15.106:0/4147105055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f3c0c7e30 0x7f1f3c1f6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:57.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.884+0000 7f1f3affd640 1 -- 172.21.15.106:0/4147105055 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f3c0c8d90 msgr2=0x7f1f3c0d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:57.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.884+0000 7f1f3affd640 1 --2- 172.21.15.106:0/4147105055 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f3c0c8d90 0x7f1f3c0d0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:57.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.884+0000 7f1f3affd640 1 -- 172.21.15.106:0/4147105055 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f3c1f6740 con 0x7f1f3c0b4ce0 2024-04-25T10:30:57.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.884+0000 7f1f3a7fc640 1 --2- 172.21.15.106:0/4147105055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f3c0c7e30 0x7f1f3c1f6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:57.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.884+0000 7f1f3affd640 1 --2- 172.21.15.106:0/4147105055 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1f3c0b4ce0 0x7f1f3c1e8990 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c00c780 tx=0x7f1f2c00cc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:57.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.885+0000 7f1f23fff640 1 -- 172.21.15.106:0/4147105055 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f2c018070 con 0x7f1f3c0b4ce0 2024-04-25T10:30:57.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.885+0000 7f1f23fff640 1 -- 172.21.15.106:0/4147105055 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f2c00ce30 con 0x7f1f3c0b4ce0 2024-04-25T10:30:57.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.885+0000 7f1f41a1a640 1 -- 172.21.15.106:0/4147105055 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f3c1f2fe0 con 0x7f1f3c0b4ce0 2024-04-25T10:30:57.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.885+0000 7f1f23fff640 1 -- 172.21.15.106:0/4147105055 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f2c00a860 con 0x7f1f3c0b4ce0 2024-04-25T10:30:57.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.885+0000 7f1f41a1a640 1 -- 172.21.15.106:0/4147105055 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f3c1f35a0 con 0x7f1f3c0b4ce0 2024-04-25T10:30:57.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.886+0000 7f1f41a1a640 1 -- 172.21.15.106:0/4147105055 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f3c0cdd00 con 0x7f1f3c0b4ce0 2024-04-25T10:30:57.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.886+0000 7f1f23fff640 1 -- 172.21.15.106:0/4147105055 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1f2c01b070 con 0x7f1f3c0b4ce0 2024-04-25T10:30:57.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.886+0000 7f1f23fff640 1 --2- 172.21.15.106:0/4147105055 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f1f280744e0 0x7f1f280769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:57.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.887+0000 7f1f3a7fc640 1 --2- 172.21.15.106:0/4147105055 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f1f280744e0 0x7f1f280769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:57.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.887+0000 7f1f23fff640 1 -- 172.21.15.106:0/4147105055 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1f2c0946d0 con 0x7f1f3c0b4ce0 2024-04-25T10:30:57.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.887+0000 7f1f3a7fc640 1 --2- 172.21.15.106:0/4147105055 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f1f280744e0 0x7f1f280769f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1f30009500 tx=0x7f1f30009110 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:57.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:57.891+0000 7f1f23fff640 1 -- 172.21.15.106:0/4147105055 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1f3c0cdd00 con 0x7f1f3c0b4ce0 2024-04-25T10:30:58.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.066+0000 7f1f41a1a640 1 -- 172.21.15.106:0/4147105055 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1f3c06a3d0 con 0x7f1f3c0b4ce0 2024-04-25T10:30:58.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.067+0000 7f1f23fff640 1 -- 172.21.15.106:0/4147105055 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16641 (secure 0 0 0) 0x7f1f2c05e800 con 0x7f1f3c0b4ce0 2024-04-25T10:30:58.069 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:30:58.069 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":20,"fsid":"3ee77985-e8e4-41cf-81e9-7c774cf7891f","created":"2024-04-25T10:30:07.981054+0000","modified":"2024-04-25T10:30:33.409267+0000","last_up_change":"2024-04-25T10:30:19.184325+0000","last_in_change":"2024-04-25T10:30:10.217553+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:30:20.111046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:30:30.314914+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:30:30.715500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a5962cb1-dbbd-47b1-8d15-9bae114bfdf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1240717079},{"type":"v1","addr":"172.21.15.106:6801","nonce":1240717079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1240717079},{"type":"v1","addr":"172.21.15.106:6803","nonce":1240717079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1240717079},{"type":"v1","addr":"172.21.15.106:6807","nonce":1240717079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1240717079},{"type":"v1","addr":"172.21.15.106:6805","nonce":1240717079}]},"public_addr":"172.21.15.106:6801/1240717079","cluster_addr":"172.21.15.106:6803/1240717079","heartbeat_back_addr":"172.21.15.106:6807/1240717079","heartbeat_front_addr":"172.21.15.106:6805/1240717079","state":["exists","up"]},{"osd":1,"uuid":"94436e13-4769-4752-8397-b5e396a6f7ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4089653051},{"type":"v1","addr":"172.21.15.106:6825","nonce":4089653051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4089653051},{"type":"v1","addr":"172.21.15.106:6827","nonce":4089653051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4089653051},{"type":"v1","addr":"172.21.15.106:6831","nonce":4089653051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4089653051},{"type":"v1","addr":"172.21.15.106:6829","nonce":4089653051}]},"public_addr":"172.21.15.106:6825/4089653051","cluster_addr":"172.21.15.106:6827/4089653051","heartbeat_back_addr":"172.21.15.106:6831/4089653051","heartbeat_front_addr":"172.21.15.106:6829/4089653051","state":["exists","up"]},{"osd":2,"uuid":"429aae0b-70c6-4e7b-9043-6a11334c7efc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":1441018239},{"type":"v1","addr":"172.21.15.106:6809","nonce":1441018239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1441018239},{"type":"v1","addr":"172.21.15.106:6811","nonce":1441018239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1441018239},{"type":"v1","addr":"172.21.15.106:6815","nonce":1441018239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":1441018239},{"type":"v1","addr":"172.21.15.106:6813","nonce":1441018239}]},"public_addr":"172.21.15.106:6809/1441018239","cluster_addr":"172.21.15.106:6811/1441018239","heartbeat_back_addr":"172.21.15.106:6815/1441018239","heartbeat_front_addr":"172.21.15.106:6813/1441018239","state":["exists","up"]},{"osd":3,"uuid":"9593a4d1-772f-4667-ab0d-2982f841e342","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":1846968711},{"type":"v1","addr":"172.21.15.106:6817","nonce":1846968711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1846968711},{"type":"v1","addr":"172.21.15.106:6819","nonce":1846968711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":1846968711},{"type":"v1","addr":"172.21.15.106:6823","nonce":1846968711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":1846968711},{"type":"v1","addr":"172.21.15.106:6821","nonce":1846968711}]},"public_addr":"172.21.15.106:6817/1846968711","cluster_addr":"172.21.15.106:6819/1846968711","heartbeat_back_addr":"172.21.15.106:6823/1846968711","heartbeat_front_addr":"172.21.15.106:6821/1846968711","state":["exists","up"]},{"osd":4,"uuid":"1b61f16d-5c60-480a-b449-c745aa3f6b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":729224739},{"type":"v1","addr":"172.21.15.134:6825","nonce":729224739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":729224739},{"type":"v1","addr":"172.21.15.134:6827","nonce":729224739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":729224739},{"type":"v1","addr":"172.21.15.134:6831","nonce":729224739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":729224739},{"type":"v1","addr":"172.21.15.134:6829","nonce":729224739}]},"public_addr":"172.21.15.134:6825/729224739","cluster_addr":"172.21.15.134:6827/729224739","heartbeat_back_addr":"172.21.15.134:6831/729224739","heartbeat_front_addr":"172.21.15.134:6829/729224739","state":["exists","up"]},{"osd":5,"uuid":"944cdd7f-47c0-4b71-b7e0-d093542f4528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3132275652},{"type":"v1","addr":"172.21.15.134:6809","nonce":3132275652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3132275652},{"type":"v1","addr":"172.21.15.134:6811","nonce":3132275652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3132275652},{"type":"v1","addr":"172.21.15.134:6815","nonce":3132275652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3132275652},{"type":"v1","addr":"172.21.15.134:6813","nonce":3132275652}]},"public_addr":"172.21.15.134:6809/3132275652","cluster_addr":"172.21.15.134:6811/3132275652","heartbeat_back_addr":"172.21.15.134:6815/3132275652","heartbeat_front_addr":"172.21.15.134:6813/3132275652","state":["exists","up"]},{"osd":6,"uuid":"cc2446e2-1d21-4135-ac04-6af7c990058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":992702668},{"type":"v1","addr":"172.21.15.134:6801","nonce":992702668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":992702668},{"type":"v1","addr":"172.21.15.134:6803","nonce":992702668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":992702668},{"type":"v1","addr":"172.21.15.134:6807","nonce":992702668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":992702668},{"type":"v1","addr":"172.21.15.134:6805","nonce":992702668}]},"public_addr":"172.21.15.134:6801/992702668","cluster_addr":"172.21.15.134:6803/992702668","heartbeat_back_addr":"172.21.15.134:6807/992702668","heartbeat_front_addr":"172.21.15.134:6805/992702668","state":["exists","up"]},{"osd":7,"uuid":"3df1fc7a-82fc-4257-8629-98027b6e2aed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":3291062284},{"type":"v1","addr":"172.21.15.134:6817","nonce":3291062284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":3291062284},{"type":"v1","addr":"172.21.15.134:6819","nonce":3291062284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":3291062284},{"type":"v1","addr":"172.21.15.134:6823","nonce":3291062284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":3291062284},{"type":"v1","addr":"172.21.15.134:6821","nonce":3291062284}]},"public_addr":"172.21.15.134:6817/3291062284","cluster_addr":"172.21.15.134:6819/3291062284","heartbeat_back_addr":"172.21.15.134:6823/3291062284","heartbeat_front_addr":"172.21.15.134:6821/3291062284","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.403941+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.382563+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.411544+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.330444+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.405430+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.387737+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.408922+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.454450+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:30:58.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.069+0000 7f1f41a1a640 1 -- 172.21.15.106:0/4147105055 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f1f280744e0 msgr2=0x7f1f280769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:58.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.069+0000 7f1f41a1a640 1 --2- 172.21.15.106:0/4147105055 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f1f280744e0 0x7f1f280769f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1f30009500 tx=0x7f1f30009110 comp rx=0 tx=0).stop 2024-04-25T10:30:58.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.069+0000 7f1f41a1a640 1 -- 172.21.15.106:0/4147105055 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1f3c0b4ce0 msgr2=0x7f1f3c1e8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:58.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.069+0000 7f1f41a1a640 1 --2- 172.21.15.106:0/4147105055 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1f3c0b4ce0 0x7f1f3c1e8990 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c00c780 tx=0x7f1f2c00cc40 comp rx=0 tx=0).stop 2024-04-25T10:30:58.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.070+0000 7f1f41a1a640 1 -- 172.21.15.106:0/4147105055 shutdown_connections 2024-04-25T10:30:58.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.070+0000 7f1f41a1a640 1 --2- 172.21.15.106:0/4147105055 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f1f280744e0 0x7f1f280769f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:58.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.070+0000 7f1f41a1a640 1 --2- 172.21.15.106:0/4147105055 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f3c0c8d90 0x7f1f3c0d0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:58.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.070+0000 7f1f41a1a640 1 --2- 172.21.15.106:0/4147105055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f3c0c7e30 0x7f1f3c1f6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:58.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.070+0000 7f1f41a1a640 1 --2- 172.21.15.106:0/4147105055 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1f3c0b4ce0 0x7f1f3c1e8990 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:58.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.070+0000 7f1f41a1a640 1 -- 172.21.15.106:0/4147105055 >> 172.21.15.106:0/4147105055 conn(0x7f1f3c068b10 msgr2=0x7f1f3c0cc630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:58.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.070+0000 7f1f41a1a640 1 -- 172.21.15.106:0/4147105055 shutdown_connections 2024-04-25T10:30:58.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.070+0000 7f1f41a1a640 1 -- 172.21.15.106:0/4147105055 wait complete. 2024-04-25T10:30:58.082 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:30:20.111046+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:30:30.314914+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:30:30.715500+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:30:58.083 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:30:58.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.192+0000 7fdccb0ee640 1 Processor -- start 2024-04-25T10:30:58.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.192+0000 7fdccb0ee640 1 -- start start 2024-04-25T10:30:58.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.194+0000 7fdccb0ee640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdcc40cc600 0x7fdcc40c5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:58.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.194+0000 7fdccb0ee640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fdcc40c58d0 0x7fdcc40cce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:58.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.194+0000 7fdccb0ee640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdcc415b8c0 con 0x7fdcc40cb2b0 2024-04-25T10:30:58.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.194+0000 7fdccb0ee640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fdcc415eea0 con 0x7fdcc40c58d0 2024-04-25T10:30:58.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.194+0000 7fdccb0ee640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fdcc415b5e0 con 0x7fdcc40cc600 2024-04-25T10:30:58.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.194+0000 7fdcc3fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fdcc40c58d0 0x7fdcc40cce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:58.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.194+0000 7fdcc3fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fdcc40c58d0 0x7fdcc40cce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35096/0 (socket says 172.21.15.106:35096) 2024-04-25T10:30:58.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.194+0000 7fdcc3fff640 1 -- 172.21.15.106:0/819838009 learned_addr learned my addr 172.21.15.106:0/819838009 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:58.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.194+0000 7fdcc8e8a640 1 --2- 172.21.15.106:0/819838009 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdcc40cc600 0x7fdcc40c5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:58.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.194+0000 7fdcc3fff640 1 -- 172.21.15.106:0/819838009 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdcc40cc600 msgr2=0x7fdcc40c5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:58.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.194+0000 7fdcc3fff640 1 --2- 172.21.15.106:0/819838009 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdcc40cc600 0x7fdcc40c5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:58.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.194+0000 7fdcc3fff640 1 -- 172.21.15.106:0/819838009 >> v1:172.21.15.134:6789/0 conn(0x7fdcc40cb2b0 legacy=0x7fdcc4070bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:58.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.194+0000 7fdcc3fff640 1 -- 172.21.15.106:0/819838009 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcc415e0e0 con 0x7fdcc40c58d0 2024-04-25T10:30:58.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.195+0000 7fdcc3fff640 1 --2- 172.21.15.106:0/819838009 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdcc40c58d0 0x7fdcc40cce90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8009870 tx=0x7fdcb802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34da2a4a6ab5128c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:58.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.195+0000 7fdcc37fe640 1 -- 172.21.15.106:0/819838009 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb8037070 con 0x7fdcc40c58d0 2024-04-25T10:30:58.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.195+0000 7fdcc37fe640 1 -- 172.21.15.106:0/819838009 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcb8032440 con 0x7fdcc40c58d0 2024-04-25T10:30:58.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.195+0000 7fdcc37fe640 1 -- 172.21.15.106:0/819838009 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb8032810 con 0x7fdcc40c58d0 2024-04-25T10:30:58.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.196+0000 7fdccb0ee640 1 -- 172.21.15.106:0/819838009 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdcc40c58d0 msgr2=0x7fdcc40cce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:58.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.196+0000 7fdccb0ee640 1 --2- 172.21.15.106:0/819838009 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdcc40c58d0 0x7fdcc40cce90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8009870 tx=0x7fdcb802a270 comp rx=0 tx=0).stop 2024-04-25T10:30:58.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.196+0000 7fdccb0ee640 1 -- 172.21.15.106:0/819838009 shutdown_connections 2024-04-25T10:30:58.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.196+0000 7fdccb0ee640 1 --2- 172.21.15.106:0/819838009 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdcc40c58d0 0x7fdcc40cce90 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:58.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.196+0000 7fdccb0ee640 1 --2- 172.21.15.106:0/819838009 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdcc40cc600 0x7fdcc40c5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:58.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.196+0000 7fdccb0ee640 1 -- 172.21.15.106:0/819838009 >> 172.21.15.106:0/819838009 conn(0x7fdcc40727d0 msgr2=0x7fdcc4072c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:58.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.196+0000 7fdccb0ee640 1 -- 172.21.15.106:0/819838009 shutdown_connections 2024-04-25T10:30:58.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.196+0000 7fdccb0ee640 1 -- 172.21.15.106:0/819838009 wait complete. 2024-04-25T10:30:58.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.197+0000 7fdccb0ee640 1 Processor -- start 2024-04-25T10:30:58.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.197+0000 7fdccb0ee640 1 -- start start 2024-04-25T10:30:58.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.199+0000 7fdccb0ee640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdcc40c58d0 0x7fdcc414f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:58.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.199+0000 7fdccb0ee640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdcc40cb2b0 0x7fdcc414e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:58.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.199+0000 7fdccb0ee640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdcc40cc600 0x7fdcc40c5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:58.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.199+0000 7fdccb0ee640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc415e980 con 0x7fdcc40cb2b0 2024-04-25T10:30:58.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.199+0000 7fdccb0ee640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc415e6a0 con 0x7fdcc40c58d0 2024-04-25T10:30:58.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.199+0000 7fdccb0ee640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fdcc40cbff0 con 0x7fdcc40cc600 2024-04-25T10:30:58.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.199+0000 7fdcc3fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdcc40cb2b0 0x7fdcc414e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:58.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.199+0000 7fdcc3fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdcc40cb2b0 0x7fdcc414e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35106/0 (socket says 172.21.15.106:35106) 2024-04-25T10:30:58.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.199+0000 7fdcc3fff640 1 -- 172.21.15.106:0/868759398 learned_addr learned my addr 172.21.15.106:0/868759398 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:58.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.199+0000 7fdcc968b640 1 --2- 172.21.15.106:0/868759398 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdcc40cc600 0x7fdcc40c5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:58.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.199+0000 7fdcc8e8a640 1 --2- 172.21.15.106:0/868759398 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdcc40c58d0 0x7fdcc414f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:58.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.199+0000 7fdcc3fff640 1 -- 172.21.15.106:0/868759398 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdcc40cc600 msgr2=0x7fdcc40c5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:58.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.199+0000 7fdcc3fff640 1 --2- 172.21.15.106:0/868759398 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdcc40cc600 0x7fdcc40c5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:58.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.199+0000 7fdcc3fff640 1 -- 172.21.15.106:0/868759398 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdcc40c58d0 msgr2=0x7fdcc414f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:58.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.199+0000 7fdcc3fff640 1 --2- 172.21.15.106:0/868759398 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdcc40c58d0 0x7fdcc414f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:58.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.199+0000 7fdcc3fff640 1 -- 172.21.15.106:0/868759398 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcc415bd00 con 0x7fdcc40cb2b0 2024-04-25T10:30:58.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.199+0000 7fdcc3fff640 1 --2- 172.21.15.106:0/868759398 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdcc40cb2b0 0x7fdcc414e5f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fdcb802a2b0 tx=0x7fdcb80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:58.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.199+0000 7fdcc17fa640 1 -- 172.21.15.106:0/868759398 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb8037040 con 0x7fdcc40cb2b0 2024-04-25T10:30:58.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.200+0000 7fdcc17fa640 1 -- 172.21.15.106:0/868759398 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcb8042070 con 0x7fdcc40cb2b0 2024-04-25T10:30:58.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.200+0000 7fdccb0ee640 1 -- 172.21.15.106:0/868759398 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcc415bfe0 con 0x7fdcc40cb2b0 2024-04-25T10:30:58.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.200+0000 7fdcc17fa640 1 -- 172.21.15.106:0/868759398 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb8030880 con 0x7fdcc40cb2b0 2024-04-25T10:30:58.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.200+0000 7fdccb0ee640 1 -- 172.21.15.106:0/868759398 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcc415c5a0 con 0x7fdcc40cb2b0 2024-04-25T10:30:58.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.201+0000 7fdccb0ee640 1 -- 172.21.15.106:0/868759398 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcc406a3d0 con 0x7fdcc40cb2b0 2024-04-25T10:30:58.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.201+0000 7fdcc17fa640 1 -- 172.21.15.106:0/868759398 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fdcb8030a20 con 0x7fdcc40cb2b0 2024-04-25T10:30:58.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.202+0000 7fdcc17fa640 1 --2- 172.21.15.106:0/868759398 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fdcb00744b0 0x7fdcb00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:58.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.202+0000 7fdcc17fa640 1 -- 172.21.15.106:0/868759398 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fdcb80b5a00 con 0x7fdcc40cb2b0 2024-04-25T10:30:58.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.202+0000 7fdcc8e8a640 1 --2- 172.21.15.106:0/868759398 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fdcb00744b0 0x7fdcb00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:58.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.202+0000 7fdcc8e8a640 1 --2- 172.21.15.106:0/868759398 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fdcb00744b0 0x7fdcb00769c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fdcc40749e0 tx=0x7fdcb4009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:58.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.207+0000 7fdcc17fa640 1 -- 172.21.15.106:0/868759398 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdcb807fb40 con 0x7fdcc40cb2b0 2024-04-25T10:30:58.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.381+0000 7fdccb0ee640 1 -- 172.21.15.106:0/868759398 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdcc40cbdd0 con 0x7fdcc40cb2b0 2024-04-25T10:30:58.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.381+0000 7fdcc17fa640 1 -- 172.21.15.106:0/868759398 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fdcb8083e40 con 0x7fdcc40cb2b0 2024-04-25T10:30:58.383 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-25T10:30:58.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.384+0000 7fdccb0ee640 1 -- 172.21.15.106:0/868759398 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fdcb00744b0 msgr2=0x7fdcb00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:58.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.384+0000 7fdccb0ee640 1 --2- 172.21.15.106:0/868759398 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fdcb00744b0 0x7fdcb00769c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fdcc40749e0 tx=0x7fdcb4009190 comp rx=0 tx=0).stop 2024-04-25T10:30:58.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.384+0000 7fdccb0ee640 1 -- 172.21.15.106:0/868759398 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdcc40cb2b0 msgr2=0x7fdcc414e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:58.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.384+0000 7fdccb0ee640 1 --2- 172.21.15.106:0/868759398 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdcc40cb2b0 0x7fdcc414e5f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fdcb802a2b0 tx=0x7fdcb80046c0 comp rx=0 tx=0).stop 2024-04-25T10:30:58.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.384+0000 7fdccb0ee640 1 -- 172.21.15.106:0/868759398 shutdown_connections 2024-04-25T10:30:58.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.384+0000 7fdccb0ee640 1 --2- 172.21.15.106:0/868759398 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fdcb00744b0 0x7fdcb00769c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:58.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.384+0000 7fdccb0ee640 1 --2- 172.21.15.106:0/868759398 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdcc40cc600 0x7fdcc40c5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:58.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.384+0000 7fdccb0ee640 1 --2- 172.21.15.106:0/868759398 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdcc40cb2b0 0x7fdcc414e5f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:58.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.384+0000 7fdccb0ee640 1 --2- 172.21.15.106:0/868759398 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdcc40c58d0 0x7fdcc414f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:58.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.384+0000 7fdccb0ee640 1 -- 172.21.15.106:0/868759398 >> 172.21.15.106:0/868759398 conn(0x7fdcc40727d0 msgr2=0x7fdcc40ccc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:58.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.384+0000 7fdccb0ee640 1 -- 172.21.15.106:0/868759398 shutdown_connections 2024-04-25T10:30:58.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.384+0000 7fdccb0ee640 1 -- 172.21.15.106:0/868759398 wait complete. 2024-04-25T10:30:58.396 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:30:58.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.505+0000 7fbad2c92640 1 Processor -- start 2024-04-25T10:30:58.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.505+0000 7fbad2c92640 1 -- start start 2024-04-25T10:30:58.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.507+0000 7fbad2c92640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fbacc0cae80 0x7fbacc15e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:58.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.507+0000 7fbad2c92640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbacc0c9b00 con 0x7fbacc0cb290 2024-04-25T10:30:58.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.507+0000 7fbad2c92640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbacc0c9da0 con 0x7fbacc0c8050 2024-04-25T10:30:58.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.507+0000 7fbad2c92640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fbacc0ca080 con 0x7fbacc0cae80 2024-04-25T10:30:58.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.507+0000 7fbacbfff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fbacc0cae80 0x7fbacc15e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:58.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.507+0000 7fbad122f640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fbacc0cb290 0x7fbacc1547a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:53518/0 (socket says 172.21.15.106:53518) 2024-04-25T10:30:58.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.507+0000 7fbad122f640 1 -- 172.21.15.106:0/1603217136 learned_addr learned my addr 172.21.15.106:0/1603217136 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:30:58.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.507+0000 7fbacb7fe640 1 -- 172.21.15.106:0/1603217136 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3755243027 0 0) 0x7fbacc0c9b00 con 0x7fbacc0cb290 2024-04-25T10:30:58.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.507+0000 7fbacb7fe640 1 -- 172.21.15.106:0/1603217136 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbab4003520 con 0x7fbacc0cb290 2024-04-25T10:30:58.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.507+0000 7fbacbfff640 1 -- 172.21.15.106:0/1603217136 >> v1:172.21.15.134:6789/0 conn(0x7fbacc0c8050 legacy=0x7fbacc0c8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:58.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.507+0000 7fbacbfff640 1 -- 172.21.15.106:0/1603217136 >> v1:172.21.15.106:6789/0 conn(0x7fbacc0cb290 legacy=0x7fbacc1547a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:58.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.507+0000 7fbacbfff640 1 -- 172.21.15.106:0/1603217136 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbacc0ca360 con 0x7fbacc0cae80 2024-04-25T10:30:58.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.508+0000 7fbacbfff640 1 --2- 172.21.15.106:0/1603217136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbacc0cae80 0x7fbacc15e490 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbac00097a0 tx=0x7fbac002a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad9810437e49fd44 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:58.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.508+0000 7fbacb7fe640 1 -- 172.21.15.106:0/1603217136 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac0037070 con 0x7fbacc0cae80 2024-04-25T10:30:58.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.508+0000 7fbacb7fe640 1 -- 172.21.15.106:0/1603217136 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbac0032440 con 0x7fbacc0cae80 2024-04-25T10:30:58.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.508+0000 7fbacb7fe640 1 -- 172.21.15.106:0/1603217136 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac00327f0 con 0x7fbacc0cae80 2024-04-25T10:30:58.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.509+0000 7fbad2c92640 1 -- 172.21.15.106:0/1603217136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbacc0cae80 msgr2=0x7fbacc15e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:58.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.509+0000 7fbad2c92640 1 --2- 172.21.15.106:0/1603217136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbacc0cae80 0x7fbacc15e490 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbac00097a0 tx=0x7fbac002a230 comp rx=0 tx=0).stop 2024-04-25T10:30:58.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.509+0000 7fbad2c92640 1 -- 172.21.15.106:0/1603217136 shutdown_connections 2024-04-25T10:30:58.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.509+0000 7fbad2c92640 1 --2- 172.21.15.106:0/1603217136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbacc0cae80 0x7fbacc15e490 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:58.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.509+0000 7fbad2c92640 1 -- 172.21.15.106:0/1603217136 >> 172.21.15.106:0/1603217136 conn(0x7fbacc06c9b0 msgr2=0x7fbacc06cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:58.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.509+0000 7fbad2c92640 1 -- 172.21.15.106:0/1603217136 shutdown_connections 2024-04-25T10:30:58.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.509+0000 7fbad2c92640 1 -- 172.21.15.106:0/1603217136 wait complete. 2024-04-25T10:30:58.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.510+0000 7fbad2c92640 1 Processor -- start 2024-04-25T10:30:58.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.510+0000 7fbad2c92640 1 -- start start 2024-04-25T10:30:58.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.512+0000 7fbad2c92640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbacc0cae80 0x7fbacc147d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:58.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.512+0000 7fbad2c92640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbacc0cb290 0x7fbacc158cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:58.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.512+0000 7fbad2c92640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbacc15a530 0x7fbacc06ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:58.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.512+0000 7fbad2c92640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbacc0c9500 con 0x7fbacc0cb290 2024-04-25T10:30:58.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.512+0000 7fbad2c92640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbacc0c9380 con 0x7fbacc0cae80 2024-04-25T10:30:58.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.512+0000 7fbad2c92640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fbacc154eb0 con 0x7fbacc15a530 2024-04-25T10:30:58.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.512+0000 7fbacbfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbacc0cb290 0x7fbacc158cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:58.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.512+0000 7fbacbfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbacc0cb290 0x7fbacc158cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35110/0 (socket says 172.21.15.106:35110) 2024-04-25T10:30:58.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.512+0000 7fbacbfff640 1 -- 172.21.15.106:0/133673292 learned_addr learned my addr 172.21.15.106:0/133673292 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:58.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.512+0000 7fbad122f640 1 --2- 172.21.15.106:0/133673292 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbacc15a530 0x7fbacc06ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:58.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.512+0000 7fbad0a2e640 1 --2- 172.21.15.106:0/133673292 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbacc0cae80 0x7fbacc147d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:58.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.512+0000 7fbacbfff640 1 -- 172.21.15.106:0/133673292 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbacc15a530 msgr2=0x7fbacc06ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:58.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.512+0000 7fbacbfff640 1 --2- 172.21.15.106:0/133673292 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbacc15a530 0x7fbacc06ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:58.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.512+0000 7fbacbfff640 1 -- 172.21.15.106:0/133673292 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbacc0cae80 msgr2=0x7fbacc147d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:58.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.512+0000 7fbacbfff640 1 --2- 172.21.15.106:0/133673292 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbacc0cae80 0x7fbacc147d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:58.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.512+0000 7fbacbfff640 1 -- 172.21.15.106:0/133673292 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbacc159210 con 0x7fbacc0cb290 2024-04-25T10:30:58.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.512+0000 7fbacbfff640 1 --2- 172.21.15.106:0/133673292 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbacc0cb290 0x7fbacc158cd0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbac00097a0 tx=0x7fbac00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:58.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.512+0000 7fbac97fa640 1 -- 172.21.15.106:0/133673292 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac0037050 con 0x7fbacc0cb290 2024-04-25T10:30:58.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.512+0000 7fbac97fa640 1 -- 172.21.15.106:0/133673292 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbac0038040 con 0x7fbacc0cb290 2024-04-25T10:30:58.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.512+0000 7fbac97fa640 1 -- 172.21.15.106:0/133673292 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac0004470 con 0x7fbacc0cb290 2024-04-25T10:30:58.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.512+0000 7fbad2c92640 1 -- 172.21.15.106:0/133673292 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbacc1f3fe0 con 0x7fbacc0cb290 2024-04-25T10:30:58.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.512+0000 7fbad2c92640 1 -- 172.21.15.106:0/133673292 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbacc1f45a0 con 0x7fbacc0cb290 2024-04-25T10:30:58.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.514+0000 7fbad2c92640 1 -- 172.21.15.106:0/133673292 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbacc0c84a0 con 0x7fbacc0cb290 2024-04-25T10:30:58.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.514+0000 7fbac97fa640 1 -- 172.21.15.106:0/133673292 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbac003c6d0 con 0x7fbacc0cb290 2024-04-25T10:30:58.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.515+0000 7fbac97fa640 1 --2- 172.21.15.106:0/133673292 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fbab80744b0 0x7fbab80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:58.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.515+0000 7fbac97fa640 1 -- 172.21.15.106:0/133673292 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fbac00b5b50 con 0x7fbacc0cb290 2024-04-25T10:30:58.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.515+0000 7fbad0a2e640 1 --2- 172.21.15.106:0/133673292 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fbab80744b0 0x7fbab80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:58.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.515+0000 7fbad0a2e640 1 --2- 172.21.15.106:0/133673292 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fbab80744b0 0x7fbab80769c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbabc0016e0 tx=0x7fbabc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:58.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.519+0000 7fbac97fa640 1 -- 172.21.15.106:0/133673292 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbacc0c84a0 con 0x7fbacc0cb290 2024-04-25T10:30:58.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.694+0000 7fbad2c92640 1 -- 172.21.15.106:0/133673292 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbacc06a3d0 con 0x7fbacc0cb290 2024-04-25T10:30:58.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.694+0000 7fbac97fa640 1 -- 172.21.15.106:0/133673292 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fbac0086070 con 0x7fbacc0cb290 2024-04-25T10:30:58.695 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-25T10:30:58.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.696+0000 7fbad2c92640 1 -- 172.21.15.106:0/133673292 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fbab80744b0 msgr2=0x7fbab80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:58.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.696+0000 7fbad2c92640 1 --2- 172.21.15.106:0/133673292 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fbab80744b0 0x7fbab80769c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbabc0016e0 tx=0x7fbabc009040 comp rx=0 tx=0).stop 2024-04-25T10:30:58.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.696+0000 7fbad2c92640 1 -- 172.21.15.106:0/133673292 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbacc0cb290 msgr2=0x7fbacc158cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:58.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.696+0000 7fbad2c92640 1 --2- 172.21.15.106:0/133673292 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbacc0cb290 0x7fbacc158cd0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbac00097a0 tx=0x7fbac00046c0 comp rx=0 tx=0).stop 2024-04-25T10:30:58.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.696+0000 7fbad2c92640 1 -- 172.21.15.106:0/133673292 shutdown_connections 2024-04-25T10:30:58.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.696+0000 7fbad2c92640 1 --2- 172.21.15.106:0/133673292 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fbab80744b0 0x7fbab80769c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:58.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.696+0000 7fbad2c92640 1 --2- 172.21.15.106:0/133673292 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbacc15a530 0x7fbacc06ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:58.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.696+0000 7fbad2c92640 1 --2- 172.21.15.106:0/133673292 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbacc0cb290 0x7fbacc158cd0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:58.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.696+0000 7fbad2c92640 1 --2- 172.21.15.106:0/133673292 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbacc0cae80 0x7fbacc147d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:58.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.696+0000 7fbad2c92640 1 -- 172.21.15.106:0/133673292 >> 172.21.15.106:0/133673292 conn(0x7fbacc06c9b0 msgr2=0x7fbacc153790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:58.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.696+0000 7fbad2c92640 1 -- 172.21.15.106:0/133673292 shutdown_connections 2024-04-25T10:30:58.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.696+0000 7fbad2c92640 1 -- 172.21.15.106:0/133673292 wait complete. 2024-04-25T10:30:58.708 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:30:58.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.817+0000 7fdbdcc69640 1 Processor -- start 2024-04-25T10:30:58.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.817+0000 7fdbdcc69640 1 -- start start 2024-04-25T10:30:58.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.818+0000 7fdbdcc69640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fdbd80cbff0 0x7fdbd80cb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:58.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.818+0000 7fdbdcc69640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fdbd80cce00 0x7fdbd80cd210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:58.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.818+0000 7fdbdcc69640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbd815dae0 con 0x7fdbd8156ec0 2024-04-25T10:30:58.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.818+0000 7fdbdcc69640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fdbd8161270 con 0x7fdbd80cbff0 2024-04-25T10:30:58.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.818+0000 7fdbdcc69640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fdbd81604b0 con 0x7fdbd80cce00 2024-04-25T10:30:58.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.819+0000 7fdbd659c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fdbd80cbff0 0x7fdbd80cb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:58.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.819+0000 7fdbd659c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fdbd80cbff0 0x7fdbd80cb270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35118/0 (socket says 172.21.15.106:35118) 2024-04-25T10:30:58.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.819+0000 7fdbd5d9b640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fdbd8156ec0 0x7fdbd80c61a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:53526/0 (socket says 172.21.15.106:53526) 2024-04-25T10:30:58.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.819+0000 7fdbd659c640 1 -- 172.21.15.106:0/2225929301 learned_addr learned my addr 172.21.15.106:0/2225929301 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:58.822 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.819+0000 7fdbd6d9d640 1 --2- 172.21.15.106:0/2225929301 >> v2:172.21.15.134:3300/0 conn(0x7fdbd80cce00 0x7fdbd80cd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:58.822 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.819+0000 7fdbd659c640 1 -- 172.21.15.106:0/2225929301 >> v2:172.21.15.134:3300/0 conn(0x7fdbd80cce00 msgr2=0x7fdbd80cd210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:58.822 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.819+0000 7fdbd659c640 1 --2- 172.21.15.106:0/2225929301 >> v2:172.21.15.134:3300/0 conn(0x7fdbd80cce00 0x7fdbd80cd210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:58.822 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.819+0000 7fdbd659c640 1 -- 172.21.15.106:0/2225929301 >> v1:172.21.15.106:6789/0 conn(0x7fdbd8156ec0 legacy=0x7fdbd80c61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:58.822 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.819+0000 7fdbd659c640 1 -- 172.21.15.106:0/2225929301 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbd8160790 con 0x7fdbd80cbff0 2024-04-25T10:30:58.822 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.819+0000 7fdbd559a640 1 -- 172.21.15.106:0/2225929301 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1576702396 0 0) 0x7fdbd815dae0 con 0x7fdbd8156ec0 2024-04-25T10:30:58.822 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.820+0000 7fdbd659c640 1 --2- 172.21.15.106:0/2225929301 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdbd80cbff0 0x7fdbd80cb270 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8073c10 tx=0x7fdbd002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c39209c6433e157 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:58.822 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.820+0000 7fdbd559a640 1 -- 172.21.15.106:0/2225929301 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd0037070 con 0x7fdbd80cbff0 2024-04-25T10:30:58.822 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.820+0000 7fdbd559a640 1 -- 172.21.15.106:0/2225929301 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbd002ae50 con 0x7fdbd80cbff0 2024-04-25T10:30:58.822 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.820+0000 7fdbd559a640 1 -- 172.21.15.106:0/2225929301 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd0032860 con 0x7fdbd80cbff0 2024-04-25T10:30:58.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.820+0000 7fdbdcc69640 1 -- 172.21.15.106:0/2225929301 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdbd80cbff0 msgr2=0x7fdbd80cb270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:58.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.820+0000 7fdbdcc69640 1 --2- 172.21.15.106:0/2225929301 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdbd80cbff0 0x7fdbd80cb270 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8073c10 tx=0x7fdbd002a300 comp rx=0 tx=0).stop 2024-04-25T10:30:58.824 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.820+0000 7fdbdcc69640 1 -- 172.21.15.106:0/2225929301 shutdown_connections 2024-04-25T10:30:58.824 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.820+0000 7fdbdcc69640 1 --2- 172.21.15.106:0/2225929301 >> v2:172.21.15.134:3300/0 conn(0x7fdbd80cce00 0x7fdbd80cd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:58.824 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.820+0000 7fdbdcc69640 1 --2- 172.21.15.106:0/2225929301 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdbd80cbff0 0x7fdbd80cb270 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:58.824 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.820+0000 7fdbdcc69640 1 -- 172.21.15.106:0/2225929301 >> 172.21.15.106:0/2225929301 conn(0x7fdbd80727d0 msgr2=0x7fdbd8072c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:58.824 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.820+0000 7fdbdcc69640 1 -- 172.21.15.106:0/2225929301 shutdown_connections 2024-04-25T10:30:58.824 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.821+0000 7fdbdcc69640 1 -- 172.21.15.106:0/2225929301 wait complete. 2024-04-25T10:30:58.824 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.822+0000 7fdbdcc69640 1 Processor -- start 2024-04-25T10:30:58.824 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.822+0000 7fdbdcc69640 1 -- start start 2024-04-25T10:30:58.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.824+0000 7fdbdcc69640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdbd80cce00 0x7fdbd81f33d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:58.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.824+0000 7fdbdcc69640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdbd8156ec0 0x7fdbd8150e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:58.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.824+0000 7fdbdcc69640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdbd8156610 0x7fdbd814b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:58.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.824+0000 7fdbdcc69640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd81572d0 con 0x7fdbd80cce00 2024-04-25T10:30:58.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.824+0000 7fdbdcc69640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd8160d50 con 0x7fdbd8156ec0 2024-04-25T10:30:58.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.824+0000 7fdbdcc69640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fdbd815d730 con 0x7fdbd8156610 2024-04-25T10:30:58.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.824+0000 7fdbd659c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdbd80cce00 0x7fdbd81f33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:58.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.824+0000 7fdbd659c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdbd80cce00 0x7fdbd81f33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35128/0 (socket says 172.21.15.106:35128) 2024-04-25T10:30:58.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.824+0000 7fdbd659c640 1 -- 172.21.15.106:0/807953881 learned_addr learned my addr 172.21.15.106:0/807953881 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:58.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.824+0000 7fdbd6d9d640 1 --2- 172.21.15.106:0/807953881 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdbd8156610 0x7fdbd814b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:58.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.824+0000 7fdbd5d9b640 1 --2- 172.21.15.106:0/807953881 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdbd8156ec0 0x7fdbd8150e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:58.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.824+0000 7fdbd659c640 1 -- 172.21.15.106:0/807953881 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdbd8156610 msgr2=0x7fdbd814b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:58.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.824+0000 7fdbd659c640 1 --2- 172.21.15.106:0/807953881 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdbd8156610 0x7fdbd814b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:58.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.824+0000 7fdbd659c640 1 -- 172.21.15.106:0/807953881 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdbd8156ec0 msgr2=0x7fdbd8150e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:58.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.824+0000 7fdbd659c640 1 --2- 172.21.15.106:0/807953881 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdbd8156ec0 0x7fdbd8150e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:58.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.824+0000 7fdbd659c640 1 -- 172.21.15.106:0/807953881 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbd80c9fc0 con 0x7fdbd80cce00 2024-04-25T10:30:58.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.824+0000 7fdbd659c640 1 --2- 172.21.15.106:0/807953881 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdbd80cce00 0x7fdbd81f33d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0000c00 tx=0x7fdbd00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:58.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.824+0000 7fdbceffd640 1 -- 172.21.15.106:0/807953881 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd0037040 con 0x7fdbd80cce00 2024-04-25T10:30:58.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.824+0000 7fdbceffd640 1 -- 172.21.15.106:0/807953881 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbd0004250 con 0x7fdbd80cce00 2024-04-25T10:30:58.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.824+0000 7fdbceffd640 1 -- 172.21.15.106:0/807953881 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd0003610 con 0x7fdbd80cce00 2024-04-25T10:30:58.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.824+0000 7fdbdcc69640 1 -- 172.21.15.106:0/807953881 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbd81f2910 con 0x7fdbd80cce00 2024-04-25T10:30:58.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.824+0000 7fdbdcc69640 1 -- 172.21.15.106:0/807953881 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbd81f2bd0 con 0x7fdbd80cce00 2024-04-25T10:30:58.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.825+0000 7fdbdcc69640 1 -- 172.21.15.106:0/807953881 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbd806a3d0 con 0x7fdbd80cce00 2024-04-25T10:30:58.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.826+0000 7fdbceffd640 1 -- 172.21.15.106:0/807953881 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fdbd003a070 con 0x7fdbd80cce00 2024-04-25T10:30:58.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.826+0000 7fdbceffd640 1 --2- 172.21.15.106:0/807953881 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fdbb80744b0 0x7fdbb80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:58.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.827+0000 7fdbd5d9b640 1 --2- 172.21.15.106:0/807953881 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fdbb80744b0 0x7fdbb80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:58.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.827+0000 7fdbceffd640 1 -- 172.21.15.106:0/807953881 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fdbd00b56d0 con 0x7fdbd80cce00 2024-04-25T10:30:58.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.827+0000 7fdbd5d9b640 1 --2- 172.21.15.106:0/807953881 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fdbb80744b0 0x7fdbb80769c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8076420 tx=0x7fdbc0007e00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:58.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:58.831+0000 7fdbceffd640 1 -- 172.21.15.106:0/807953881 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdbd0041020 con 0x7fdbd80cce00 2024-04-25T10:30:59.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.007+0000 7fdbdcc69640 1 -- 172.21.15.106:0/807953881 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fdbd80cbdd0 con 0x7fdbd80cce00 2024-04-25T10:30:59.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.008+0000 7fdbceffd640 1 -- 172.21.15.106:0/807953881 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fdbd007f8b0 con 0x7fdbd80cce00 2024-04-25T10:30:59.009 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-25T10:30:59.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.010+0000 7fdbdcc69640 1 -- 172.21.15.106:0/807953881 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fdbb80744b0 msgr2=0x7fdbb80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:59.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.010+0000 7fdbdcc69640 1 --2- 172.21.15.106:0/807953881 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fdbb80744b0 0x7fdbb80769c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8076420 tx=0x7fdbc0007e00 comp rx=0 tx=0).stop 2024-04-25T10:30:59.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.010+0000 7fdbdcc69640 1 -- 172.21.15.106:0/807953881 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdbd80cce00 msgr2=0x7fdbd81f33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:59.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.010+0000 7fdbdcc69640 1 --2- 172.21.15.106:0/807953881 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdbd80cce00 0x7fdbd81f33d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0000c00 tx=0x7fdbd00046c0 comp rx=0 tx=0).stop 2024-04-25T10:30:59.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.010+0000 7fdbdcc69640 1 -- 172.21.15.106:0/807953881 shutdown_connections 2024-04-25T10:30:59.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.010+0000 7fdbdcc69640 1 --2- 172.21.15.106:0/807953881 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fdbb80744b0 0x7fdbb80769c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:59.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.010+0000 7fdbdcc69640 1 --2- 172.21.15.106:0/807953881 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdbd8156610 0x7fdbd814b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:59.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.010+0000 7fdbdcc69640 1 --2- 172.21.15.106:0/807953881 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdbd8156ec0 0x7fdbd8150e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:59.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.010+0000 7fdbdcc69640 1 --2- 172.21.15.106:0/807953881 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdbd80cce00 0x7fdbd81f33d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:59.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.010+0000 7fdbdcc69640 1 -- 172.21.15.106:0/807953881 >> 172.21.15.106:0/807953881 conn(0x7fdbd80727d0 msgr2=0x7fdbd806f110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:59.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.010+0000 7fdbdcc69640 1 -- 172.21.15.106:0/807953881 shutdown_connections 2024-04-25T10:30:59.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.010+0000 7fdbdcc69640 1 -- 172.21.15.106:0/807953881 wait complete. 2024-04-25T10:30:59.022 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:30:59.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.129+0000 7f4f1bd46640 1 Processor -- start 2024-04-25T10:30:59.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.129+0000 7f4f1bd46640 1 -- start start 2024-04-25T10:30:59.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.131+0000 7f4f1bd46640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4f140cc600 0x7f4f140c5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:59.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.131+0000 7f4f1bd46640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f4f140c58d0 0x7f4f140cce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:59.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.131+0000 7f4f1bd46640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4f140cb2b0 0x7f4f140cd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:59.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.131+0000 7f4f1bd46640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f4f1415eea0 con 0x7f4f140cb2b0 2024-04-25T10:30:59.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.131+0000 7f4f1bd46640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f4f1415b8c0 con 0x7f4f140c58d0 2024-04-25T10:30:59.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.131+0000 7f4f1bd46640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f4f1415b5e0 con 0x7f4f140cc600 2024-04-25T10:30:59.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.131+0000 7f4f1a2e3640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4f140cb2b0 0x7f4f140cd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:59.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.131+0000 7f4f1a2e3640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4f140cb2b0 0x7f4f140cd3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35134/0 (socket says 172.21.15.106:35134) 2024-04-25T10:30:59.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.131+0000 7f4f1a2e3640 1 -- 172.21.15.106:0/4150189507 learned_addr learned my addr 172.21.15.106:0/4150189507 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:59.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.131+0000 7f4f192e1640 1 --2- 172.21.15.106:0/4150189507 >> v2:172.21.15.134:3300/0 conn(0x7f4f140c58d0 0x7f4f140cce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:59.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.131+0000 7f4f19ae2640 1 --2- 172.21.15.106:0/4150189507 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4f140cc600 0x7f4f140c5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:59.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.131+0000 7f4f1a2e3640 1 -- 172.21.15.106:0/4150189507 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4f140cc600 msgr2=0x7f4f140c5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:59.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.131+0000 7f4f1a2e3640 1 --2- 172.21.15.106:0/4150189507 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4f140cc600 0x7f4f140c5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:59.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.131+0000 7f4f1a2e3640 1 -- 172.21.15.106:0/4150189507 >> v2:172.21.15.134:3300/0 conn(0x7f4f140c58d0 msgr2=0x7f4f140cce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:59.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.131+0000 7f4f1a2e3640 1 --2- 172.21.15.106:0/4150189507 >> v2:172.21.15.134:3300/0 conn(0x7f4f140c58d0 0x7f4f140cce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:59.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.131+0000 7f4f1a2e3640 1 -- 172.21.15.106:0/4150189507 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f1415e0e0 con 0x7f4f140cb2b0 2024-04-25T10:30:59.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.132+0000 7f4f1a2e3640 1 --2- 172.21.15.106:0/4150189507 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4f140cb2b0 0x7f4f140cd3d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4f14073550 tx=0x7f4f0c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dac09aa9cad52404 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:59.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.132+0000 7f4f18ae0640 1 -- 172.21.15.106:0/4150189507 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c037070 con 0x7f4f140cb2b0 2024-04-25T10:30:59.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.132+0000 7f4f18ae0640 1 -- 172.21.15.106:0/4150189507 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f0c032440 con 0x7f4f140cb2b0 2024-04-25T10:30:59.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.132+0000 7f4f18ae0640 1 -- 172.21.15.106:0/4150189507 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c032810 con 0x7f4f140cb2b0 2024-04-25T10:30:59.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.133+0000 7f4f1bd46640 1 -- 172.21.15.106:0/4150189507 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4f140cb2b0 msgr2=0x7f4f140cd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:59.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.133+0000 7f4f1bd46640 1 --2- 172.21.15.106:0/4150189507 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4f140cb2b0 0x7f4f140cd3d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4f14073550 tx=0x7f4f0c02a1e0 comp rx=0 tx=0).stop 2024-04-25T10:30:59.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.133+0000 7f4f1bd46640 1 -- 172.21.15.106:0/4150189507 shutdown_connections 2024-04-25T10:30:59.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.133+0000 7f4f1bd46640 1 --2- 172.21.15.106:0/4150189507 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4f140cb2b0 0x7f4f140cd3d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:59.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.133+0000 7f4f1bd46640 1 --2- 172.21.15.106:0/4150189507 >> v2:172.21.15.134:3300/0 conn(0x7f4f140c58d0 0x7f4f140cce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:59.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.133+0000 7f4f1bd46640 1 --2- 172.21.15.106:0/4150189507 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4f140cc600 0x7f4f140c5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:59.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.133+0000 7f4f1bd46640 1 -- 172.21.15.106:0/4150189507 >> 172.21.15.106:0/4150189507 conn(0x7f4f140727d0 msgr2=0x7f4f14072c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:59.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.133+0000 7f4f1bd46640 1 -- 172.21.15.106:0/4150189507 shutdown_connections 2024-04-25T10:30:59.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.133+0000 7f4f1bd46640 1 -- 172.21.15.106:0/4150189507 wait complete. 2024-04-25T10:30:59.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.134+0000 7f4f1bd46640 1 Processor -- start 2024-04-25T10:30:59.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.134+0000 7f4f1bd46640 1 -- start start 2024-04-25T10:30:59.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.136+0000 7f4f1bd46640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4f140c58d0 0x7f4f141f3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:59.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.136+0000 7f4f1bd46640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f140cb2b0 0x7f4f141f6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:59.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.136+0000 7f4f1bd46640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4f140cc600 0x7f4f1414db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:59.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.136+0000 7f4f1bd46640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1415e980 con 0x7f4f140cc600 2024-04-25T10:30:59.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.136+0000 7f4f1bd46640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1415e6a0 con 0x7f4f140cb2b0 2024-04-25T10:30:59.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.136+0000 7f4f1bd46640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f4f140cbff0 con 0x7f4f140c58d0 2024-04-25T10:30:59.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.136+0000 7f4f1a2e3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4f140cc600 0x7f4f1414db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:59.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.136+0000 7f4f1a2e3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4f140cc600 0x7f4f1414db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35146/0 (socket says 172.21.15.106:35146) 2024-04-25T10:30:59.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.136+0000 7f4f1a2e3640 1 -- 172.21.15.106:0/2782113532 learned_addr learned my addr 172.21.15.106:0/2782113532 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:59.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.136+0000 7f4f192e1640 1 --2- 172.21.15.106:0/2782113532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f140cb2b0 0x7f4f141f6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:59.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.136+0000 7f4f19ae2640 1 --2- 172.21.15.106:0/2782113532 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4f140c58d0 0x7f4f141f3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:59.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.136+0000 7f4f1a2e3640 1 -- 172.21.15.106:0/2782113532 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4f140c58d0 msgr2=0x7f4f141f3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:59.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.136+0000 7f4f1a2e3640 1 --2- 172.21.15.106:0/2782113532 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4f140c58d0 0x7f4f141f3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:59.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.136+0000 7f4f1a2e3640 1 -- 172.21.15.106:0/2782113532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f140cb2b0 msgr2=0x7f4f141f6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:59.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.136+0000 7f4f1a2e3640 1 --2- 172.21.15.106:0/2782113532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f140cb2b0 0x7f4f141f6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:59.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.136+0000 7f4f1a2e3640 1 -- 172.21.15.106:0/2782113532 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f141f6d80 con 0x7f4f140cc600 2024-04-25T10:30:59.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.136+0000 7f4f1a2e3640 1 --2- 172.21.15.106:0/2782113532 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4f140cc600 0x7f4f1414db80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c0090b0 tx=0x7f4f0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:59.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.136+0000 7f4f0a7fc640 1 -- 172.21.15.106:0/2782113532 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c041020 con 0x7f4f140cc600 2024-04-25T10:30:59.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.136+0000 7f4f0a7fc640 1 -- 172.21.15.106:0/2782113532 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f0c03d640 con 0x7f4f140cc600 2024-04-25T10:30:59.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.136+0000 7f4f1bd46640 1 -- 172.21.15.106:0/2782113532 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f1406e180 con 0x7f4f140cc600 2024-04-25T10:30:59.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.136+0000 7f4f0a7fc640 1 -- 172.21.15.106:0/2782113532 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c037040 con 0x7f4f140cc600 2024-04-25T10:30:59.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.136+0000 7f4f1bd46640 1 -- 172.21.15.106:0/2782113532 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f1406e700 con 0x7f4f140cc600 2024-04-25T10:30:59.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.137+0000 7f4f1bd46640 1 -- 172.21.15.106:0/2782113532 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f141f6f50 con 0x7f4f140cc600 2024-04-25T10:30:59.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.138+0000 7f4f0a7fc640 1 -- 172.21.15.106:0/2782113532 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4f0c044020 con 0x7f4f140cc600 2024-04-25T10:30:59.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.139+0000 7f4f0a7fc640 1 --2- 172.21.15.106:0/2782113532 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f4ef80744b0 0x7f4ef80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:59.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.139+0000 7f4f19ae2640 1 --2- 172.21.15.106:0/2782113532 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f4ef80744b0 0x7f4ef80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:59.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.139+0000 7f4f0a7fc640 1 -- 172.21.15.106:0/2782113532 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4f0c0b56a0 con 0x7f4f140cc600 2024-04-25T10:30:59.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.139+0000 7f4f19ae2640 1 --2- 172.21.15.106:0/2782113532 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f4ef80744b0 0x7f4ef80769c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4f140749e0 tx=0x7f4f1000b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:59.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.143+0000 7f4f0a7fc640 1 -- 172.21.15.106:0/2782113532 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4f0c04b030 con 0x7f4f140cc600 2024-04-25T10:30:59.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.356+0000 7f4f1bd46640 1 -- 172.21.15.106:0/2782113532 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4f1406a3d0 con 0x7f4f140cc600 2024-04-25T10:30:59.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.356+0000 7f4f0a7fc640 1 -- 172.21.15.106:0/2782113532 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+6918 (secure 0 0 0) 0x7f4f0c07f900 con 0x7f4f140cc600 2024-04-25T10:30:59.358 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:30:59.358 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/2834155852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":2834155852},{"type":"v1","addr":"172.21.15.106:6835","nonce":2834155852}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:30:31.761107+0000","modified":"2024-04-25T10:30:54.574122+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4290,"mds_1":4287,"mds_2":4284,"mds_3":4269,"mds_4":4266},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4266":{"gid":4266,"name":"c","rank":4,"incarnation":16,"state":"up:creating","state_seq":1,"addr":"172.21.15.106:6837/2632121774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":2632121774},{"type":"v1","addr":"172.21.15.106:6837","nonce":2632121774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4269":{"gid":4269,"name":"e","rank":3,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.106:6839/486339834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":486339834},{"type":"v1","addr":"172.21.15.106:6839","nonce":486339834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4284":{"gid":4284,"name":"b","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.134:6836/2781656864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6833","nonce":2781656864},{"type":"v1","addr":"172.21.15.134:6836","nonce":2781656864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4287":{"gid":4287,"name":"f","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.134:6835/1993918351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1993918351},{"type":"v1","addr":"172.21.15.134:6835","nonce":1993918351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4290":{"gid":4290,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.134:6837/2662003420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2662003420},{"type":"v1","addr":"172.21.15.134:6837","nonce":2662003420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4290,"qdb_cluster":[4290,4287,4284,4269]},"id":1}]} 2024-04-25T10:30:59.358 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 16 2024-04-25T10:30:59.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.358+0000 7f4f1bd46640 1 -- 172.21.15.106:0/2782113532 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f4ef80744b0 msgr2=0x7f4ef80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:59.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.358+0000 7f4f1bd46640 1 --2- 172.21.15.106:0/2782113532 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f4ef80744b0 0x7f4ef80769c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4f140749e0 tx=0x7f4f1000b040 comp rx=0 tx=0).stop 2024-04-25T10:30:59.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.358+0000 7f4f1bd46640 1 -- 172.21.15.106:0/2782113532 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4f140cc600 msgr2=0x7f4f1414db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:59.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.358+0000 7f4f1bd46640 1 --2- 172.21.15.106:0/2782113532 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4f140cc600 0x7f4f1414db80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c0090b0 tx=0x7f4f0c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:30:59.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.358+0000 7f4f1bd46640 1 -- 172.21.15.106:0/2782113532 shutdown_connections 2024-04-25T10:30:59.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.358+0000 7f4f1bd46640 1 --2- 172.21.15.106:0/2782113532 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f4ef80744b0 0x7f4ef80769c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:59.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.358+0000 7f4f1bd46640 1 --2- 172.21.15.106:0/2782113532 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4f140cc600 0x7f4f1414db80 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:59.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.358+0000 7f4f1bd46640 1 --2- 172.21.15.106:0/2782113532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f140cb2b0 0x7f4f141f6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:59.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.358+0000 7f4f1bd46640 1 --2- 172.21.15.106:0/2782113532 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4f140c58d0 0x7f4f141f3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:59.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.358+0000 7f4f1bd46640 1 -- 172.21.15.106:0/2782113532 >> 172.21.15.106:0/2782113532 conn(0x7f4f140727d0 msgr2=0x7f4f1415d840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:59.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.359+0000 7f4f1bd46640 1 -- 172.21.15.106:0/2782113532 shutdown_connections 2024-04-25T10:30:59.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.359+0000 7f4f1bd46640 1 -- 172.21.15.106:0/2782113532 wait complete. 2024-04-25T10:30:59.371 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:30:59.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.477+0000 7f55c1e77640 1 Processor -- start 2024-04-25T10:30:59.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.477+0000 7f55c1e77640 1 -- start start 2024-04-25T10:30:59.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.479+0000 7f55c1e77640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f55bc0cb270 0x7f55bc156ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:59.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.479+0000 7f55c1e77640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f55bc0cce00 0x7f55bc0cd210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:59.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.479+0000 7f55c1e77640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55bc15dae0 con 0x7f55bc0cbff0 2024-04-25T10:30:59.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.479+0000 7f55c1e77640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f55bc161270 con 0x7f55bc0cb270 2024-04-25T10:30:59.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.479+0000 7f55c1e77640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f55bc157400 con 0x7f55bc0cce00 2024-04-25T10:30:59.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.479+0000 7f55b3fff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f55bc0cb270 0x7f55bc156ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:59.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.479+0000 7f55bbfff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f55bc0cce00 0x7f55bc0cd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:59.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.479+0000 7f55bb7fe640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f55bc0cbff0 0x7f55bc0c61a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.106:41202/0 (socket says 172.21.15.106:41202) 2024-04-25T10:30:59.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.479+0000 7f55bb7fe640 1 -- 172.21.15.106:0/4078038958 learned_addr learned my addr 172.21.15.106:0/4078038958 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:30:59.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.479+0000 7f55bbfff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f55bc0cce00 0x7f55bc0cd210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.106:54652/0 (socket says 172.21.15.106:54652) 2024-04-25T10:30:59.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.479+0000 7f55b3fff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f55bc0cb270 0x7f55bc156ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.106:55562/0 (socket says 172.21.15.106:55562) 2024-04-25T10:30:59.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.479+0000 7f55bbfff640 1 -- 172.21.15.106:0/4078038958 >> v2:172.21.15.134:3300/0 conn(0x7f55bc0cb270 msgr2=0x7f55bc156ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:59.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.479+0000 7f55bbfff640 1 --2- 172.21.15.106:0/4078038958 >> v2:172.21.15.134:3300/0 conn(0x7f55bc0cb270 0x7f55bc156ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:59.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.479+0000 7f55bbfff640 1 -- 172.21.15.106:0/4078038958 >> v1:172.21.15.134:6789/0 conn(0x7f55bc0cbff0 legacy=0x7f55bc0c61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:59.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.480+0000 7f55bbfff640 1 -- 172.21.15.106:0/4078038958 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55bc1604b0 con 0x7f55bc0cce00 2024-04-25T10:30:59.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.479+0000 7f55baffd640 1 -- 172.21.15.106:0/4078038958 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1360112811 0 0) 0x7f55bc15dae0 con 0x7f55bc0cbff0 2024-04-25T10:30:59.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.480+0000 7f55b3fff640 1 --2- 172.21.15.106:0/4078038958 >> v2:172.21.15.134:3300/0 conn(0x7f55bc0cb270 0x7f55bc156ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:59.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.480+0000 7f55bbfff640 1 --2- 172.21.15.106:0/4078038958 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f55bc0cce00 0x7f55bc0cd210 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f55b4009d90 tx=0x7f55b402a6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b390586444f7bfee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:59.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.480+0000 7f55baffd640 1 -- 172.21.15.106:0/4078038958 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b4007740 con 0x7f55bc0cce00 2024-04-25T10:30:59.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.480+0000 7f55baffd640 1 -- 172.21.15.106:0/4078038958 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55b40078e0 con 0x7f55bc0cce00 2024-04-25T10:30:59.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.480+0000 7f55baffd640 1 -- 172.21.15.106:0/4078038958 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b4007cb0 con 0x7f55bc0cce00 2024-04-25T10:30:59.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.481+0000 7f55c1e77640 1 -- 172.21.15.106:0/4078038958 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f55bc0cce00 msgr2=0x7f55bc0cd210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:59.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.481+0000 7f55c1e77640 1 --2- 172.21.15.106:0/4078038958 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f55bc0cce00 0x7f55bc0cd210 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f55b4009d90 tx=0x7f55b402a6f0 comp rx=0 tx=0).stop 2024-04-25T10:30:59.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.481+0000 7f55c1e77640 1 -- 172.21.15.106:0/4078038958 shutdown_connections 2024-04-25T10:30:59.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.481+0000 7f55c1e77640 1 --2- 172.21.15.106:0/4078038958 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f55bc0cce00 0x7f55bc0cd210 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:59.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.481+0000 7f55c1e77640 1 --2- 172.21.15.106:0/4078038958 >> v2:172.21.15.134:3300/0 conn(0x7f55bc0cb270 0x7f55bc156ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:59.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.481+0000 7f55c1e77640 1 -- 172.21.15.106:0/4078038958 >> 172.21.15.106:0/4078038958 conn(0x7f55bc0727d0 msgr2=0x7f55bc072c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:59.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.481+0000 7f55c1e77640 1 -- 172.21.15.106:0/4078038958 shutdown_connections 2024-04-25T10:30:59.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.481+0000 7f55c1e77640 1 -- 172.21.15.106:0/4078038958 wait complete. 2024-04-25T10:30:59.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.483+0000 7f55c1e77640 1 Processor -- start 2024-04-25T10:30:59.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.483+0000 7f55c1e77640 1 -- start start 2024-04-25T10:30:59.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.484+0000 7f55c1e77640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f55bc0cb270 0x7f55bc151cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:59.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.484+0000 7f55c1e77640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55bc0cce00 0x7f55bc0c8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:59.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.484+0000 7f55c1e77640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55bc0d04d0 0x7f55bc0d08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:59.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.484+0000 7f55c1e77640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f55bc160d50 con 0x7f55bc0d04d0 2024-04-25T10:30:59.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.484+0000 7f55c1e77640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f55bc160a70 con 0x7f55bc0cce00 2024-04-25T10:30:59.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.484+0000 7f55c1e77640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f55bc15d730 con 0x7f55bc0cb270 2024-04-25T10:30:59.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.484+0000 7f55bbfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55bc0d04d0 0x7f55bc0d08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:59.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.484+0000 7f55bb7fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f55bc0cb270 0x7f55bc151cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:59.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.484+0000 7f55b3fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55bc0cce00 0x7f55bc0c8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:59.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.484+0000 7f55bbfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55bc0d04d0 0x7f55bc0d08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35156/0 (socket says 172.21.15.106:35156) 2024-04-25T10:30:59.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.484+0000 7f55bbfff640 1 -- 172.21.15.106:0/3428912453 learned_addr learned my addr 172.21.15.106:0/3428912453 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:59.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.484+0000 7f55bb7fe640 1 -- 172.21.15.106:0/3428912453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55bc0cce00 msgr2=0x7f55bc0c8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:59.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.484+0000 7f55bb7fe640 1 --2- 172.21.15.106:0/3428912453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55bc0cce00 0x7f55bc0c8ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:59.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.484+0000 7f55bb7fe640 1 -- 172.21.15.106:0/3428912453 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55bc0d04d0 msgr2=0x7f55bc0d08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:59.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.484+0000 7f55bb7fe640 1 --2- 172.21.15.106:0/3428912453 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55bc0d04d0 0x7f55bc0d08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:59.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.484+0000 7f55bb7fe640 1 -- 172.21.15.106:0/3428912453 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55bc15b840 con 0x7f55bc0cb270 2024-04-25T10:30:59.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.485+0000 7f55bbfff640 1 --2- 172.21.15.106:0/3428912453 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55bc0d04d0 0x7f55bc0d08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:59.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.485+0000 7f55b3fff640 1 --2- 172.21.15.106:0/3428912453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55bc0cce00 0x7f55bc0c8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:59.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.485+0000 7f55bb7fe640 1 --2- 172.21.15.106:0/3428912453 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f55bc0cb270 0x7f55bc151cd0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f55ac000c00 tx=0x7f55ac00c7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:59.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.485+0000 7f55b8ff9640 1 -- 172.21.15.106:0/3428912453 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ac017070 con 0x7f55bc0cb270 2024-04-25T10:30:59.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.485+0000 7f55b8ff9640 1 -- 172.21.15.106:0/3428912453 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55ac005ce0 con 0x7f55bc0cb270 2024-04-25T10:30:59.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.485+0000 7f55b8ff9640 1 -- 172.21.15.106:0/3428912453 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ac007310 con 0x7f55bc0cb270 2024-04-25T10:30:59.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.485+0000 7f55c1e77640 1 -- 172.21.15.106:0/3428912453 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55bc1fb400 con 0x7f55bc0cb270 2024-04-25T10:30:59.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.485+0000 7f55c1e77640 1 -- 172.21.15.106:0/3428912453 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f55bc1fa920 con 0x7f55bc0cb270 2024-04-25T10:30:59.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.486+0000 7f55c1e77640 1 -- 172.21.15.106:0/3428912453 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55bc1599c0 con 0x7f55bc0cb270 2024-04-25T10:30:59.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.486+0000 7f55b8ff9640 1 -- 172.21.15.106:0/3428912453 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f55ac01a070 con 0x7f55bc0cb270 2024-04-25T10:30:59.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.487+0000 7f55b8ff9640 1 --2- 172.21.15.106:0/3428912453 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f559c0744b0 0x7f559c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:59.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.487+0000 7f55b3fff640 1 --2- 172.21.15.106:0/3428912453 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f559c0744b0 0x7f559c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:59.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.487+0000 7f55b8ff9640 1 -- 172.21.15.106:0/3428912453 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f55ac0944e0 con 0x7f55bc0cb270 2024-04-25T10:30:59.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.487+0000 7f55b3fff640 1 --2- 172.21.15.106:0/3428912453 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f559c0744b0 0x7f559c0769c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f55a4009560 tx=0x7f55a4009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:59.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.492+0000 7f55b8ff9640 1 -- 172.21.15.106:0/3428912453 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f55ac05e6b0 con 0x7f55bc0cb270 2024-04-25T10:30:59.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.675+0000 7f55c1e77640 1 -- 172.21.15.106:0/3428912453 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f55bc0c59c0 con 0x7f55bc0cb270 2024-04-25T10:30:59.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.677+0000 7f55b8ff9640 1 -- 172.21.15.106:0/3428912453 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16641 (secure 0 0 0) 0x7f55bc0c59c0 con 0x7f55bc0cb270 2024-04-25T10:30:59.678 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:30:59.679 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":20,"fsid":"3ee77985-e8e4-41cf-81e9-7c774cf7891f","created":"2024-04-25T10:30:07.981054+0000","modified":"2024-04-25T10:30:33.409267+0000","last_up_change":"2024-04-25T10:30:19.184325+0000","last_in_change":"2024-04-25T10:30:10.217553+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:30:20.111046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:30:30.314914+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:30:30.715500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a5962cb1-dbbd-47b1-8d15-9bae114bfdf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1240717079},{"type":"v1","addr":"172.21.15.106:6801","nonce":1240717079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1240717079},{"type":"v1","addr":"172.21.15.106:6803","nonce":1240717079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1240717079},{"type":"v1","addr":"172.21.15.106:6807","nonce":1240717079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1240717079},{"type":"v1","addr":"172.21.15.106:6805","nonce":1240717079}]},"public_addr":"172.21.15.106:6801/1240717079","cluster_addr":"172.21.15.106:6803/1240717079","heartbeat_back_addr":"172.21.15.106:6807/1240717079","heartbeat_front_addr":"172.21.15.106:6805/1240717079","state":["exists","up"]},{"osd":1,"uuid":"94436e13-4769-4752-8397-b5e396a6f7ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4089653051},{"type":"v1","addr":"172.21.15.106:6825","nonce":4089653051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4089653051},{"type":"v1","addr":"172.21.15.106:6827","nonce":4089653051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4089653051},{"type":"v1","addr":"172.21.15.106:6831","nonce":4089653051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4089653051},{"type":"v1","addr":"172.21.15.106:6829","nonce":4089653051}]},"public_addr":"172.21.15.106:6825/4089653051","cluster_addr":"172.21.15.106:6827/4089653051","heartbeat_back_addr":"172.21.15.106:6831/4089653051","heartbeat_front_addr":"172.21.15.106:6829/4089653051","state":["exists","up"]},{"osd":2,"uuid":"429aae0b-70c6-4e7b-9043-6a11334c7efc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":1441018239},{"type":"v1","addr":"172.21.15.106:6809","nonce":1441018239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1441018239},{"type":"v1","addr":"172.21.15.106:6811","nonce":1441018239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1441018239},{"type":"v1","addr":"172.21.15.106:6815","nonce":1441018239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":1441018239},{"type":"v1","addr":"172.21.15.106:6813","nonce":1441018239}]},"public_addr":"172.21.15.106:6809/1441018239","cluster_addr":"172.21.15.106:6811/1441018239","heartbeat_back_addr":"172.21.15.106:6815/1441018239","heartbeat_front_addr":"172.21.15.106:6813/1441018239","state":["exists","up"]},{"osd":3,"uuid":"9593a4d1-772f-4667-ab0d-2982f841e342","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":1846968711},{"type":"v1","addr":"172.21.15.106:6817","nonce":1846968711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1846968711},{"type":"v1","addr":"172.21.15.106:6819","nonce":1846968711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":1846968711},{"type":"v1","addr":"172.21.15.106:6823","nonce":1846968711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":1846968711},{"type":"v1","addr":"172.21.15.106:6821","nonce":1846968711}]},"public_addr":"172.21.15.106:6817/1846968711","cluster_addr":"172.21.15.106:6819/1846968711","heartbeat_back_addr":"172.21.15.106:6823/1846968711","heartbeat_front_addr":"172.21.15.106:6821/1846968711","state":["exists","up"]},{"osd":4,"uuid":"1b61f16d-5c60-480a-b449-c745aa3f6b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":729224739},{"type":"v1","addr":"172.21.15.134:6825","nonce":729224739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":729224739},{"type":"v1","addr":"172.21.15.134:6827","nonce":729224739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":729224739},{"type":"v1","addr":"172.21.15.134:6831","nonce":729224739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":729224739},{"type":"v1","addr":"172.21.15.134:6829","nonce":729224739}]},"public_addr":"172.21.15.134:6825/729224739","cluster_addr":"172.21.15.134:6827/729224739","heartbeat_back_addr":"172.21.15.134:6831/729224739","heartbeat_front_addr":"172.21.15.134:6829/729224739","state":["exists","up"]},{"osd":5,"uuid":"944cdd7f-47c0-4b71-b7e0-d093542f4528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3132275652},{"type":"v1","addr":"172.21.15.134:6809","nonce":3132275652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3132275652},{"type":"v1","addr":"172.21.15.134:6811","nonce":3132275652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3132275652},{"type":"v1","addr":"172.21.15.134:6815","nonce":3132275652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3132275652},{"type":"v1","addr":"172.21.15.134:6813","nonce":3132275652}]},"public_addr":"172.21.15.134:6809/3132275652","cluster_addr":"172.21.15.134:6811/3132275652","heartbeat_back_addr":"172.21.15.134:6815/3132275652","heartbeat_front_addr":"172.21.15.134:6813/3132275652","state":["exists","up"]},{"osd":6,"uuid":"cc2446e2-1d21-4135-ac04-6af7c990058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":992702668},{"type":"v1","addr":"172.21.15.134:6801","nonce":992702668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":992702668},{"type":"v1","addr":"172.21.15.134:6803","nonce":992702668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":992702668},{"type":"v1","addr":"172.21.15.134:6807","nonce":992702668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":992702668},{"type":"v1","addr":"172.21.15.134:6805","nonce":992702668}]},"public_addr":"172.21.15.134:6801/992702668","cluster_addr":"172.21.15.134:6803/992702668","heartbeat_back_addr":"172.21.15.134:6807/992702668","heartbeat_front_addr":"172.21.15.134:6805/992702668","state":["exists","up"]},{"osd":7,"uuid":"3df1fc7a-82fc-4257-8629-98027b6e2aed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":3291062284},{"type":"v1","addr":"172.21.15.134:6817","nonce":3291062284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":3291062284},{"type":"v1","addr":"172.21.15.134:6819","nonce":3291062284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":3291062284},{"type":"v1","addr":"172.21.15.134:6823","nonce":3291062284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":3291062284},{"type":"v1","addr":"172.21.15.134:6821","nonce":3291062284}]},"public_addr":"172.21.15.134:6817/3291062284","cluster_addr":"172.21.15.134:6819/3291062284","heartbeat_back_addr":"172.21.15.134:6823/3291062284","heartbeat_front_addr":"172.21.15.134:6821/3291062284","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.403941+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.382563+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.411544+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.330444+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.405430+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.387737+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.408922+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.454450+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:30:59.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.679+0000 7f55c1e77640 1 -- 172.21.15.106:0/3428912453 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f559c0744b0 msgr2=0x7f559c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:59.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.679+0000 7f55c1e77640 1 --2- 172.21.15.106:0/3428912453 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f559c0744b0 0x7f559c0769c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f55a4009560 tx=0x7f55a4009190 comp rx=0 tx=0).stop 2024-04-25T10:30:59.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.679+0000 7f55c1e77640 1 -- 172.21.15.106:0/3428912453 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f55bc0cb270 msgr2=0x7f55bc151cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:59.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.679+0000 7f55c1e77640 1 --2- 172.21.15.106:0/3428912453 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f55bc0cb270 0x7f55bc151cd0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f55ac000c00 tx=0x7f55ac00c7f0 comp rx=0 tx=0).stop 2024-04-25T10:30:59.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.679+0000 7f55c1e77640 1 -- 172.21.15.106:0/3428912453 shutdown_connections 2024-04-25T10:30:59.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.679+0000 7f55c1e77640 1 --2- 172.21.15.106:0/3428912453 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f559c0744b0 0x7f559c0769c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:59.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.679+0000 7f55c1e77640 1 --2- 172.21.15.106:0/3428912453 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55bc0d04d0 0x7f55bc0d08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:59.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.679+0000 7f55c1e77640 1 --2- 172.21.15.106:0/3428912453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55bc0cce00 0x7f55bc0c8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:59.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.679+0000 7f55c1e77640 1 --2- 172.21.15.106:0/3428912453 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f55bc0cb270 0x7f55bc151cd0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:59.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.679+0000 7f55c1e77640 1 -- 172.21.15.106:0/3428912453 >> 172.21.15.106:0/3428912453 conn(0x7f55bc0727d0 msgr2=0x7f55bc1529d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:59.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.679+0000 7f55c1e77640 1 -- 172.21.15.106:0/3428912453 shutdown_connections 2024-04-25T10:30:59.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.679+0000 7f55c1e77640 1 -- 172.21.15.106:0/3428912453 wait complete. 2024-04-25T10:30:59.692 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:30:59.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.799+0000 7ff3cca96640 1 Processor -- start 2024-04-25T10:30:59.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.799+0000 7ff3cca96640 1 -- start start 2024-04-25T10:30:59.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.800+0000 7ff3cca96640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3c80c5400 0x7ff3c80c5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:59.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.800+0000 7ff3cca96640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff3c815e020 0x7ff3c815e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:59.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.800+0000 7ff3cca96640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3c80caa00 con 0x7ff3c8154c60 2024-04-25T10:30:59.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.800+0000 7ff3cca96640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7ff3c80cb880 con 0x7ff3c815e020 2024-04-25T10:30:59.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.800+0000 7ff3cca96640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff3c80cac70 con 0x7ff3c80c5400 2024-04-25T10:30:59.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.800+0000 7ff3c659c640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7ff3c8154c60 0x7ff3c8070ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:53534/0 (socket says 172.21.15.106:53534) 2024-04-25T10:30:59.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.800+0000 7ff3c659c640 1 -- 172.21.15.106:0/1425620937 learned_addr learned my addr 172.21.15.106:0/1425620937 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:30:59.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.800+0000 7ff3c6d9d640 1 --2- 172.21.15.106:0/1425620937 >> v2:172.21.15.106:3300/0 conn(0x7ff3c815e020 0x7ff3c815e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:59.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.800+0000 7ff3bffff640 1 --2- 172.21.15.106:0/1425620937 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3c80c5400 0x7ff3c80c5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:59.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.801+0000 7ff3c6d9d640 1 -- 172.21.15.106:0/1425620937 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3c80c5400 msgr2=0x7ff3c80c5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:59.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.801+0000 7ff3c6d9d640 1 --2- 172.21.15.106:0/1425620937 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3c80c5400 0x7ff3c80c5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:59.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.801+0000 7ff3c6d9d640 1 -- 172.21.15.106:0/1425620937 >> v1:172.21.15.106:6789/0 conn(0x7ff3c8154c60 legacy=0x7ff3c8070ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:59.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.801+0000 7ff3c6d9d640 1 -- 172.21.15.106:0/1425620937 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3c80caf50 con 0x7ff3c815e020 2024-04-25T10:30:59.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.801+0000 7ff3c5d9b640 1 -- 172.21.15.106:0/1425620937 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 664711722 0 0) 0x7ff3c80caa00 con 0x7ff3c8154c60 2024-04-25T10:30:59.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.801+0000 7ff3c6d9d640 1 --2- 172.21.15.106:0/1425620937 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff3c815e020 0x7ff3c815e430 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff3c0009dc0 tx=0x7ff3c002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbbaf8e1716b2a08 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:59.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.801+0000 7ff3c5d9b640 1 -- 172.21.15.106:0/1425620937 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c0007740 con 0x7ff3c815e020 2024-04-25T10:30:59.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.801+0000 7ff3c5d9b640 1 -- 172.21.15.106:0/1425620937 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3c00078e0 con 0x7ff3c815e020 2024-04-25T10:30:59.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.801+0000 7ff3c5d9b640 1 -- 172.21.15.106:0/1425620937 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c0007cb0 con 0x7ff3c815e020 2024-04-25T10:30:59.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.802+0000 7ff3cca96640 1 -- 172.21.15.106:0/1425620937 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff3c815e020 msgr2=0x7ff3c815e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:59.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.802+0000 7ff3cca96640 1 --2- 172.21.15.106:0/1425620937 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff3c815e020 0x7ff3c815e430 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff3c0009dc0 tx=0x7ff3c002a710 comp rx=0 tx=0).stop 2024-04-25T10:30:59.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.802+0000 7ff3cca96640 1 -- 172.21.15.106:0/1425620937 shutdown_connections 2024-04-25T10:30:59.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.802+0000 7ff3cca96640 1 --2- 172.21.15.106:0/1425620937 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff3c815e020 0x7ff3c815e430 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:59.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.802+0000 7ff3cca96640 1 --2- 172.21.15.106:0/1425620937 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3c80c5400 0x7ff3c80c5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:59.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.802+0000 7ff3cca96640 1 -- 172.21.15.106:0/1425620937 >> 172.21.15.106:0/1425620937 conn(0x7ff3c80cff60 msgr2=0x7ff3c80d0370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:59.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.802+0000 7ff3cca96640 1 -- 172.21.15.106:0/1425620937 shutdown_connections 2024-04-25T10:30:59.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.802+0000 7ff3cca96640 1 -- 172.21.15.106:0/1425620937 wait complete. 2024-04-25T10:30:59.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.804+0000 7ff3cca96640 1 Processor -- start 2024-04-25T10:30:59.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.804+0000 7ff3cca96640 1 -- start start 2024-04-25T10:30:59.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.805+0000 7ff3cca96640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3c80c5400 0x7ff3c80ce930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:59.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.805+0000 7ff3cca96640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff3c8154c60 0x7ff3c814f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:59.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.805+0000 7ff3cca96640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3c815e020 0x7ff3c80cc420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:59.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.805+0000 7ff3cca96640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c80713f0 con 0x7ff3c8154c60 2024-04-25T10:30:59.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.805+0000 7ff3cca96640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c80cb510 con 0x7ff3c80c5400 2024-04-25T10:30:59.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.805+0000 7ff3cca96640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff3c8071570 con 0x7ff3c815e020 2024-04-25T10:30:59.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.805+0000 7ff3bffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff3c8154c60 0x7ff3c814f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:59.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.805+0000 7ff3bffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff3c8154c60 0x7ff3c814f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35180/0 (socket says 172.21.15.106:35180) 2024-04-25T10:30:59.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.805+0000 7ff3bffff640 1 -- 172.21.15.106:0/1438107326 learned_addr learned my addr 172.21.15.106:0/1438107326 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:30:59.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.805+0000 7ff3c659c640 1 --2- 172.21.15.106:0/1438107326 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3c80c5400 0x7ff3c80ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:59.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.805+0000 7ff3c6d9d640 1 --2- 172.21.15.106:0/1438107326 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3c815e020 0x7ff3c80cc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:59.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.806+0000 7ff3bffff640 1 -- 172.21.15.106:0/1438107326 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3c815e020 msgr2=0x7ff3c80cc420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:59.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.806+0000 7ff3bffff640 1 --2- 172.21.15.106:0/1438107326 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3c815e020 0x7ff3c80cc420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:59.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.806+0000 7ff3bffff640 1 -- 172.21.15.106:0/1438107326 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3c80c5400 msgr2=0x7ff3c80ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:59.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.806+0000 7ff3bffff640 1 --2- 172.21.15.106:0/1438107326 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3c80c5400 0x7ff3c80ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:59.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.806+0000 7ff3bffff640 1 -- 172.21.15.106:0/1438107326 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3c80cee70 con 0x7ff3c8154c60 2024-04-25T10:30:59.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.806+0000 7ff3bffff640 1 --2- 172.21.15.106:0/1438107326 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff3c8154c60 0x7ff3c814f8a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff3b000e8a0 tx=0x7ff3b000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:59.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.806+0000 7ff3beffd640 1 -- 172.21.15.106:0/1438107326 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b0017070 con 0x7ff3c8154c60 2024-04-25T10:30:59.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.806+0000 7ff3beffd640 1 -- 172.21.15.106:0/1438107326 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3b000c940 con 0x7ff3c8154c60 2024-04-25T10:30:59.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.806+0000 7ff3cca96640 1 -- 172.21.15.106:0/1438107326 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3c80cde90 con 0x7ff3c8154c60 2024-04-25T10:30:59.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.806+0000 7ff3beffd640 1 -- 172.21.15.106:0/1438107326 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b000cd30 con 0x7ff3c8154c60 2024-04-25T10:30:59.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.806+0000 7ff3cca96640 1 -- 172.21.15.106:0/1438107326 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3c815cc30 con 0x7ff3c8154c60 2024-04-25T10:30:59.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.807+0000 7ff3cca96640 1 -- 172.21.15.106:0/1438107326 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3c806a3d0 con 0x7ff3c8154c60 2024-04-25T10:30:59.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.808+0000 7ff3beffd640 1 -- 172.21.15.106:0/1438107326 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff3b001a070 con 0x7ff3c8154c60 2024-04-25T10:30:59.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.808+0000 7ff3beffd640 1 --2- 172.21.15.106:0/1438107326 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7ff3a80744b0 0x7ff3a80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:59.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.808+0000 7ff3c659c640 1 --2- 172.21.15.106:0/1438107326 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7ff3a80744b0 0x7ff3a80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:59.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.808+0000 7ff3beffd640 1 -- 172.21.15.106:0/1438107326 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff3b0094fb0 con 0x7ff3c8154c60 2024-04-25T10:30:59.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.809+0000 7ff3c659c640 1 --2- 172.21.15.106:0/1438107326 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7ff3a80744b0 0x7ff3a80769c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8007fd0 tx=0x7ff3b8007de0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:59.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:30:59.813+0000 7ff3beffd640 1 -- 172.21.15.106:0/1438107326 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff3b005f190 con 0x7ff3c8154c60 2024-04-25T10:31:00.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:00.022+0000 7ff3cca96640 1 -- 172.21.15.106:0/1438107326 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff3c80cbe80 con 0x7ff3c8154c60 2024-04-25T10:31:00.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:00.023+0000 7ff3beffd640 1 -- 172.21.15.106:0/1438107326 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+6918 (secure 0 0 0) 0x7ff3b0063490 con 0x7ff3c8154c60 2024-04-25T10:31:00.024 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:31:00.024 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/2834155852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":2834155852},{"type":"v1","addr":"172.21.15.106:6835","nonce":2834155852}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:30:31.761107+0000","modified":"2024-04-25T10:30:54.574122+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4290,"mds_1":4287,"mds_2":4284,"mds_3":4269,"mds_4":4266},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4266":{"gid":4266,"name":"c","rank":4,"incarnation":16,"state":"up:creating","state_seq":1,"addr":"172.21.15.106:6837/2632121774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":2632121774},{"type":"v1","addr":"172.21.15.106:6837","nonce":2632121774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4269":{"gid":4269,"name":"e","rank":3,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.106:6839/486339834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":486339834},{"type":"v1","addr":"172.21.15.106:6839","nonce":486339834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4284":{"gid":4284,"name":"b","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.134:6836/2781656864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6833","nonce":2781656864},{"type":"v1","addr":"172.21.15.134:6836","nonce":2781656864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4287":{"gid":4287,"name":"f","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.134:6835/1993918351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1993918351},{"type":"v1","addr":"172.21.15.134:6835","nonce":1993918351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4290":{"gid":4290,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.134:6837/2662003420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2662003420},{"type":"v1","addr":"172.21.15.134:6837","nonce":2662003420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4290,"qdb_cluster":[4290,4287,4284,4269]},"id":1}]} 2024-04-25T10:31:00.025 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 16 2024-04-25T10:31:00.026 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:00.025+0000 7ff3cca96640 1 -- 172.21.15.106:0/1438107326 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7ff3a80744b0 msgr2=0x7ff3a80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:00.026 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:00.025+0000 7ff3cca96640 1 --2- 172.21.15.106:0/1438107326 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7ff3a80744b0 0x7ff3a80769c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8007fd0 tx=0x7ff3b8007de0 comp rx=0 tx=0).stop 2024-04-25T10:31:00.026 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:00.025+0000 7ff3cca96640 1 -- 172.21.15.106:0/1438107326 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff3c8154c60 msgr2=0x7ff3c814f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:00.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:00.025+0000 7ff3cca96640 1 --2- 172.21.15.106:0/1438107326 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff3c8154c60 0x7ff3c814f8a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff3b000e8a0 tx=0x7ff3b000ed60 comp rx=0 tx=0).stop 2024-04-25T10:31:00.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:00.025+0000 7ff3cca96640 1 -- 172.21.15.106:0/1438107326 shutdown_connections 2024-04-25T10:31:00.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:00.025+0000 7ff3cca96640 1 --2- 172.21.15.106:0/1438107326 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7ff3a80744b0 0x7ff3a80769c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:00.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:00.025+0000 7ff3cca96640 1 --2- 172.21.15.106:0/1438107326 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3c815e020 0x7ff3c80cc420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:00.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:00.025+0000 7ff3cca96640 1 --2- 172.21.15.106:0/1438107326 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff3c8154c60 0x7ff3c814f8a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:00.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:00.025+0000 7ff3cca96640 1 --2- 172.21.15.106:0/1438107326 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3c80c5400 0x7ff3c80ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:00.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:00.025+0000 7ff3cca96640 1 -- 172.21.15.106:0/1438107326 >> 172.21.15.106:0/1438107326 conn(0x7ff3c80cff60 msgr2=0x7ff3c80b55b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:00.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:00.025+0000 7ff3cca96640 1 -- 172.21.15.106:0/1438107326 shutdown_connections 2024-04-25T10:31:00.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:00.025+0000 7ff3cca96640 1 -- 172.21.15.106:0/1438107326 wait complete. 2024-04-25T10:31:00.643 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:30:31.761107+0000', 'modified': '2024-04-25T10:30:54.574122+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 4290, 'mds_1': 4287, 'mds_2': 4284, 'mds_3': 4269, 'mds_4': 4266}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4266': {'gid': 4266, 'name': 'c', 'rank': 4, 'incarnation': 16, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.106:6837/2632121774', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6836', 'nonce': 2632121774}, {'type': 'v1', 'addr': '172.21.15.106:6837', 'nonce': 2632121774}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4269': {'gid': 4269, 'name': 'e', 'rank': 3, 'incarnation': 14, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.106:6839/486339834', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6838', 'nonce': 486339834}, {'type': 'v1', 'addr': '172.21.15.106:6839', 'nonce': 486339834}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4284': {'gid': 4284, 'name': 'b', 'rank': 2, 'incarnation': 12, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.134:6836/2781656864', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6833', 'nonce': 2781656864}, {'type': 'v1', 'addr': '172.21.15.134:6836', 'nonce': 2781656864}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4287': {'gid': 4287, 'name': 'f', 'rank': 1, 'incarnation': 10, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.134:6835/1993918351', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6832', 'nonce': 1993918351}, {'type': 'v1', 'addr': '172.21.15.134:6835', 'nonce': 1993918351}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4290': {'gid': 4290, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.134:6837/2662003420', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6834', 'nonce': 2662003420}, {'type': 'v1', 'addr': '172.21.15.134:6837', 'nonce': 2662003420}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4290, 'qdb_cluster': [4290, 4287, 4284, 4269]} 2024-04-25T10:31:00.643 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_4266:up:creating 2024-04-25T10:31:01.644 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:31:01.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.761+0000 7fb0c1346640 1 Processor -- start 2024-04-25T10:31:01.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.761+0000 7fb0c1346640 1 -- start start 2024-04-25T10:31:01.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.763+0000 7fb0c1346640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fb0bc0c7e30 0x7fb0bc0c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:01.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.763+0000 7fb0c1346640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb0bc0b4ce0 0x7fb0bc15ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:01.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.763+0000 7fb0c1346640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0bc0ce630 con 0x7fb0bc156ef0 2024-04-25T10:31:01.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.763+0000 7fb0c1346640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fb0bc0b50f0 con 0x7fb0bc0b4ce0 2024-04-25T10:31:01.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.763+0000 7fb0c1346640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fb0bc0c7c60 con 0x7fb0bc0c7e30 2024-04-25T10:31:01.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.763+0000 7fb0bb7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb0bc0b4ce0 0x7fb0bc15ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:01.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.763+0000 7fb0ba7fc640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fb0bc156ef0 0x7fb0bc06d000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:53544/0 (socket says 172.21.15.106:53544) 2024-04-25T10:31:01.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.763+0000 7fb0bb7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb0bc0b4ce0 0x7fb0bc15ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35192/0 (socket says 172.21.15.106:35192) 2024-04-25T10:31:01.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.763+0000 7fb0ba7fc640 1 -- 172.21.15.106:0/557480261 learned_addr learned my addr 172.21.15.106:0/557480261 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:31:01.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.763+0000 7fb0baffd640 1 --2- 172.21.15.106:0/557480261 >> v2:172.21.15.134:3300/0 conn(0x7fb0bc0c7e30 0x7fb0bc0c8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:01.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.763+0000 7fb0bb7fe640 1 -- 172.21.15.106:0/557480261 >> v2:172.21.15.134:3300/0 conn(0x7fb0bc0c7e30 msgr2=0x7fb0bc0c8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:01.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.763+0000 7fb0bb7fe640 1 --2- 172.21.15.106:0/557480261 >> v2:172.21.15.134:3300/0 conn(0x7fb0bc0c7e30 0x7fb0bc0c8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:01.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.763+0000 7fb0bb7fe640 1 -- 172.21.15.106:0/557480261 >> v1:172.21.15.106:6789/0 conn(0x7fb0bc156ef0 legacy=0x7fb0bc06d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:01.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.763+0000 7fb0bb7fe640 1 -- 172.21.15.106:0/557480261 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0bc0c7980 con 0x7fb0bc0b4ce0 2024-04-25T10:31:01.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.763+0000 7fb0b9ffb640 1 -- 172.21.15.106:0/557480261 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2068286154 0 0) 0x7fb0bc0ce630 con 0x7fb0bc156ef0 2024-04-25T10:31:01.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.764+0000 7fb0bb7fe640 1 --2- 172.21.15.106:0/557480261 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0bc0b4ce0 0x7fb0bc15ce30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc0c7de0 tx=0x7fb0b402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9264a35250accf0e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:01.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.764+0000 7fb0b9ffb640 1 -- 172.21.15.106:0/557480261 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b4031600 con 0x7fb0bc0b4ce0 2024-04-25T10:31:01.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.764+0000 7fb0b9ffb640 1 -- 172.21.15.106:0/557480261 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0b40317a0 con 0x7fb0bc0b4ce0 2024-04-25T10:31:01.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.764+0000 7fb0b9ffb640 1 -- 172.21.15.106:0/557480261 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b4031b70 con 0x7fb0bc0b4ce0 2024-04-25T10:31:01.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.765+0000 7fb0c1346640 1 -- 172.21.15.106:0/557480261 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0bc0b4ce0 msgr2=0x7fb0bc15ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:01.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.765+0000 7fb0c1346640 1 --2- 172.21.15.106:0/557480261 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0bc0b4ce0 0x7fb0bc15ce30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc0c7de0 tx=0x7fb0b402a5d0 comp rx=0 tx=0).stop 2024-04-25T10:31:01.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.765+0000 7fb0c1346640 1 -- 172.21.15.106:0/557480261 shutdown_connections 2024-04-25T10:31:01.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.765+0000 7fb0c1346640 1 --2- 172.21.15.106:0/557480261 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0bc0b4ce0 0x7fb0bc15ce30 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:01.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.765+0000 7fb0c1346640 1 --2- 172.21.15.106:0/557480261 >> v2:172.21.15.134:3300/0 conn(0x7fb0bc0c7e30 0x7fb0bc0c8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:01.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.765+0000 7fb0c1346640 1 -- 172.21.15.106:0/557480261 >> 172.21.15.106:0/557480261 conn(0x7fb0bc068b10 msgr2=0x7fb0bc0d1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:01.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.765+0000 7fb0c1346640 1 -- 172.21.15.106:0/557480261 shutdown_connections 2024-04-25T10:31:01.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.765+0000 7fb0c1346640 1 -- 172.21.15.106:0/557480261 wait complete. 2024-04-25T10:31:01.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.767+0000 7fb0c1346640 1 Processor -- start 2024-04-25T10:31:01.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.767+0000 7fb0c1346640 1 -- start start 2024-04-25T10:31:01.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.768+0000 7fb0c1346640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0bc0b4ce0 0x7fb0bc154880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:01.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.768+0000 7fb0c1346640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0bc0c7e30 0x7fb0bc1e9840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:01.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.768+0000 7fb0c1346640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb0bc156ef0 0x7fb0bc14f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:01.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.768+0000 7fb0c1346640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb0bc06d710 con 0x7fb0bc0c7e30 2024-04-25T10:31:01.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.768+0000 7fb0c1346640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb0bc0c72f0 con 0x7fb0bc0b4ce0 2024-04-25T10:31:01.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.768+0000 7fb0c1346640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb0bc06d890 con 0x7fb0bc156ef0 2024-04-25T10:31:01.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.768+0000 7fb0ba7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0bc0c7e30 0x7fb0bc1e9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:01.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.768+0000 7fb0ba7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0bc0c7e30 0x7fb0bc1e9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35200/0 (socket says 172.21.15.106:35200) 2024-04-25T10:31:01.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.768+0000 7fb0ba7fc640 1 -- 172.21.15.106:0/3547217154 learned_addr learned my addr 172.21.15.106:0/3547217154 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:31:01.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.768+0000 7fb0baffd640 1 --2- 172.21.15.106:0/3547217154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0bc0b4ce0 0x7fb0bc154880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:01.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.768+0000 7fb0bb7fe640 1 --2- 172.21.15.106:0/3547217154 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb0bc156ef0 0x7fb0bc14f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:01.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.768+0000 7fb0ba7fc640 1 -- 172.21.15.106:0/3547217154 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb0bc156ef0 msgr2=0x7fb0bc14f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:01.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.768+0000 7fb0ba7fc640 1 --2- 172.21.15.106:0/3547217154 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb0bc156ef0 0x7fb0bc14f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:01.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.768+0000 7fb0ba7fc640 1 -- 172.21.15.106:0/3547217154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0bc0b4ce0 msgr2=0x7fb0bc154880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:01.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.768+0000 7fb0ba7fc640 1 --2- 172.21.15.106:0/3547217154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0bc0b4ce0 0x7fb0bc154880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:01.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.768+0000 7fb0ba7fc640 1 -- 172.21.15.106:0/3547217154 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0bc154dc0 con 0x7fb0bc0c7e30 2024-04-25T10:31:01.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.769+0000 7fb0ba7fc640 1 --2- 172.21.15.106:0/3547217154 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0bc0c7e30 0x7fb0bc1e9840 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb0b000c400 tx=0x7fb0b000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:01.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.769+0000 7fb0a37fe640 1 -- 172.21.15.106:0/3547217154 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b0017070 con 0x7fb0bc0c7e30 2024-04-25T10:31:01.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.769+0000 7fb0a37fe640 1 -- 172.21.15.106:0/3547217154 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0b00074e0 con 0x7fb0bc0c7e30 2024-04-25T10:31:01.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.769+0000 7fb0c1346640 1 -- 172.21.15.106:0/3547217154 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0bc0d0270 con 0x7fb0bc0c7e30 2024-04-25T10:31:01.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.769+0000 7fb0a37fe640 1 -- 172.21.15.106:0/3547217154 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b00059e0 con 0x7fb0bc0c7e30 2024-04-25T10:31:01.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.769+0000 7fb0c1346640 1 -- 172.21.15.106:0/3547217154 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0bc0d0830 con 0x7fb0bc0c7e30 2024-04-25T10:31:01.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.770+0000 7fb0c1346640 1 -- 172.21.15.106:0/3547217154 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0bc0b7f40 con 0x7fb0bc0c7e30 2024-04-25T10:31:01.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.771+0000 7fb0a37fe640 1 -- 172.21.15.106:0/3547217154 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb0b001a070 con 0x7fb0bc0c7e30 2024-04-25T10:31:01.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.771+0000 7fb0a37fe640 1 --2- 172.21.15.106:0/3547217154 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fb0a80744e0 0x7fb0a80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:01.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.771+0000 7fb0baffd640 1 --2- 172.21.15.106:0/3547217154 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fb0a80744e0 0x7fb0a80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:01.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.771+0000 7fb0a37fe640 1 -- 172.21.15.106:0/3547217154 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb0b0094450 con 0x7fb0bc0c7e30 2024-04-25T10:31:01.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.771+0000 7fb0baffd640 1 --2- 172.21.15.106:0/3547217154 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fb0a80744e0 0x7fb0a80769f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac009aa0 tx=0x7fb0ac009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:01.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.776+0000 7fb0a37fe640 1 -- 172.21.15.106:0/3547217154 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb0bc0b7f40 con 0x7fb0bc0c7e30 2024-04-25T10:31:01.785 DEBUG:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:31:01.788 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:31:01.817 INFO:tasks.ceph.mds.d.smithi134.stderr:2024-04-25T10:31:01.816+0000 7f3ef08bf640 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-04-25T10:31:01.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.993+0000 7fb0c1346640 1 -- 172.21.15.106:0/3547217154 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb0bc0cffd0 con 0x7fb0bc0c7e30 2024-04-25T10:31:01.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.994+0000 7fb0a37fe640 1 -- 172.21.15.106:0/3547217154 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+6922 (secure 0 0 0) 0x7fb0b005e770 con 0x7fb0bc0c7e30 2024-04-25T10:31:01.995 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:31:01.995 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/2834155852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":2834155852},{"type":"v1","addr":"172.21.15.106:6835","nonce":2834155852}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:30:31.761107+0000","modified":"2024-04-25T10:31:00.589522+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4290,"mds_1":4287,"mds_2":4284,"mds_3":4269,"mds_4":4266},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4266":{"gid":4266,"name":"c","rank":4,"incarnation":16,"state":"up:active","state_seq":10,"addr":"172.21.15.106:6837/2632121774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":2632121774},{"type":"v1","addr":"172.21.15.106:6837","nonce":2632121774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4269":{"gid":4269,"name":"e","rank":3,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.106:6839/486339834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":486339834},{"type":"v1","addr":"172.21.15.106:6839","nonce":486339834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4284":{"gid":4284,"name":"b","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.134:6836/2781656864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6833","nonce":2781656864},{"type":"v1","addr":"172.21.15.134:6836","nonce":2781656864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4287":{"gid":4287,"name":"f","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.134:6835/1993918351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1993918351},{"type":"v1","addr":"172.21.15.134:6835","nonce":1993918351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4290":{"gid":4290,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.134:6837/2662003420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2662003420},{"type":"v1","addr":"172.21.15.134:6837","nonce":2662003420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4290,"qdb_cluster":[4266,4269,4284,4287,4290]},"id":1}]} 2024-04-25T10:31:01.996 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 17 2024-04-25T10:31:01.997 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.996+0000 7fb0c1346640 1 -- 172.21.15.106:0/3547217154 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fb0a80744e0 msgr2=0x7fb0a80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:01.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.996+0000 7fb0c1346640 1 --2- 172.21.15.106:0/3547217154 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fb0a80744e0 0x7fb0a80769f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac009aa0 tx=0x7fb0ac009190 comp rx=0 tx=0).stop 2024-04-25T10:31:01.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.996+0000 7fb0c1346640 1 -- 172.21.15.106:0/3547217154 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0bc0c7e30 msgr2=0x7fb0bc1e9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:01.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.996+0000 7fb0c1346640 1 --2- 172.21.15.106:0/3547217154 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0bc0c7e30 0x7fb0bc1e9840 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb0b000c400 tx=0x7fb0b000c8c0 comp rx=0 tx=0).stop 2024-04-25T10:31:01.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.996+0000 7fb0c1346640 1 -- 172.21.15.106:0/3547217154 shutdown_connections 2024-04-25T10:31:01.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.996+0000 7fb0c1346640 1 --2- 172.21.15.106:0/3547217154 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fb0a80744e0 0x7fb0a80769f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:01.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.996+0000 7fb0c1346640 1 --2- 172.21.15.106:0/3547217154 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb0bc156ef0 0x7fb0bc14f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:01.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.996+0000 7fb0c1346640 1 --2- 172.21.15.106:0/3547217154 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0bc0c7e30 0x7fb0bc1e9840 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:01.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.996+0000 7fb0c1346640 1 --2- 172.21.15.106:0/3547217154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0bc0b4ce0 0x7fb0bc154880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:01.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.996+0000 7fb0c1346640 1 -- 172.21.15.106:0/3547217154 >> 172.21.15.106:0/3547217154 conn(0x7fb0bc068b10 msgr2=0x7fb0bc06d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:01.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.996+0000 7fb0c1346640 1 -- 172.21.15.106:0/3547217154 shutdown_connections 2024-04-25T10:31:01.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:01.996+0000 7fb0c1346640 1 -- 172.21.15.106:0/3547217154 wait complete. 2024-04-25T10:31:02.612 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:30:31.761107+0000', 'modified': '2024-04-25T10:31:00.589522+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 4290, 'mds_1': 4287, 'mds_2': 4284, 'mds_3': 4269, 'mds_4': 4266}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4266': {'gid': 4266, 'name': 'c', 'rank': 4, 'incarnation': 16, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.106:6837/2632121774', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6836', 'nonce': 2632121774}, {'type': 'v1', 'addr': '172.21.15.106:6837', 'nonce': 2632121774}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4269': {'gid': 4269, 'name': 'e', 'rank': 3, 'incarnation': 14, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.106:6839/486339834', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6838', 'nonce': 486339834}, {'type': 'v1', 'addr': '172.21.15.106:6839', 'nonce': 486339834}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4284': {'gid': 4284, 'name': 'b', 'rank': 2, 'incarnation': 12, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.134:6836/2781656864', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6833', 'nonce': 2781656864}, {'type': 'v1', 'addr': '172.21.15.134:6836', 'nonce': 2781656864}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4287': {'gid': 4287, 'name': 'f', 'rank': 1, 'incarnation': 10, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.134:6835/1993918351', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6832', 'nonce': 1993918351}, {'type': 'v1', 'addr': '172.21.15.134:6835', 'nonce': 1993918351}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4290': {'gid': 4290, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.134:6837/2662003420', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6834', 'nonce': 2662003420}, {'type': 'v1', 'addr': '172.21.15.134:6837', 'nonce': 2662003420}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4290, 'qdb_cluster': [4266, 4269, 4284, 4287, 4290]} 2024-04-25T10:31:02.612 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-25T10:31:02.613 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:31:02.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.722+0000 7fda24692640 1 Processor -- start 2024-04-25T10:31:02.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.722+0000 7fda24692640 1 -- start start 2024-04-25T10:31:02.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.723+0000 7fda24692640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fda1c0cbff0 0x7fda1c0cb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:02.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.723+0000 7fda24692640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fda1c156ec0 0x7fda1c0c61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:02.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.723+0000 7fda24692640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda1c15dae0 con 0x7fda1c0c66e0 2024-04-25T10:31:02.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.723+0000 7fda24692640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fda1c161270 con 0x7fda1c0cbff0 2024-04-25T10:31:02.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.723+0000 7fda24692640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fda1c1604b0 con 0x7fda1c156ec0 2024-04-25T10:31:02.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.724+0000 7fda2242e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fda1c0cbff0 0x7fda1c0cb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:02.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.724+0000 7fda2242e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fda1c0cbff0 0x7fda1c0cb270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35212/0 (socket says 172.21.15.106:35212) 2024-04-25T10:31:02.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.724+0000 7fda2242e640 1 -- 172.21.15.106:0/3473669999 learned_addr learned my addr 172.21.15.106:0/3473669999 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:31:02.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.724+0000 7fda21c2d640 1 --2- 172.21.15.106:0/3473669999 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fda1c156ec0 0x7fda1c0c61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:02.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.724+0000 7fda2242e640 1 -- 172.21.15.106:0/3473669999 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fda1c156ec0 msgr2=0x7fda1c0c61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:02.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.724+0000 7fda2242e640 1 --2- 172.21.15.106:0/3473669999 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fda1c156ec0 0x7fda1c0c61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:02.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.724+0000 7fda2242e640 1 -- 172.21.15.106:0/3473669999 >> v1:172.21.15.134:6789/0 conn(0x7fda1c0c66e0 legacy=0x7fda1c0cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:02.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.724+0000 7fda2242e640 1 -- 172.21.15.106:0/3473669999 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda1c160790 con 0x7fda1c0cbff0 2024-04-25T10:31:02.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.724+0000 7fda2142c640 1 -- 172.21.15.106:0/3473669999 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1010287736 0 0) 0x7fda1c15dae0 con 0x7fda1c0c66e0 2024-04-25T10:31:02.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.725+0000 7fda2242e640 1 --2- 172.21.15.106:0/3473669999 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda1c0cbff0 0x7fda1c0cb270 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fda18009e50 tx=0x7fda1802a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e6110d8f3b057ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:02.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.725+0000 7fda2142c640 1 -- 172.21.15.106:0/3473669999 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda18036070 con 0x7fda1c0cbff0 2024-04-25T10:31:02.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.725+0000 7fda2142c640 1 -- 172.21.15.106:0/3473669999 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda18031600 con 0x7fda1c0cbff0 2024-04-25T10:31:02.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.725+0000 7fda2142c640 1 -- 172.21.15.106:0/3473669999 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda180319d0 con 0x7fda1c0cbff0 2024-04-25T10:31:02.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.725+0000 7fda24692640 1 -- 172.21.15.106:0/3473669999 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda1c0cbff0 msgr2=0x7fda1c0cb270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:02.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.725+0000 7fda24692640 1 --2- 172.21.15.106:0/3473669999 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda1c0cbff0 0x7fda1c0cb270 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fda18009e50 tx=0x7fda1802a7a0 comp rx=0 tx=0).stop 2024-04-25T10:31:02.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.725+0000 7fda24692640 1 -- 172.21.15.106:0/3473669999 shutdown_connections 2024-04-25T10:31:02.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.725+0000 7fda24692640 1 --2- 172.21.15.106:0/3473669999 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fda1c156ec0 0x7fda1c0c61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:02.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.725+0000 7fda24692640 1 --2- 172.21.15.106:0/3473669999 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda1c0cbff0 0x7fda1c0cb270 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:02.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.725+0000 7fda24692640 1 -- 172.21.15.106:0/3473669999 >> 172.21.15.106:0/3473669999 conn(0x7fda1c0727d0 msgr2=0x7fda1c072c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:02.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.725+0000 7fda24692640 1 -- 172.21.15.106:0/3473669999 shutdown_connections 2024-04-25T10:31:02.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.725+0000 7fda24692640 1 -- 172.21.15.106:0/3473669999 wait complete. 2024-04-25T10:31:02.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.727+0000 7fda24692640 1 Processor -- start 2024-04-25T10:31:02.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.727+0000 7fda24692640 1 -- start start 2024-04-25T10:31:02.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.728+0000 7fda24692640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fda1c0c66e0 0x7fda1c1592d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:02.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.728+0000 7fda24692640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda1c156ec0 0x7fda1c14ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:02.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.728+0000 7fda24692640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda1c0d03b0 0x7fda1c157980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:02.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.728+0000 7fda24692640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fda1c1572d0 con 0x7fda1c156ec0 2024-04-25T10:31:02.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.728+0000 7fda24692640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fda1c160d50 con 0x7fda1c0d03b0 2024-04-25T10:31:02.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.728+0000 7fda24692640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fda1c15d730 con 0x7fda1c0c66e0 2024-04-25T10:31:02.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.729+0000 7fda21c2d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda1c156ec0 0x7fda1c14ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:02.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.729+0000 7fda2242e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fda1c0c66e0 0x7fda1c1592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:02.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.729+0000 7fda21c2d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda1c156ec0 0x7fda1c14ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35222/0 (socket says 172.21.15.106:35222) 2024-04-25T10:31:02.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.729+0000 7fda21c2d640 1 -- 172.21.15.106:0/1083144575 learned_addr learned my addr 172.21.15.106:0/1083144575 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:31:02.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.729+0000 7fda22c2f640 1 --2- 172.21.15.106:0/1083144575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda1c0d03b0 0x7fda1c157980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:02.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.729+0000 7fda21c2d640 1 -- 172.21.15.106:0/1083144575 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fda1c0c66e0 msgr2=0x7fda1c1592d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:02.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.729+0000 7fda21c2d640 1 --2- 172.21.15.106:0/1083144575 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fda1c0c66e0 0x7fda1c1592d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:02.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.729+0000 7fda21c2d640 1 -- 172.21.15.106:0/1083144575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda1c0d03b0 msgr2=0x7fda1c157980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:02.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.729+0000 7fda21c2d640 1 --2- 172.21.15.106:0/1083144575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda1c0d03b0 0x7fda1c157980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:02.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.729+0000 7fda21c2d640 1 -- 172.21.15.106:0/1083144575 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda1c159810 con 0x7fda1c156ec0 2024-04-25T10:31:02.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.729+0000 7fda21c2d640 1 --2- 172.21.15.106:0/1083144575 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda1c156ec0 0x7fda1c14ec20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fda0c00e8a0 tx=0x7fda0c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:02.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.729+0000 7fda12ffd640 1 -- 172.21.15.106:0/1083144575 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda0c017070 con 0x7fda1c156ec0 2024-04-25T10:31:02.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.729+0000 7fda12ffd640 1 -- 172.21.15.106:0/1083144575 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda0c00c940 con 0x7fda1c156ec0 2024-04-25T10:31:02.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.729+0000 7fda24692640 1 -- 172.21.15.106:0/1083144575 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda1c160130 con 0x7fda1c156ec0 2024-04-25T10:31:02.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.729+0000 7fda12ffd640 1 -- 172.21.15.106:0/1083144575 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda0c00cd30 con 0x7fda1c156ec0 2024-04-25T10:31:02.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.729+0000 7fda24692640 1 -- 172.21.15.106:0/1083144575 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda1c15a440 con 0x7fda1c156ec0 2024-04-25T10:31:02.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.731+0000 7fda12ffd640 1 -- 172.21.15.106:0/1083144575 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fda0c01a070 con 0x7fda1c156ec0 2024-04-25T10:31:02.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.731+0000 7fda12ffd640 1 --2- 172.21.15.106:0/1083144575 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fda000744b0 0x7fda000769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:02.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.731+0000 7fda12ffd640 1 -- 172.21.15.106:0/1083144575 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fda0c094ff0 con 0x7fda1c156ec0 2024-04-25T10:31:02.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.731+0000 7fda2242e640 1 --2- 172.21.15.106:0/1083144575 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fda000744b0 0x7fda000769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:02.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.732+0000 7fda2242e640 1 --2- 172.21.15.106:0/1083144575 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fda000744b0 0x7fda000769c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fda18033000 tx=0x7fda18009000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:02.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.733+0000 7fda2b617740 1 Processor -- start 2024-04-25T10:31:02.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.733+0000 7fda2b617740 1 -- start start 2024-04-25T10:31:02.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.734+0000 7fda2b617740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b7dbe4e270 0x55b7dc1f05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:02.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.734+0000 7fda2b617740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b7dbe253b0 0x55b7dc1ea880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:02.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.734+0000 7fda2b617740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b7dbe2d590 0x55b7dc1eadc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:02.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.734+0000 7fda2b617740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55b7dc154740 con 0x55b7dbe253b0 2024-04-25T10:31:02.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.734+0000 7fda2b617740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b7dc0f03a0 con 0x55b7dbe4e270 2024-04-25T10:31:02.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.734+0000 7fda2b617740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55b7dc150700 con 0x55b7dbe2d590 2024-04-25T10:31:02.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.734+0000 7fda21c2d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b7dbe253b0 0x55b7dc1ea880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:02.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.734+0000 7fda21c2d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b7dbe253b0 0x55b7dc1ea880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35238/0 (socket says 172.21.15.106:35238) 2024-04-25T10:31:02.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.734+0000 7fda21c2d640 1 -- 172.21.15.106:0/1971720239 learned_addr learned my addr 172.21.15.106:0/1971720239 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:31:02.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.734+0000 7fda22c2f640 1 --2- 172.21.15.106:0/1971720239 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b7dbe2d590 0x55b7dc1eadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:02.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.734+0000 7fda2242e640 1 --2- 172.21.15.106:0/1971720239 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b7dbe4e270 0x55b7dc1f05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:02.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.734+0000 7fda21c2d640 1 -- 172.21.15.106:0/1971720239 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b7dbe2d590 msgr2=0x55b7dc1eadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:02.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.734+0000 7fda21c2d640 1 --2- 172.21.15.106:0/1971720239 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b7dbe2d590 0x55b7dc1eadc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:02.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.734+0000 7fda21c2d640 1 -- 172.21.15.106:0/1971720239 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b7dbe4e270 msgr2=0x55b7dc1f05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:02.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.734+0000 7fda21c2d640 1 --2- 172.21.15.106:0/1971720239 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b7dbe4e270 0x55b7dc1f05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:02.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.734+0000 7fda21c2d640 1 -- 172.21.15.106:0/1971720239 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7dc10c750 con 0x55b7dbe253b0 2024-04-25T10:31:02.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.734+0000 7fda21c2d640 1 --2- 172.21.15.106:0/1971720239 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b7dbe253b0 0x55b7dc1ea880 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fda0c093540 tx=0x7fda0c05f090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad31f477208c4e3e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:02.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.735+0000 7fd9fffff640 1 -- 172.21.15.106:0/1971720239 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda0c021040 con 0x55b7dbe253b0 2024-04-25T10:31:02.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.735+0000 7fd9fffff640 1 -- 172.21.15.106:0/1971720239 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda0c05f1c0 con 0x55b7dbe253b0 2024-04-25T10:31:02.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.735+0000 7fd9fffff640 1 -- 172.21.15.106:0/1971720239 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda0c063230 con 0x55b7dbe253b0 2024-04-25T10:31:02.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.735+0000 7fda2b617740 1 -- 172.21.15.106:0/1971720239 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b7dbe253b0 msgr2=0x55b7dc1ea880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:02.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.735+0000 7fda2b617740 1 --2- 172.21.15.106:0/1971720239 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b7dbe253b0 0x55b7dc1ea880 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fda0c093540 tx=0x7fda0c05f090 comp rx=0 tx=0).stop 2024-04-25T10:31:02.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.735+0000 7fda2b617740 1 -- 172.21.15.106:0/1971720239 shutdown_connections 2024-04-25T10:31:02.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.735+0000 7fda2b617740 1 --2- 172.21.15.106:0/1971720239 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b7dbe2d590 0x55b7dc1eadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:02.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.735+0000 7fda2b617740 1 --2- 172.21.15.106:0/1971720239 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b7dbe253b0 0x55b7dc1ea880 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:02.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.735+0000 7fda2b617740 1 --2- 172.21.15.106:0/1971720239 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b7dbe4e270 0x55b7dc1f05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:02.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.735+0000 7fda2b617740 1 -- 172.21.15.106:0/1971720239 >> 172.21.15.106:0/1971720239 conn(0x55b7dbe423c0 msgr2=0x55b7dc1eed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:02.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.736+0000 7fda2b617740 1 -- 172.21.15.106:0/1971720239 shutdown_connections 2024-04-25T10:31:02.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.736+0000 7fda2b617740 1 -- 172.21.15.106:0/1971720239 wait complete. 2024-04-25T10:31:02.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.736+0000 7fda2b617740 1 -- start start 2024-04-25T10:31:02.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.736+0000 7fda2b617740 1 Processor -- start 2024-04-25T10:31:02.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.738+0000 7fda2b617740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b7dbe253b0 0x55b7dc2302d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:02.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.738+0000 7fda2b617740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b7dbe2d590 0x55b7dc217860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:02.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.738+0000 7fda2b617740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b7dbe4e270 0x55b7dc215380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:02.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.738+0000 7fda2b617740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55b7dc22ed00 con 0x55b7dbe4e270 2024-04-25T10:31:02.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.738+0000 7fda2b617740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b7dc0dbfe0 con 0x55b7dbe2d590 2024-04-25T10:31:02.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.738+0000 7fda2b617740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55b7dc1eebe0 con 0x55b7dbe253b0 2024-04-25T10:31:02.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.738+0000 7fda2b617740 10 client.0 _ll_register_callbacks cb 0x55b7dc0f2150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:31:02.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.738+0000 7fda22c2f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b7dbe4e270 0x55b7dc215380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:02.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.738+0000 7fda22c2f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b7dbe4e270 0x55b7dc215380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35242/0 (socket says 172.21.15.106:35242) 2024-04-25T10:31:02.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.738+0000 7fda22c2f640 1 -- 172.21.15.106:0/1041719874 learned_addr learned my addr 172.21.15.106:0/1041719874 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:31:02.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.738+0000 7fda2242e640 1 --2- 172.21.15.106:0/1041719874 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b7dbe253b0 0x55b7dc2302d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:02.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.738+0000 7fda21c2d640 1 --2- 172.21.15.106:0/1041719874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b7dbe2d590 0x55b7dc217860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:02.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.738+0000 7fda22c2f640 1 -- 172.21.15.106:0/1041719874 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b7dbe253b0 msgr2=0x55b7dc2302d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:02.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.738+0000 7fda22c2f640 1 --2- 172.21.15.106:0/1041719874 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b7dbe253b0 0x55b7dc2302d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:02.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.738+0000 7fda22c2f640 1 -- 172.21.15.106:0/1041719874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b7dbe2d590 msgr2=0x55b7dc217860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:02.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.738+0000 7fda22c2f640 1 --2- 172.21.15.106:0/1041719874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b7dbe2d590 0x55b7dc217860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:02.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.738+0000 7fda22c2f640 1 -- 172.21.15.106:0/1041719874 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7dbe662d0 con 0x55b7dbe4e270 2024-04-25T10:31:02.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.738+0000 7fda22c2f640 1 --2- 172.21.15.106:0/1041719874 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b7dbe4e270 0x55b7dc215380 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fda1c072070 tx=0x7fda14012750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f05d959ddea0e25d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:02.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.738+0000 7fd9fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T10:31:02.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.739+0000 7fd9fd7fa640 1 -- 172.21.15.106:0/1041719874 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda14005320 con 0x55b7dbe4e270 2024-04-25T10:31:02.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.739+0000 7fd9fd7fa640 1 -- 172.21.15.106:0/1041719874 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fda14007050 con 0x55b7dbe4e270 2024-04-25T10:31:02.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.739+0000 7fda2b617740 1 -- 172.21.15.106:0/1041719874 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda0c05f1c0 con 0x55b7dbe4e270 2024-04-25T10:31:02.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.739+0000 7fd9fd7fa640 1 -- 172.21.15.106:0/1041719874 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fda14007290 con 0x55b7dbe4e270 2024-04-25T10:31:02.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.739+0000 7fda2b617740 10 client.4701 fetch_fsmap learned FSMap version 17 2024-04-25T10:31:02.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.739+0000 7fda2b617740 1 -- 172.21.15.106:0/1041719874 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55b7dbf306b0 con 0x55b7dbe4e270 2024-04-25T10:31:02.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.739+0000 7fd9fd7fa640 1 -- 172.21.15.106:0/1041719874 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda0c05f1c0 con 0x55b7dbe4e270 2024-04-25T10:31:02.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.739+0000 7fd9fd7fa640 1 -- 172.21.15.106:0/1041719874 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7fda14007430 con 0x55b7dbe4e270 2024-04-25T10:31:02.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.739+0000 7fda2b617740 10 client.4701 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T10:31:02.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.739+0000 7fda2b617740 10 client.4701 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:31:02.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.739+0000 7fda2b617740 20 client.4701 populate_metadata read hostname 'smithi106' 2024-04-25T10:31:02.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.739+0000 7fda2b617740 1 --2- 172.21.15.106:0/1041719874 >> [v2:172.21.15.106:6836/2632121774,v1:172.21.15.106:6837/2632121774] conn(0x55b7dbe0ce40 0x55b7dc215da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:02.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.739+0000 7fda2b617740 4 client.4701 mds_command: new command op to 4266 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:31:02.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.739+0000 7fda2b617740 1 -- 172.21.15.106:0/1041719874 --> [v2:172.21.15.106:6836/2632121774,v1:172.21.15.106:6837/2632121774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b7dc162350 con 0x55b7dbe0ce40 2024-04-25T10:31:02.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.739+0000 7fda2242e640 1 --2- 172.21.15.106:0/1041719874 >> [v2:172.21.15.106:6836/2632121774,v1:172.21.15.106:6837/2632121774] conn(0x55b7dbe0ce40 0x55b7dc215da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:02.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.740+0000 7fd9fd7fa640 10 client.4701 ms_handle_connect on v2:172.21.15.106:6836/2632121774 2024-04-25T10:31:02.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:02.740+0000 7fda2242e640 1 --2- 172.21.15.106:0/1041719874 >> [v2:172.21.15.106:6836/2632121774,v1:172.21.15.106:6837/2632121774] conn(0x55b7dbe0ce40 0x55b7dc215da0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6c400c9490f557b5 server_cookie=60de285baee0288e in_seq=0 out_seq=0 2024-04-25T10:31:05.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.596+0000 7fd9fd7fa640 1 -- 172.21.15.106:0/1041719874 <== mds.4 v2:172.21.15.106:6836/2632121774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b7dc162350 con 0x55b7dbe0ce40 2024-04-25T10:31:05.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.596+0000 7fd9fd7fa640 10 client.4701 handle_command_reply: tid=0 2024-04-25T10:31:05.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.596+0000 7fda2b617740 1 client.4701 shutdown 2024-04-25T10:31:05.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.596+0000 7fda2b617740 1 -- 172.21.15.106:0/1041719874 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b7dbe4e270 msgr2=0x55b7dc215380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:05.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.596+0000 7fda2b617740 1 --2- 172.21.15.106:0/1041719874 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b7dbe4e270 0x55b7dc215380 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fda1c072070 tx=0x7fda14012750 comp rx=0 tx=0).stop 2024-04-25T10:31:05.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.597+0000 7fda2b617740 1 -- 172.21.15.106:0/1041719874 shutdown_connections 2024-04-25T10:31:05.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.597+0000 7fda2b617740 1 --2- 172.21.15.106:0/1041719874 >> [v2:172.21.15.106:6836/2632121774,v1:172.21.15.106:6837/2632121774] conn(0x55b7dbe0ce40 0x55b7dc215da0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.597+0000 7fda2b617740 1 --2- 172.21.15.106:0/1041719874 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b7dbe4e270 0x55b7dc215380 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.597+0000 7fda2b617740 1 --2- 172.21.15.106:0/1041719874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b7dbe2d590 0x55b7dc217860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.597+0000 7fda2b617740 1 --2- 172.21.15.106:0/1041719874 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b7dbe253b0 0x55b7dc2302d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.597+0000 7fd9fd7fa640 0 client.4701 ms_handle_reset on v2:172.21.15.106:6836/2632121774 2024-04-25T10:31:05.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.597+0000 7fda2b617740 1 -- 172.21.15.106:0/1041719874 >> 172.21.15.106:0/1041719874 conn(0x55b7dbe423c0 msgr2=0x55b7dc1e9b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:05.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.597+0000 7fda2b617740 1 -- 172.21.15.106:0/1041719874 shutdown_connections 2024-04-25T10:31:05.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.597+0000 7fda2b617740 1 -- 172.21.15.106:0/1041719874 wait complete. 2024-04-25T10:31:05.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.597+0000 7fda2b617740 20 client.4701 trim_cache size 0 max 16384 2024-04-25T10:31:05.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.617+0000 7fda2b617740 1 Processor -- start 2024-04-25T10:31:05.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.617+0000 7fda2b617740 1 -- start start 2024-04-25T10:31:05.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.619+0000 7fda2b617740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b7dbe253b0 0x55b7dc254560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:05.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.619+0000 7fda2b617740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b7dbe2d590 0x55b7dc1efff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:05.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.619+0000 7fda2b617740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b7dbe4e270 0x55b7dc1f0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:05.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.619+0000 7fda2b617740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fda1c1ff550 con 0x55b7dbe4e270 2024-04-25T10:31:05.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.619+0000 7fda2b617740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fda1c06f220 con 0x55b7dbe2d590 2024-04-25T10:31:05.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.619+0000 7fda2b617740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55b7dc227370 con 0x55b7dbe253b0 2024-04-25T10:31:05.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.619+0000 7fda22c2f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b7dbe4e270 0x55b7dc1f0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:05.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.619+0000 7fda22c2f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b7dbe4e270 0x55b7dc1f0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58866/0 (socket says 172.21.15.106:58866) 2024-04-25T10:31:05.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.619+0000 7fda22c2f640 1 -- 172.21.15.106:0/1219541784 learned_addr learned my addr 172.21.15.106:0/1219541784 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:31:05.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.619+0000 7fda2242e640 1 --2- 172.21.15.106:0/1219541784 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b7dbe253b0 0x55b7dc254560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:05.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.619+0000 7fda21c2d640 1 --2- 172.21.15.106:0/1219541784 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b7dbe2d590 0x55b7dc1efff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:05.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.619+0000 7fda22c2f640 1 -- 172.21.15.106:0/1219541784 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b7dbe253b0 msgr2=0x55b7dc254560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:05.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.619+0000 7fda22c2f640 1 --2- 172.21.15.106:0/1219541784 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b7dbe253b0 0x55b7dc254560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.619+0000 7fda22c2f640 1 -- 172.21.15.106:0/1219541784 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b7dbe2d590 msgr2=0x55b7dc1efff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:05.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.619+0000 7fda22c2f640 1 --2- 172.21.15.106:0/1219541784 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b7dbe2d590 0x55b7dc1efff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.619+0000 7fda22c2f640 1 -- 172.21.15.106:0/1219541784 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7dc2179e0 con 0x55b7dbe4e270 2024-04-25T10:31:05.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.619+0000 7fda22c2f640 1 --2- 172.21.15.106:0/1219541784 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b7dbe4e270 0x55b7dc1f0530 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fda14001720 tx=0x7fda1400f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13007ab13ceca0f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:05.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.619+0000 7fd9ff7fe640 1 -- 172.21.15.106:0/1219541784 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda1402e070 con 0x55b7dbe4e270 2024-04-25T10:31:05.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.619+0000 7fd9ff7fe640 1 -- 172.21.15.106:0/1219541784 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda140068f0 con 0x55b7dbe4e270 2024-04-25T10:31:05.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.619+0000 7fd9ff7fe640 1 -- 172.21.15.106:0/1219541784 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda14006ce0 con 0x55b7dbe4e270 2024-04-25T10:31:05.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.620+0000 7fda2b617740 1 -- 172.21.15.106:0/1219541784 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b7dbe4e270 msgr2=0x55b7dc1f0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:05.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.620+0000 7fda2b617740 1 --2- 172.21.15.106:0/1219541784 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b7dbe4e270 0x55b7dc1f0530 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fda14001720 tx=0x7fda1400f000 comp rx=0 tx=0).stop 2024-04-25T10:31:05.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.620+0000 7fda2b617740 1 -- 172.21.15.106:0/1219541784 shutdown_connections 2024-04-25T10:31:05.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.620+0000 7fda2b617740 1 --2- 172.21.15.106:0/1219541784 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b7dbe4e270 0x55b7dc1f0530 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.620+0000 7fda2b617740 1 --2- 172.21.15.106:0/1219541784 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b7dbe2d590 0x55b7dc1efff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.620+0000 7fda2b617740 1 --2- 172.21.15.106:0/1219541784 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b7dbe253b0 0x55b7dc254560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.620+0000 7fda2b617740 1 -- 172.21.15.106:0/1219541784 >> 172.21.15.106:0/1219541784 conn(0x55b7dbe423c0 msgr2=0x55b7dc2300e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:05.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.620+0000 7fda2b617740 1 -- 172.21.15.106:0/1219541784 shutdown_connections 2024-04-25T10:31:05.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.620+0000 7fda2b617740 1 -- 172.21.15.106:0/1219541784 wait complete. 2024-04-25T10:31:05.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.621+0000 7fda2b617740 1 -- start start 2024-04-25T10:31:05.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.621+0000 7fda2b617740 1 Processor -- start 2024-04-25T10:31:05.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.623+0000 7fda2b617740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b7dbe253b0 0x55b7dc230370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:05.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.623+0000 7fda2b617740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b7dbe2d590 0x55b7dc2408e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:05.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.623+0000 7fda2b617740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b7dbe4e270 0x55b7dc2ae310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:05.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.623+0000 7fda2b617740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55b7dc215c20 con 0x55b7dbe4e270 2024-04-25T10:31:05.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.623+0000 7fda2b617740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b7dc20e7b0 con 0x55b7dbe253b0 2024-04-25T10:31:05.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.623+0000 7fda2b617740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55b7dc243b40 con 0x55b7dbe2d590 2024-04-25T10:31:05.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.623+0000 7fda2b617740 10 client.0 _ll_register_callbacks cb 0x7fda14001270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:31:05.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.623+0000 7fda22c2f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b7dbe4e270 0x55b7dc2ae310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:05.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.623+0000 7fda22c2f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b7dbe4e270 0x55b7dc2ae310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58874/0 (socket says 172.21.15.106:58874) 2024-04-25T10:31:05.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.623+0000 7fda22c2f640 1 -- 172.21.15.106:0/1087812585 learned_addr learned my addr 172.21.15.106:0/1087812585 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:31:05.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.623+0000 7fda2242e640 1 --2- 172.21.15.106:0/1087812585 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b7dbe253b0 0x55b7dc230370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:05.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.623+0000 7fda21c2d640 1 --2- 172.21.15.106:0/1087812585 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b7dbe2d590 0x55b7dc2408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:05.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.623+0000 7fda22c2f640 1 -- 172.21.15.106:0/1087812585 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b7dbe2d590 msgr2=0x55b7dc2408e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:05.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.623+0000 7fda22c2f640 1 --2- 172.21.15.106:0/1087812585 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b7dbe2d590 0x55b7dc2408e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.623+0000 7fda22c2f640 1 -- 172.21.15.106:0/1087812585 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b7dbe253b0 msgr2=0x55b7dc230370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:05.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.623+0000 7fda22c2f640 1 --2- 172.21.15.106:0/1087812585 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b7dbe253b0 0x55b7dc230370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.623+0000 7fda22c2f640 1 -- 172.21.15.106:0/1087812585 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7dc23a940 con 0x55b7dbe4e270 2024-04-25T10:31:05.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.623+0000 7fda22c2f640 1 --2- 172.21.15.106:0/1087812585 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b7dbe4e270 0x55b7dc2ae310 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fda140017a0 tx=0x7fda14000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d308e6648e7fcb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:05.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.623+0000 7fd9fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T10:31:05.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.624+0000 7fd9fd7fa640 1 -- 172.21.15.106:0/1087812585 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda1402e040 con 0x55b7dbe4e270 2024-04-25T10:31:05.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.624+0000 7fd9fd7fa640 1 -- 172.21.15.106:0/1087812585 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fda14007840 con 0x55b7dbe4e270 2024-04-25T10:31:05.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.624+0000 7fda2b617740 1 -- 172.21.15.106:0/1087812585 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda140068f0 con 0x55b7dbe4e270 2024-04-25T10:31:05.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.624+0000 7fd9fd7fa640 1 -- 172.21.15.106:0/1087812585 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fda14040050 con 0x55b7dbe4e270 2024-04-25T10:31:05.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.624+0000 7fda2b617740 10 client.4707 fetch_fsmap learned FSMap version 17 2024-04-25T10:31:05.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.624+0000 7fda2b617740 1 -- 172.21.15.106:0/1087812585 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55b7dc23abc0 con 0x55b7dbe4e270 2024-04-25T10:31:05.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.624+0000 7fd9fd7fa640 1 -- 172.21.15.106:0/1087812585 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda140068f0 con 0x55b7dbe4e270 2024-04-25T10:31:05.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.624+0000 7fd9fd7fa640 1 -- 172.21.15.106:0/1087812585 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7fda14043020 con 0x55b7dbe4e270 2024-04-25T10:31:05.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.624+0000 7fda2b617740 10 client.4707 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T10:31:05.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.624+0000 7fda2b617740 10 client.4707 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:31:05.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.624+0000 7fda2b617740 20 client.4707 populate_metadata read hostname 'smithi106' 2024-04-25T10:31:05.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.624+0000 7fda2b617740 1 --2- 172.21.15.106:0/1087812585 >> [v2:172.21.15.106:6836/2632121774,v1:172.21.15.106:6837/2632121774] conn(0x55b7dc20a4f0 0x55b7dc2bb710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:05.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.624+0000 7fda2b617740 4 client.4707 mds_command: new command op to 4266 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:31:05.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.624+0000 7fda2b617740 1 -- 172.21.15.106:0/1087812585 --> [v2:172.21.15.106:6836/2632121774,v1:172.21.15.106:6837/2632121774] -- command(tid 0: {"prefix": "status"}) -- 0x55b7dc15c1c0 con 0x55b7dc20a4f0 2024-04-25T10:31:05.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.624+0000 7fda2242e640 1 --2- 172.21.15.106:0/1087812585 >> [v2:172.21.15.106:6836/2632121774,v1:172.21.15.106:6837/2632121774] conn(0x55b7dc20a4f0 0x55b7dc2bb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:05.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.625+0000 7fd9fd7fa640 10 client.4707 ms_handle_connect on v2:172.21.15.106:6836/2632121774 2024-04-25T10:31:05.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.625+0000 7fda2242e640 1 --2- 172.21.15.106:0/1087812585 >> [v2:172.21.15.106:6836/2632121774,v1:172.21.15.106:6837/2632121774] conn(0x55b7dc20a4f0 0x55b7dc2bb710 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cdf0b684b4fdf388 server_cookie=c3d2fe5b132970a9 in_seq=0 out_seq=0 2024-04-25T10:31:05.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.625+0000 7fd9fd7fa640 1 -- 172.21.15.106:0/1087812585 <== mds.4 v2:172.21.15.106:6836/2632121774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55b7dc15c1c0 con 0x55b7dc20a4f0 2024-04-25T10:31:05.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.625+0000 7fd9fd7fa640 10 client.4707 handle_command_reply: tid=0 2024-04-25T10:31:05.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.625+0000 7fda2b617740 1 client.4707 shutdown 2024-04-25T10:31:05.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.625+0000 7fda2b617740 1 -- 172.21.15.106:0/1087812585 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b7dbe4e270 msgr2=0x55b7dc2ae310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:05.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.625+0000 7fda2b617740 1 --2- 172.21.15.106:0/1087812585 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b7dbe4e270 0x55b7dc2ae310 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fda140017a0 tx=0x7fda14000ed0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.625+0000 7fda2b617740 1 -- 172.21.15.106:0/1087812585 shutdown_connections 2024-04-25T10:31:05.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.625+0000 7fda2b617740 1 --2- 172.21.15.106:0/1087812585 >> [v2:172.21.15.106:6836/2632121774,v1:172.21.15.106:6837/2632121774] conn(0x55b7dc20a4f0 0x55b7dc2bb710 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.625+0000 7fda2b617740 1 --2- 172.21.15.106:0/1087812585 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b7dbe4e270 0x55b7dc2ae310 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.625+0000 7fda2b617740 1 --2- 172.21.15.106:0/1087812585 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b7dbe2d590 0x55b7dc2408e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.625+0000 7fda2b617740 1 --2- 172.21.15.106:0/1087812585 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b7dbe253b0 0x55b7dc230370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.625+0000 7fd9fd7fa640 0 client.4707 ms_handle_reset on v2:172.21.15.106:6836/2632121774 2024-04-25T10:31:05.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.625+0000 7fda2b617740 1 -- 172.21.15.106:0/1087812585 >> 172.21.15.106:0/1087812585 conn(0x55b7dbe423c0 msgr2=0x55b7dc1efff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:05.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.625+0000 7fda2b617740 1 -- 172.21.15.106:0/1087812585 shutdown_connections 2024-04-25T10:31:05.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.626+0000 7fda2b617740 1 -- 172.21.15.106:0/1087812585 wait complete. 2024-04-25T10:31:05.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.626+0000 7fda2b617740 20 client.4707 trim_cache size 0 max 16384 2024-04-25T10:31:05.634 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-25T10:31:05.634 INFO:teuthology.orchestra.run.smithi106.stdout: "cluster_fsid": "3ee77985-e8e4-41cf-81e9-7c774cf7891f", 2024-04-25T10:31:05.634 INFO:teuthology.orchestra.run.smithi106.stdout: "whoami": 4, 2024-04-25T10:31:05.634 INFO:teuthology.orchestra.run.smithi106.stdout: "id": 4266, 2024-04-25T10:31:05.634 INFO:teuthology.orchestra.run.smithi106.stdout: "want_state": "up:active", 2024-04-25T10:31:05.634 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "up:active", 2024-04-25T10:31:05.634 INFO:teuthology.orchestra.run.smithi106.stdout: "fs_name": "cephfs", 2024-04-25T10:31:05.634 INFO:teuthology.orchestra.run.smithi106.stdout: "rank_uptime": 11.048014581, 2024-04-25T10:31:05.634 INFO:teuthology.orchestra.run.smithi106.stdout: "mdsmap_epoch": 17, 2024-04-25T10:31:05.635 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch": 20, 2024-04-25T10:31:05.635 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch_barrier": 20, 2024-04-25T10:31:05.635 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 39.827052471999998 2024-04-25T10:31:05.635 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-25T10:31:05.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.626+0000 7fda24692640 1 -- 172.21.15.106:0/1083144575 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fda000744b0 msgr2=0x7fda000769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:05.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.626+0000 7fda24692640 1 --2- 172.21.15.106:0/1083144575 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fda000744b0 0x7fda000769c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fda18033000 tx=0x7fda18009000 comp rx=0 tx=0).stop 2024-04-25T10:31:05.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.626+0000 7fda24692640 1 -- 172.21.15.106:0/1083144575 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda1c156ec0 msgr2=0x7fda1c14ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:05.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.626+0000 7fda24692640 1 --2- 172.21.15.106:0/1083144575 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda1c156ec0 0x7fda1c14ec20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fda0c00e8a0 tx=0x7fda0c00ed60 comp rx=0 tx=0).stop 2024-04-25T10:31:05.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.626+0000 7fda24692640 1 -- 172.21.15.106:0/1083144575 shutdown_connections 2024-04-25T10:31:05.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.626+0000 7fda24692640 1 --2- 172.21.15.106:0/1083144575 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fda000744b0 0x7fda000769c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.626+0000 7fda24692640 1 --2- 172.21.15.106:0/1083144575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda1c0d03b0 0x7fda1c157980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.626+0000 7fda24692640 1 --2- 172.21.15.106:0/1083144575 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda1c156ec0 0x7fda1c14ec20 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.626+0000 7fda24692640 1 --2- 172.21.15.106:0/1083144575 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fda1c0c66e0 0x7fda1c1592d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.626+0000 7fda24692640 1 -- 172.21.15.106:0/1083144575 >> 172.21.15.106:0/1083144575 conn(0x7fda1c0727d0 msgr2=0x7fda1c0ccca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:05.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.626+0000 7fda24692640 1 -- 172.21.15.106:0/1083144575 shutdown_connections 2024-04-25T10:31:05.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.626+0000 7fda24692640 1 -- 172.21.15.106:0/1083144575 wait complete. 2024-04-25T10:31:05.640 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-04-25T10:31:05.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.750+0000 7fbd4355f640 1 Processor -- start 2024-04-25T10:31:05.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.751+0000 7fbd4355f640 1 -- start start 2024-04-25T10:31:05.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.752+0000 7fbd4355f640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fbd3c0cc600 0x7fbd3c0c5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:05.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.752+0000 7fbd4355f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbd3c0c58d0 0x7fbd3c0cce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:05.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.752+0000 7fbd4355f640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd3c15b8c0 con 0x7fbd3c0cb2b0 2024-04-25T10:31:05.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.752+0000 7fbd4355f640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fbd3c15eea0 con 0x7fbd3c0cc600 2024-04-25T10:31:05.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.752+0000 7fbd4355f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fbd3c15b5e0 con 0x7fbd3c0c58d0 2024-04-25T10:31:05.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.752+0000 7fbd412fb640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fbd3c0cc600 0x7fbd3c0c5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:05.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.752+0000 7fbd412fb640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fbd3c0cc600 0x7fbd3c0c5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58886/0 (socket says 172.21.15.106:58886) 2024-04-25T10:31:05.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.752+0000 7fbd412fb640 1 -- 172.21.15.106:0/3385112359 learned_addr learned my addr 172.21.15.106:0/3385112359 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:31:05.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.752+0000 7fbd40afa640 1 --2- 172.21.15.106:0/3385112359 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbd3c0c58d0 0x7fbd3c0cce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:05.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.752+0000 7fbd412fb640 1 -- 172.21.15.106:0/3385112359 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbd3c0c58d0 msgr2=0x7fbd3c0cce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:05.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.752+0000 7fbd412fb640 1 --2- 172.21.15.106:0/3385112359 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbd3c0c58d0 0x7fbd3c0cce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.752+0000 7fbd412fb640 1 -- 172.21.15.106:0/3385112359 >> v1:172.21.15.134:6789/0 conn(0x7fbd3c0cb2b0 legacy=0x7fbd3c070bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:05.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.752+0000 7fbd412fb640 1 -- 172.21.15.106:0/3385112359 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd3c15e0e0 con 0x7fbd3c0cc600 2024-04-25T10:31:05.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.752+0000 7fbd40afa640 1 --2- 172.21.15.106:0/3385112359 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbd3c0c58d0 0x7fbd3c0cce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:31:05.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.753+0000 7fbd412fb640 1 --2- 172.21.15.106:0/3385112359 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd3c0cc600 0x7fbd3c0c5390 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbd38009e50 tx=0x7fbd3802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f415d4bdba433dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:05.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.753+0000 7fbd33fff640 1 -- 172.21.15.106:0/3385112359 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd38036070 con 0x7fbd3c0cc600 2024-04-25T10:31:05.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.753+0000 7fbd33fff640 1 -- 172.21.15.106:0/3385112359 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd38031600 con 0x7fbd3c0cc600 2024-04-25T10:31:05.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.753+0000 7fbd33fff640 1 -- 172.21.15.106:0/3385112359 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd380319d0 con 0x7fbd3c0cc600 2024-04-25T10:31:05.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.754+0000 7fbd4355f640 1 -- 172.21.15.106:0/3385112359 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd3c0cc600 msgr2=0x7fbd3c0c5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:05.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.754+0000 7fbd4355f640 1 --2- 172.21.15.106:0/3385112359 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd3c0cc600 0x7fbd3c0c5390 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbd38009e50 tx=0x7fbd3802a710 comp rx=0 tx=0).stop 2024-04-25T10:31:05.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.754+0000 7fbd4355f640 1 -- 172.21.15.106:0/3385112359 shutdown_connections 2024-04-25T10:31:05.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.754+0000 7fbd4355f640 1 --2- 172.21.15.106:0/3385112359 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbd3c0c58d0 0x7fbd3c0cce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.754+0000 7fbd4355f640 1 --2- 172.21.15.106:0/3385112359 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd3c0cc600 0x7fbd3c0c5390 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.754+0000 7fbd4355f640 1 -- 172.21.15.106:0/3385112359 >> 172.21.15.106:0/3385112359 conn(0x7fbd3c0727d0 msgr2=0x7fbd3c072c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:05.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.754+0000 7fbd4355f640 1 -- 172.21.15.106:0/3385112359 shutdown_connections 2024-04-25T10:31:05.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.754+0000 7fbd4355f640 1 -- 172.21.15.106:0/3385112359 wait complete. 2024-04-25T10:31:05.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.756+0000 7fbd4355f640 1 Processor -- start 2024-04-25T10:31:05.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.756+0000 7fbd4355f640 1 -- start start 2024-04-25T10:31:05.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.757+0000 7fbd4355f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd3c0c58d0 0x7fbd3c1f10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:05.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.757+0000 7fbd4355f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbd3c0cb2b0 0x7fbd3c0cde60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:05.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.757+0000 7fbd4355f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd3c0cc600 0x7fbd3c0cfc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:05.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.757+0000 7fbd4355f640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3c15e980 con 0x7fbd3c0c58d0 2024-04-25T10:31:05.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.757+0000 7fbd4355f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3c15e6a0 con 0x7fbd3c0cc600 2024-04-25T10:31:05.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.757+0000 7fbd4355f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fbd3c0cbff0 con 0x7fbd3c0cb2b0 2024-04-25T10:31:05.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.757+0000 7fbd412fb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd3c0c58d0 0x7fbd3c1f10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:05.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.757+0000 7fbd412fb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd3c0c58d0 0x7fbd3c1f10d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58888/0 (socket says 172.21.15.106:58888) 2024-04-25T10:31:05.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.757+0000 7fbd412fb640 1 -- 172.21.15.106:0/3529907809 learned_addr learned my addr 172.21.15.106:0/3529907809 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:31:05.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.757+0000 7fbd40afa640 1 --2- 172.21.15.106:0/3529907809 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbd3c0cb2b0 0x7fbd3c0cde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:05.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.757+0000 7fbd41afc640 1 --2- 172.21.15.106:0/3529907809 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd3c0cc600 0x7fbd3c0cfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:05.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.757+0000 7fbd412fb640 1 -- 172.21.15.106:0/3529907809 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbd3c0cb2b0 msgr2=0x7fbd3c0cde60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:05.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.757+0000 7fbd412fb640 1 --2- 172.21.15.106:0/3529907809 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbd3c0cb2b0 0x7fbd3c0cde60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.757+0000 7fbd412fb640 1 -- 172.21.15.106:0/3529907809 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd3c0cc600 msgr2=0x7fbd3c0cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:05.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.757+0000 7fbd412fb640 1 --2- 172.21.15.106:0/3529907809 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd3c0cc600 0x7fbd3c0cfc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.757+0000 7fbd412fb640 1 -- 172.21.15.106:0/3529907809 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd3c1f1610 con 0x7fbd3c0c58d0 2024-04-25T10:31:05.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.757+0000 7fbd412fb640 1 --2- 172.21.15.106:0/3529907809 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd3c0c58d0 0x7fbd3c1f10d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbd38009f70 tx=0x7fbd38004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:05.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.757+0000 7fbd31ffb640 1 -- 172.21.15.106:0/3529907809 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3803f020 con 0x7fbd3c0c58d0 2024-04-25T10:31:05.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.757+0000 7fbd31ffb640 1 -- 172.21.15.106:0/3529907809 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd38037040 con 0x7fbd3c0c58d0 2024-04-25T10:31:05.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.758+0000 7fbd4355f640 1 -- 172.21.15.106:0/3529907809 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd3c1f45c0 con 0x7fbd3c0c58d0 2024-04-25T10:31:05.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.758+0000 7fbd31ffb640 1 -- 172.21.15.106:0/3529907809 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd38036070 con 0x7fbd3c0c58d0 2024-04-25T10:31:05.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.758+0000 7fbd4355f640 1 -- 172.21.15.106:0/3529907809 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd3c1f4b80 con 0x7fbd3c0c58d0 2024-04-25T10:31:05.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.760+0000 7fbd31ffb640 1 -- 172.21.15.106:0/3529907809 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbd38042020 con 0x7fbd3c0c58d0 2024-04-25T10:31:05.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.760+0000 7fbd31ffb640 1 --2- 172.21.15.106:0/3529907809 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fbd200744b0 0x7fbd200769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:05.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.760+0000 7fbd40afa640 1 --2- 172.21.15.106:0/3529907809 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fbd200744b0 0x7fbd200769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:05.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.760+0000 7fbd31ffb640 1 -- 172.21.15.106:0/3529907809 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fbd380b5530 con 0x7fbd3c0c58d0 2024-04-25T10:31:05.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.761+0000 7fbd40afa640 1 --2- 172.21.15.106:0/3529907809 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fbd200744b0 0x7fbd200769c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c154890 tx=0x7fbd2c009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:05.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.761+0000 7fbd4a40b740 1 Processor -- start 2024-04-25T10:31:05.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.761+0000 7fbd4a40b740 1 -- start start 2024-04-25T10:31:05.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.762+0000 7fbd4a40b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55922c1cd270 0x55922c56f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:05.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.762+0000 7fbd4a40b740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55922c1a43b0 0x55922c569880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:05.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.762+0000 7fbd4a40b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55922c1ac590 0x55922c569dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:05.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.762+0000 7fbd4a40b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55922c4d3740 con 0x55922c1ac590 2024-04-25T10:31:05.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.762+0000 7fbd4a40b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55922c46f3a0 con 0x55922c1cd270 2024-04-25T10:31:05.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.762+0000 7fbd4a40b740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55922c4cf700 con 0x55922c1a43b0 2024-04-25T10:31:05.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.762+0000 7fbd41afc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55922c1ac590 0x55922c569dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:05.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.762+0000 7fbd41afc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55922c1ac590 0x55922c569dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58900/0 (socket says 172.21.15.106:58900) 2024-04-25T10:31:05.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.762+0000 7fbd41afc640 1 -- 172.21.15.106:0/342479722 learned_addr learned my addr 172.21.15.106:0/342479722 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:31:05.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.762+0000 7fbd40afa640 1 --2- 172.21.15.106:0/342479722 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55922c1a43b0 0x55922c569880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:05.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.762+0000 7fbd412fb640 1 --2- 172.21.15.106:0/342479722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55922c1cd270 0x55922c56f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:05.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.763+0000 7fbd41afc640 1 -- 172.21.15.106:0/342479722 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55922c1a43b0 msgr2=0x55922c569880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:05.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.763+0000 7fbd41afc640 1 --2- 172.21.15.106:0/342479722 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55922c1a43b0 0x55922c569880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.763+0000 7fbd41afc640 1 -- 172.21.15.106:0/342479722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55922c1cd270 msgr2=0x55922c56f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:05.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.763+0000 7fbd41afc640 1 --2- 172.21.15.106:0/342479722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55922c1cd270 0x55922c56f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.763+0000 7fbd41afc640 1 -- 172.21.15.106:0/342479722 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55922c48b750 con 0x55922c1ac590 2024-04-25T10:31:05.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.763+0000 7fbd41afc640 1 --2- 172.21.15.106:0/342479722 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55922c1ac590 0x55922c569dc0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbd340017b0 tx=0x7fbd34007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73a37953145849da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:05.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.763+0000 7fbd26ffd640 1 -- 172.21.15.106:0/342479722 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd340052e0 con 0x55922c1ac590 2024-04-25T10:31:05.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.763+0000 7fbd26ffd640 1 -- 172.21.15.106:0/342479722 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd34018070 con 0x55922c1ac590 2024-04-25T10:31:05.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.763+0000 7fbd26ffd640 1 -- 172.21.15.106:0/342479722 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3400f9b0 con 0x55922c1ac590 2024-04-25T10:31:05.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.764+0000 7fbd4a40b740 1 -- 172.21.15.106:0/342479722 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55922c1ac590 msgr2=0x55922c569dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:05.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.764+0000 7fbd4a40b740 1 --2- 172.21.15.106:0/342479722 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55922c1ac590 0x55922c569dc0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbd340017b0 tx=0x7fbd34007db0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.764+0000 7fbd4a40b740 1 -- 172.21.15.106:0/342479722 shutdown_connections 2024-04-25T10:31:05.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.764+0000 7fbd4a40b740 1 --2- 172.21.15.106:0/342479722 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55922c1ac590 0x55922c569dc0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.764+0000 7fbd4a40b740 1 --2- 172.21.15.106:0/342479722 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55922c1a43b0 0x55922c569880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.764+0000 7fbd4a40b740 1 --2- 172.21.15.106:0/342479722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55922c1cd270 0x55922c56f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.764+0000 7fbd4a40b740 1 -- 172.21.15.106:0/342479722 >> 172.21.15.106:0/342479722 conn(0x55922c1c13c0 msgr2=0x55922c56dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:05.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.764+0000 7fbd4a40b740 1 -- 172.21.15.106:0/342479722 shutdown_connections 2024-04-25T10:31:05.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.764+0000 7fbd4a40b740 1 -- 172.21.15.106:0/342479722 wait complete. 2024-04-25T10:31:05.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.764+0000 7fbd4a40b740 1 -- start start 2024-04-25T10:31:05.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.765+0000 7fbd4a40b740 1 Processor -- start 2024-04-25T10:31:05.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.766+0000 7fbd4a40b740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55922c1a43b0 0x55922c58ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:05.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.766+0000 7fbd4a40b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55922c1ac590 0x55922c5a74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:05.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.766+0000 7fbd4a40b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55922c1cd270 0x55922c58c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:05.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.766+0000 7fbd4a40b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55922c5add00 con 0x55922c1cd270 2024-04-25T10:31:05.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.766+0000 7fbd4a40b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55922c45afe0 con 0x55922c1ac590 2024-04-25T10:31:05.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.766+0000 7fbd4a40b740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55922c56dbe0 con 0x55922c1a43b0 2024-04-25T10:31:05.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.766+0000 7fbd4a40b740 10 client.0 _ll_register_callbacks cb 0x55922c471150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:31:05.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.766+0000 7fbd41afc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55922c1cd270 0x55922c58c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:05.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.766+0000 7fbd41afc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55922c1cd270 0x55922c58c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58916/0 (socket says 172.21.15.106:58916) 2024-04-25T10:31:05.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.766+0000 7fbd41afc640 1 -- 172.21.15.106:0/3078747733 learned_addr learned my addr 172.21.15.106:0/3078747733 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:31:05.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.766+0000 7fbd412fb640 1 --2- 172.21.15.106:0/3078747733 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55922c1a43b0 0x55922c58ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:05.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.766+0000 7fbd40afa640 1 --2- 172.21.15.106:0/3078747733 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55922c1ac590 0x55922c5a74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:05.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.766+0000 7fbd41afc640 1 -- 172.21.15.106:0/3078747733 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55922c1a43b0 msgr2=0x55922c58ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:05.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.766+0000 7fbd41afc640 1 --2- 172.21.15.106:0/3078747733 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55922c1a43b0 0x55922c58ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.766+0000 7fbd41afc640 1 -- 172.21.15.106:0/3078747733 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55922c1ac590 msgr2=0x55922c5a74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:05.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.766+0000 7fbd41afc640 1 --2- 172.21.15.106:0/3078747733 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55922c1ac590 0x55922c5a74e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.766+0000 7fbd41afc640 1 -- 172.21.15.106:0/3078747733 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55922c1e52d0 con 0x55922c1cd270 2024-04-25T10:31:05.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.767+0000 7fbd41afc640 1 --2- 172.21.15.106:0/3078747733 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55922c1cd270 0x55922c58c6f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fbd3400f5a0 tx=0x7fbd34000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c231dacca5633c4c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:05.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.767+0000 7fbd1bfff640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T10:31:05.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.767+0000 7fbd1bfff640 1 -- 172.21.15.106:0/3078747733 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd34010d00 con 0x55922c1cd270 2024-04-25T10:31:05.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.767+0000 7fbd1bfff640 1 -- 172.21.15.106:0/3078747733 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fbd340035f0 con 0x55922c1cd270 2024-04-25T10:31:05.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.767+0000 7fbd4a40b740 1 -- 172.21.15.106:0/3078747733 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd34018070 con 0x55922c1cd270 2024-04-25T10:31:05.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.767+0000 7fbd1bfff640 1 -- 172.21.15.106:0/3078747733 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fbd3402d050 con 0x55922c1cd270 2024-04-25T10:31:05.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.767+0000 7fbd4a40b740 10 client.4719 fetch_fsmap learned FSMap version 17 2024-04-25T10:31:05.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.767+0000 7fbd4a40b740 1 -- 172.21.15.106:0/3078747733 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55922c2af6b0 con 0x55922c1cd270 2024-04-25T10:31:05.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.767+0000 7fbd1bfff640 1 -- 172.21.15.106:0/3078747733 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd34018070 con 0x55922c1cd270 2024-04-25T10:31:05.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.767+0000 7fbd1bfff640 1 -- 172.21.15.106:0/3078747733 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7fbd34007020 con 0x55922c1cd270 2024-04-25T10:31:05.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.767+0000 7fbd4a40b740 10 client.4719 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T10:31:05.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.767+0000 7fbd4a40b740 10 client.4719 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-25T10:31:05.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.767+0000 7fbd4a40b740 20 client.4719 populate_metadata read hostname 'smithi106' 2024-04-25T10:31:05.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.767+0000 7fbd4a40b740 1 --2- 172.21.15.106:0/3078747733 >> [v2:172.21.15.106:6838/486339834,v1:172.21.15.106:6839/486339834] conn(0x55922c18be40 0x55922c5ce680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:05.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.767+0000 7fbd4a40b740 4 client.4719 mds_command: new command op to 4269 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:31:05.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.767+0000 7fbd4a40b740 1 -- 172.21.15.106:0/3078747733 --> [v2:172.21.15.106:6838/486339834,v1:172.21.15.106:6839/486339834] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55922c4e1350 con 0x55922c18be40 2024-04-25T10:31:05.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.767+0000 7fbd412fb640 1 --2- 172.21.15.106:0/3078747733 >> [v2:172.21.15.106:6838/486339834,v1:172.21.15.106:6839/486339834] conn(0x55922c18be40 0x55922c5ce680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:05.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.768+0000 7fbd1bfff640 10 client.4719 ms_handle_connect on v2:172.21.15.106:6838/486339834 2024-04-25T10:31:05.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.768+0000 7fbd412fb640 1 --2- 172.21.15.106:0/3078747733 >> [v2:172.21.15.106:6838/486339834,v1:172.21.15.106:6839/486339834] conn(0x55922c18be40 0x55922c5ce680 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ba2b8b0d18c0e68f server_cookie=13f2d9de09afea49 in_seq=0 out_seq=0 2024-04-25T10:31:05.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.768+0000 7fbd1bfff640 1 -- 172.21.15.106:0/3078747733 <== mds.3 v2:172.21.15.106:6838/486339834 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55922c4e1350 con 0x55922c18be40 2024-04-25T10:31:05.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.768+0000 7fbd1bfff640 10 client.4719 handle_command_reply: tid=0 2024-04-25T10:31:05.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.768+0000 7fbd4a40b740 1 client.4719 shutdown 2024-04-25T10:31:05.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.768+0000 7fbd4a40b740 1 -- 172.21.15.106:0/3078747733 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55922c1cd270 msgr2=0x55922c58c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:05.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.768+0000 7fbd4a40b740 1 --2- 172.21.15.106:0/3078747733 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55922c1cd270 0x55922c58c6f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fbd3400f5a0 tx=0x7fbd34000ed0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.768+0000 7fbd4a40b740 1 -- 172.21.15.106:0/3078747733 shutdown_connections 2024-04-25T10:31:05.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.768+0000 7fbd4a40b740 1 --2- 172.21.15.106:0/3078747733 >> [v2:172.21.15.106:6838/486339834,v1:172.21.15.106:6839/486339834] conn(0x55922c18be40 0x55922c5ce680 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.768+0000 7fbd4a40b740 1 --2- 172.21.15.106:0/3078747733 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55922c1cd270 0x55922c58c6f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.768+0000 7fbd1bfff640 0 client.4719 ms_handle_reset on v2:172.21.15.106:6838/486339834 2024-04-25T10:31:05.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.768+0000 7fbd4a40b740 1 --2- 172.21.15.106:0/3078747733 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55922c1ac590 0x55922c5a74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.768+0000 7fbd4a40b740 1 --2- 172.21.15.106:0/3078747733 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55922c1a43b0 0x55922c58ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.768+0000 7fbd4a40b740 1 -- 172.21.15.106:0/3078747733 >> 172.21.15.106:0/3078747733 conn(0x55922c1c13c0 msgr2=0x55922c56d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:05.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.769+0000 7fbd4a40b740 1 -- 172.21.15.106:0/3078747733 shutdown_connections 2024-04-25T10:31:05.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.769+0000 7fbd4a40b740 1 -- 172.21.15.106:0/3078747733 wait complete. 2024-04-25T10:31:05.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.769+0000 7fbd4a40b740 20 client.4719 trim_cache size 0 max 16384 2024-04-25T10:31:05.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.787+0000 7fbd4a40b740 1 Processor -- start 2024-04-25T10:31:05.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.787+0000 7fbd4a40b740 1 -- start start 2024-04-25T10:31:05.791 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.789+0000 7fbd4a40b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55922c1a43b0 0x55922c569a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:05.791 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.789+0000 7fbd4a40b740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55922c1ac590 0x55922c58fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:05.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.789+0000 7fbd4a40b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55922c1cd270 0x55922c597e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:05.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.789+0000 7fbd4a40b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3c0c5f70 con 0x55922c1a43b0 2024-04-25T10:31:05.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.789+0000 7fbd4a40b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55922c58b7d0 con 0x55922c1cd270 2024-04-25T10:31:05.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.789+0000 7fbd4a40b740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55922c5a95f0 con 0x55922c1ac590 2024-04-25T10:31:05.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.789+0000 7fbd412fb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55922c1a43b0 0x55922c569a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:05.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.789+0000 7fbd412fb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55922c1a43b0 0x55922c569a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58918/0 (socket says 172.21.15.106:58918) 2024-04-25T10:31:05.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.789+0000 7fbd412fb640 1 -- 172.21.15.106:0/13275586 learned_addr learned my addr 172.21.15.106:0/13275586 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:31:05.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.789+0000 7fbd40afa640 1 --2- 172.21.15.106:0/13275586 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55922c1ac590 0x55922c58fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:05.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.789+0000 7fbd41afc640 1 --2- 172.21.15.106:0/13275586 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55922c1cd270 0x55922c597e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:05.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.789+0000 7fbd412fb640 1 -- 172.21.15.106:0/13275586 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55922c1ac590 msgr2=0x55922c58fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:05.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.789+0000 7fbd412fb640 1 --2- 172.21.15.106:0/13275586 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55922c1ac590 0x55922c58fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.789+0000 7fbd412fb640 1 -- 172.21.15.106:0/13275586 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55922c1cd270 msgr2=0x55922c597e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:05.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.789+0000 7fbd412fb640 1 --2- 172.21.15.106:0/13275586 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55922c1cd270 0x55922c597e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.789+0000 7fbd412fb640 1 -- 172.21.15.106:0/13275586 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55922c58ca60 con 0x55922c1a43b0 2024-04-25T10:31:05.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.789+0000 7fbd412fb640 1 --2- 172.21.15.106:0/13275586 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55922c1a43b0 0x55922c569a90 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbd38009e50 tx=0x7fbd380b3930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b9e0b54db7d2a25 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:05.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.789+0000 7fbd267fc640 1 -- 172.21.15.106:0/13275586 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd38083970 con 0x55922c1a43b0 2024-04-25T10:31:05.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.789+0000 7fbd267fc640 1 -- 172.21.15.106:0/13275586 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd38088420 con 0x55922c1a43b0 2024-04-25T10:31:05.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.789+0000 7fbd267fc640 1 -- 172.21.15.106:0/13275586 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3807b900 con 0x55922c1a43b0 2024-04-25T10:31:05.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.790+0000 7fbd4a40b740 1 -- 172.21.15.106:0/13275586 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55922c1a43b0 msgr2=0x55922c569a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:05.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.790+0000 7fbd4a40b740 1 --2- 172.21.15.106:0/13275586 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55922c1a43b0 0x55922c569a90 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbd38009e50 tx=0x7fbd380b3930 comp rx=0 tx=0).stop 2024-04-25T10:31:05.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.790+0000 7fbd4a40b740 1 -- 172.21.15.106:0/13275586 shutdown_connections 2024-04-25T10:31:05.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.790+0000 7fbd4a40b740 1 --2- 172.21.15.106:0/13275586 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55922c1cd270 0x55922c597e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.790+0000 7fbd4a40b740 1 --2- 172.21.15.106:0/13275586 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55922c1ac590 0x55922c58fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.790+0000 7fbd4a40b740 1 --2- 172.21.15.106:0/13275586 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55922c1a43b0 0x55922c569a90 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.790+0000 7fbd4a40b740 1 -- 172.21.15.106:0/13275586 >> 172.21.15.106:0/13275586 conn(0x55922c1c13c0 msgr2=0x55922c568300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:05.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.790+0000 7fbd4a40b740 1 -- 172.21.15.106:0/13275586 shutdown_connections 2024-04-25T10:31:05.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.790+0000 7fbd4a40b740 1 -- 172.21.15.106:0/13275586 wait complete. 2024-04-25T10:31:05.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.791+0000 7fbd4a40b740 1 -- start start 2024-04-25T10:31:05.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.791+0000 7fbd4a40b740 1 Processor -- start 2024-04-25T10:31:05.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.792+0000 7fbd4a40b740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55922c1a43b0 0x55922c56f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:05.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.792+0000 7fbd4a40b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55922c1ac590 0x55922c56de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:05.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.792+0000 7fbd4a40b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55922c1cd270 0x55922c583880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:05.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.792+0000 7fbd4a40b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55922c58ac20 con 0x55922c1ac590 2024-04-25T10:31:05.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.792+0000 7fbd4a40b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55922c58a5d0 con 0x55922c1cd270 2024-04-25T10:31:05.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.792+0000 7fbd4a40b740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55922c5c3040 con 0x55922c1a43b0 2024-04-25T10:31:05.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.792+0000 7fbd4a40b740 10 client.0 _ll_register_callbacks cb 0x7fbd380092e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:31:05.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.792+0000 7fbd40afa640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55922c1ac590 0x55922c56de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:05.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.792+0000 7fbd40afa640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55922c1ac590 0x55922c56de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58920/0 (socket says 172.21.15.106:58920) 2024-04-25T10:31:05.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.792+0000 7fbd40afa640 1 -- 172.21.15.106:0/1987812299 learned_addr learned my addr 172.21.15.106:0/1987812299 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:31:05.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.793+0000 7fbd412fb640 1 --2- 172.21.15.106:0/1987812299 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55922c1a43b0 0x55922c56f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:05.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.793+0000 7fbd40afa640 1 -- 172.21.15.106:0/1987812299 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55922c1a43b0 msgr2=0x55922c56f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:05.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.793+0000 7fbd40afa640 1 --2- 172.21.15.106:0/1987812299 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55922c1a43b0 0x55922c56f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.793+0000 7fbd41afc640 1 --2- 172.21.15.106:0/1987812299 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55922c1cd270 0x55922c583880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:05.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.793+0000 7fbd40afa640 1 -- 172.21.15.106:0/1987812299 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55922c1cd270 msgr2=0x55922c583880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:05.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.793+0000 7fbd40afa640 1 --2- 172.21.15.106:0/1987812299 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55922c1cd270 0x55922c583880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.793+0000 7fbd40afa640 1 -- 172.21.15.106:0/1987812299 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55922c56e370 con 0x55922c1ac590 2024-04-25T10:31:05.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.793+0000 7fbd40afa640 1 --2- 172.21.15.106:0/1987812299 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55922c1ac590 0x55922c56de30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x55922c490df0 tx=0x7fbd2c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e4e79fecb1e17a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:05.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.793+0000 7fbd1bfff640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T10:31:05.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.793+0000 7fbd1bfff640 1 -- 172.21.15.106:0/1987812299 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd2c023440 con 0x55922c1ac590 2024-04-25T10:31:05.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.793+0000 7fbd1bfff640 1 -- 172.21.15.106:0/1987812299 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fbd2c00f040 con 0x55922c1ac590 2024-04-25T10:31:05.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.793+0000 7fbd4a40b740 1 -- 172.21.15.106:0/1987812299 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd38088420 con 0x55922c1ac590 2024-04-25T10:31:05.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.793+0000 7fbd1bfff640 1 -- 172.21.15.106:0/1987812299 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fbd2c0235e0 con 0x55922c1ac590 2024-04-25T10:31:05.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.793+0000 7fbd4a40b740 10 client.4725 fetch_fsmap learned FSMap version 17 2024-04-25T10:31:05.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.793+0000 7fbd4a40b740 1 -- 172.21.15.106:0/1987812299 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55922c569e20 con 0x55922c1ac590 2024-04-25T10:31:05.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.793+0000 7fbd1bfff640 1 -- 172.21.15.106:0/1987812299 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd38088420 con 0x55922c1ac590 2024-04-25T10:31:05.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.793+0000 7fbd1bfff640 1 -- 172.21.15.106:0/1987812299 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7fbd2c035050 con 0x55922c1ac590 2024-04-25T10:31:05.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.793+0000 7fbd4a40b740 10 client.4725 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T10:31:05.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.793+0000 7fbd4a40b740 10 client.4725 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-25T10:31:05.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.793+0000 7fbd4a40b740 20 client.4725 populate_metadata read hostname 'smithi106' 2024-04-25T10:31:05.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.793+0000 7fbd4a40b740 1 --2- 172.21.15.106:0/1987812299 >> [v2:172.21.15.106:6838/486339834,v1:172.21.15.106:6839/486339834] conn(0x55922c626af0 0x55922c6270f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:05.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.793+0000 7fbd4a40b740 4 client.4725 mds_command: new command op to 4269 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:31:05.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.793+0000 7fbd4a40b740 1 -- 172.21.15.106:0/1987812299 --> [v2:172.21.15.106:6838/486339834,v1:172.21.15.106:6839/486339834] -- command(tid 0: {"prefix": "status"}) -- 0x55922c4db1c0 con 0x55922c626af0 2024-04-25T10:31:05.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.794+0000 7fbd412fb640 1 --2- 172.21.15.106:0/1987812299 >> [v2:172.21.15.106:6838/486339834,v1:172.21.15.106:6839/486339834] conn(0x55922c626af0 0x55922c6270f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:05.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.794+0000 7fbd1bfff640 10 client.4725 ms_handle_connect on v2:172.21.15.106:6838/486339834 2024-04-25T10:31:05.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.794+0000 7fbd412fb640 1 --2- 172.21.15.106:0/1987812299 >> [v2:172.21.15.106:6838/486339834,v1:172.21.15.106:6839/486339834] conn(0x55922c626af0 0x55922c6270f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=190110ebc5f80385 server_cookie=505c6f2287982080 in_seq=0 out_seq=0 2024-04-25T10:31:05.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.794+0000 7fbd1bfff640 1 -- 172.21.15.106:0/1987812299 <== mds.3 v2:172.21.15.106:6838/486339834 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55922c4db1c0 con 0x55922c626af0 2024-04-25T10:31:05.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.794+0000 7fbd1bfff640 10 client.4725 handle_command_reply: tid=0 2024-04-25T10:31:05.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.794+0000 7fbd4a40b740 1 client.4725 shutdown 2024-04-25T10:31:05.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.794+0000 7fbd4a40b740 1 -- 172.21.15.106:0/1987812299 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55922c1ac590 msgr2=0x55922c56de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:05.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.794+0000 7fbd4a40b740 1 --2- 172.21.15.106:0/1987812299 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55922c1ac590 0x55922c56de30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x55922c490df0 tx=0x7fbd2c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.794+0000 7fbd4a40b740 1 -- 172.21.15.106:0/1987812299 shutdown_connections 2024-04-25T10:31:05.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.794+0000 7fbd4a40b740 1 --2- 172.21.15.106:0/1987812299 >> [v2:172.21.15.106:6838/486339834,v1:172.21.15.106:6839/486339834] conn(0x55922c626af0 0x55922c6270f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.794+0000 7fbd4a40b740 1 --2- 172.21.15.106:0/1987812299 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55922c1cd270 0x55922c583880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.794+0000 7fbd1bfff640 0 client.4725 ms_handle_reset on v2:172.21.15.106:6838/486339834 2024-04-25T10:31:05.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.794+0000 7fbd4a40b740 1 --2- 172.21.15.106:0/1987812299 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55922c1ac590 0x55922c56de30 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.794+0000 7fbd4a40b740 1 --2- 172.21.15.106:0/1987812299 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55922c1a43b0 0x55922c56f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.794+0000 7fbd4a40b740 1 -- 172.21.15.106:0/1987812299 >> 172.21.15.106:0/1987812299 conn(0x55922c1c13c0 msgr2=0x55922c5ad230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:05.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.794+0000 7fbd4a40b740 1 -- 172.21.15.106:0/1987812299 shutdown_connections 2024-04-25T10:31:05.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.794+0000 7fbd4a40b740 1 -- 172.21.15.106:0/1987812299 wait complete. 2024-04-25T10:31:05.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.795+0000 7fbd4a40b740 20 client.4725 trim_cache size 0 max 16384 2024-04-25T10:31:05.802 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-25T10:31:05.802 INFO:teuthology.orchestra.run.smithi106.stdout: "cluster_fsid": "3ee77985-e8e4-41cf-81e9-7c774cf7891f", 2024-04-25T10:31:05.803 INFO:teuthology.orchestra.run.smithi106.stdout: "whoami": 3, 2024-04-25T10:31:05.803 INFO:teuthology.orchestra.run.smithi106.stdout: "id": 4269, 2024-04-25T10:31:05.803 INFO:teuthology.orchestra.run.smithi106.stdout: "want_state": "up:active", 2024-04-25T10:31:05.803 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "up:active", 2024-04-25T10:31:05.803 INFO:teuthology.orchestra.run.smithi106.stdout: "fs_name": "cephfs", 2024-04-25T10:31:05.803 INFO:teuthology.orchestra.run.smithi106.stdout: "rank_uptime": 17.236003887999999, 2024-04-25T10:31:05.803 INFO:teuthology.orchestra.run.smithi106.stdout: "mdsmap_epoch": 17, 2024-04-25T10:31:05.803 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch": 20, 2024-04-25T10:31:05.803 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch_barrier": 20, 2024-04-25T10:31:05.803 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 39.991451108 2024-04-25T10:31:05.803 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-25T10:31:05.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.795+0000 7fbd4355f640 1 -- 172.21.15.106:0/3529907809 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fbd200744b0 msgr2=0x7fbd200769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:05.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.795+0000 7fbd4355f640 1 --2- 172.21.15.106:0/3529907809 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fbd200744b0 0x7fbd200769c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c154890 tx=0x7fbd2c009190 comp rx=0 tx=0).stop 2024-04-25T10:31:05.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.795+0000 7fbd4355f640 1 -- 172.21.15.106:0/3529907809 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd3c0c58d0 msgr2=0x7fbd3c1f10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:05.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.795+0000 7fbd4355f640 1 --2- 172.21.15.106:0/3529907809 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd3c0c58d0 0x7fbd3c1f10d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbd38009f70 tx=0x7fbd38004a40 comp rx=0 tx=0).stop 2024-04-25T10:31:05.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.795+0000 7fbd4355f640 1 -- 172.21.15.106:0/3529907809 shutdown_connections 2024-04-25T10:31:05.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.795+0000 7fbd4355f640 1 --2- 172.21.15.106:0/3529907809 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fbd200744b0 0x7fbd200769c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.795+0000 7fbd4355f640 1 --2- 172.21.15.106:0/3529907809 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd3c0cc600 0x7fbd3c0cfc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.795+0000 7fbd4355f640 1 --2- 172.21.15.106:0/3529907809 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbd3c0cb2b0 0x7fbd3c0cde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.795+0000 7fbd4355f640 1 --2- 172.21.15.106:0/3529907809 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd3c0c58d0 0x7fbd3c1f10d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.795+0000 7fbd4355f640 1 -- 172.21.15.106:0/3529907809 >> 172.21.15.106:0/3529907809 conn(0x7fbd3c0727d0 msgr2=0x7fbd3c0ca650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:05.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.795+0000 7fbd4355f640 1 -- 172.21.15.106:0/3529907809 shutdown_connections 2024-04-25T10:31:05.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.795+0000 7fbd4355f640 1 -- 172.21.15.106:0/3529907809 wait complete. 2024-04-25T10:31:05.807 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T10:31:05.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.915+0000 7f932f1ee640 1 Processor -- start 2024-04-25T10:31:05.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.915+0000 7f932f1ee640 1 -- start start 2024-04-25T10:31:05.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.916+0000 7f932f1ee640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f93280c5400 0x7f93280c5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:05.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.916+0000 7f932f1ee640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f932815e020 0x7f932815e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:05.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.916+0000 7f932f1ee640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93280caa00 con 0x7f9328154c60 2024-04-25T10:31:05.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.916+0000 7f932f1ee640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f93280cb880 con 0x7f93280c5400 2024-04-25T10:31:05.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.916+0000 7f932f1ee640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f93280cac70 con 0x7f932815e020 2024-04-25T10:31:05.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.916+0000 7f9327fff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f93280c5400 0x7f93280c5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:05.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.916+0000 7f932d78b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f932815e020 0x7f932815e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:05.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.917+0000 7f932cf8a640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f9328154c60 0x7f9328070ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.106:54870/0 (socket says 172.21.15.106:54870) 2024-04-25T10:31:05.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.917+0000 7f9327fff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f93280c5400 0x7f93280c5810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.106:49724/0 (socket says 172.21.15.106:49724) 2024-04-25T10:31:05.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.917+0000 7f932cf8a640 1 -- 172.21.15.106:0/3196697056 learned_addr learned my addr 172.21.15.106:0/3196697056 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:31:05.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.917+0000 7f9327fff640 1 -- 172.21.15.106:0/3196697056 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f932815e020 msgr2=0x7f932815e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:05.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.917+0000 7f9327fff640 1 --2- 172.21.15.106:0/3196697056 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f932815e020 0x7f932815e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.917+0000 7f9327fff640 1 -- 172.21.15.106:0/3196697056 >> v1:172.21.15.134:6789/0 conn(0x7f9328154c60 legacy=0x7f9328070ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:05.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.917+0000 7f9327fff640 1 -- 172.21.15.106:0/3196697056 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93280caf50 con 0x7f93280c5400 2024-04-25T10:31:05.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.917+0000 7f932d78b640 1 --2- 172.21.15.106:0/3196697056 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f932815e020 0x7f932815e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:31:05.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.918+0000 7f9327fff640 1 --2- 172.21.15.106:0/3196697056 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93280c5400 0x7f93280c5810 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f93140097a0 tx=0x7f931402a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=500eb6beabc565b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:05.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.918+0000 7f93277fe640 1 -- 172.21.15.106:0/3196697056 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9314037070 con 0x7f93280c5400 2024-04-25T10:31:05.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.918+0000 7f93277fe640 1 -- 172.21.15.106:0/3196697056 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f931402ada0 con 0x7f93280c5400 2024-04-25T10:31:05.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.918+0000 7f93277fe640 1 -- 172.21.15.106:0/3196697056 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9314032660 con 0x7f93280c5400 2024-04-25T10:31:05.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.918+0000 7f932f1ee640 1 -- 172.21.15.106:0/3196697056 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93280c5400 msgr2=0x7f93280c5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:05.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.918+0000 7f932f1ee640 1 --2- 172.21.15.106:0/3196697056 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93280c5400 0x7f93280c5810 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f93140097a0 tx=0x7f931402a110 comp rx=0 tx=0).stop 2024-04-25T10:31:05.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.918+0000 7f932f1ee640 1 -- 172.21.15.106:0/3196697056 shutdown_connections 2024-04-25T10:31:05.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.918+0000 7f932f1ee640 1 --2- 172.21.15.106:0/3196697056 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f932815e020 0x7f932815e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.918+0000 7f932f1ee640 1 --2- 172.21.15.106:0/3196697056 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93280c5400 0x7f93280c5810 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.918+0000 7f932f1ee640 1 -- 172.21.15.106:0/3196697056 >> 172.21.15.106:0/3196697056 conn(0x7f93280cff60 msgr2=0x7f93280d0370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:05.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.918+0000 7f932f1ee640 1 -- 172.21.15.106:0/3196697056 shutdown_connections 2024-04-25T10:31:05.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.918+0000 7f932f1ee640 1 -- 172.21.15.106:0/3196697056 wait complete. 2024-04-25T10:31:05.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.920+0000 7f932f1ee640 1 Processor -- start 2024-04-25T10:31:05.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.920+0000 7f932f1ee640 1 -- start start 2024-04-25T10:31:05.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.922+0000 7f932f1ee640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f93280c5400 0x7f9328150ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:05.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.922+0000 7f932f1ee640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9328154c60 0x7f9328147680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:05.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.922+0000 7f932f1ee640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f932815e020 0x7f9328147bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:05.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.922+0000 7f932f1ee640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f93280713f0 con 0x7f9328154c60 2024-04-25T10:31:05.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.922+0000 7f932f1ee640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f93280cb510 con 0x7f932815e020 2024-04-25T10:31:05.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.922+0000 7f932f1ee640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f9328071570 con 0x7f93280c5400 2024-04-25T10:31:05.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.922+0000 7f9327fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9328154c60 0x7f9328147680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:05.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.922+0000 7f9327fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9328154c60 0x7f9328147680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58936/0 (socket says 172.21.15.106:58936) 2024-04-25T10:31:05.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.922+0000 7f9327fff640 1 -- 172.21.15.106:0/3060512612 learned_addr learned my addr 172.21.15.106:0/3060512612 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:31:05.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.922+0000 7f932d78b640 1 --2- 172.21.15.106:0/3060512612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f932815e020 0x7f9328147bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:05.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.922+0000 7f932cf8a640 1 --2- 172.21.15.106:0/3060512612 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f93280c5400 0x7f9328150ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:05.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.922+0000 7f9327fff640 1 -- 172.21.15.106:0/3060512612 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f93280c5400 msgr2=0x7f9328150ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:05.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.922+0000 7f9327fff640 1 --2- 172.21.15.106:0/3060512612 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f93280c5400 0x7f9328150ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.922+0000 7f9327fff640 1 -- 172.21.15.106:0/3060512612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f932815e020 msgr2=0x7f9328147bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:05.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.922+0000 7f9327fff640 1 --2- 172.21.15.106:0/3060512612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f932815e020 0x7f9328147bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.922+0000 7f9327fff640 1 -- 172.21.15.106:0/3060512612 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9328148100 con 0x7f9328154c60 2024-04-25T10:31:05.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.922+0000 7f9327fff640 1 --2- 172.21.15.106:0/3060512612 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9328154c60 0x7f9328147680 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f93140098c0 tx=0x7f93140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:05.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.922+0000 7f93257fa640 1 -- 172.21.15.106:0/3060512612 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9314037040 con 0x7f9328154c60 2024-04-25T10:31:05.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.922+0000 7f93257fa640 1 -- 172.21.15.106:0/3060512612 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9314004030 con 0x7f9328154c60 2024-04-25T10:31:05.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.922+0000 7f932f1ee640 1 -- 172.21.15.106:0/3060512612 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93280ceb10 con 0x7f9328154c60 2024-04-25T10:31:05.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.922+0000 7f93257fa640 1 -- 172.21.15.106:0/3060512612 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9314006e10 con 0x7f9328154c60 2024-04-25T10:31:05.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.922+0000 7f932f1ee640 1 -- 172.21.15.106:0/3060512612 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93280cedf0 con 0x7f9328154c60 2024-04-25T10:31:05.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.924+0000 7f93257fa640 1 -- 172.21.15.106:0/3060512612 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9314003610 con 0x7f9328154c60 2024-04-25T10:31:05.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.924+0000 7f93257fa640 1 --2- 172.21.15.106:0/3060512612 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f930c0744b0 0x7f930c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:05.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.925+0000 7f93257fa640 1 -- 172.21.15.106:0/3060512612 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f93140b52b0 con 0x7f9328154c60 2024-04-25T10:31:05.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.925+0000 7f932cf8a640 1 --2- 172.21.15.106:0/3060512612 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f930c0744b0 0x7f930c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:05.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.925+0000 7f932cf8a640 1 --2- 172.21.15.106:0/3060512612 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f930c0744b0 0x7f930c0769c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f931c000f60 tx=0x7f931c007e20 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:05.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.925+0000 7f9336177740 1 Processor -- start 2024-04-25T10:31:05.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.926+0000 7f9336177740 1 -- start start 2024-04-25T10:31:05.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.927+0000 7f9336177740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5612e988c270 0x5612e9c2e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:05.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.927+0000 7f9336177740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612e98633b0 0x5612e9c28880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:05.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.927+0000 7f9336177740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5612e986b590 0x5612e9c28dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:05.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.927+0000 7f9336177740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5612e9b92740 con 0x5612e986b590 2024-04-25T10:31:05.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.927+0000 7f9336177740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5612e9b2e3a0 con 0x5612e98633b0 2024-04-25T10:31:05.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.927+0000 7f9336177740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x5612e9b8e700 con 0x5612e988c270 2024-04-25T10:31:05.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.927+0000 7f932d78b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5612e986b590 0x5612e9c28dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:05.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.927+0000 7f932d78b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5612e986b590 0x5612e9c28dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58950/0 (socket says 172.21.15.106:58950) 2024-04-25T10:31:05.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.927+0000 7f932d78b640 1 -- 172.21.15.106:0/3161405236 learned_addr learned my addr 172.21.15.106:0/3161405236 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:31:05.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.927+0000 7f9327fff640 1 --2- 172.21.15.106:0/3161405236 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612e98633b0 0x5612e9c28880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:05.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.927+0000 7f932cf8a640 1 --2- 172.21.15.106:0/3161405236 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5612e988c270 0x5612e9c2e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:05.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.927+0000 7f932d78b640 1 -- 172.21.15.106:0/3161405236 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5612e988c270 msgr2=0x5612e9c2e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:05.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.927+0000 7f932d78b640 1 --2- 172.21.15.106:0/3161405236 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5612e988c270 0x5612e9c2e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.927+0000 7f932d78b640 1 -- 172.21.15.106:0/3161405236 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612e98633b0 msgr2=0x5612e9c28880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:05.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.927+0000 7f932d78b640 1 --2- 172.21.15.106:0/3161405236 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612e98633b0 0x5612e9c28880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.927+0000 7f932d78b640 1 -- 172.21.15.106:0/3161405236 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612e9b4a750 con 0x5612e986b590 2024-04-25T10:31:05.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.927+0000 7f932cf8a640 1 --2- 172.21.15.106:0/3161405236 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5612e988c270 0x5612e9c2e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:31:05.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.927+0000 7f932d78b640 1 --2- 172.21.15.106:0/3161405236 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5612e986b590 0x5612e9c28dc0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f93200113d0 tx=0x7f9320009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1cfceaa9cd6b68c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:05.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.927+0000 7f93127fc640 1 -- 172.21.15.106:0/3161405236 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9320003b90 con 0x5612e986b590 2024-04-25T10:31:05.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.927+0000 7f93127fc640 1 -- 172.21.15.106:0/3161405236 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9320003d30 con 0x5612e986b590 2024-04-25T10:31:05.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.927+0000 7f93127fc640 1 -- 172.21.15.106:0/3161405236 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9320015900 con 0x5612e986b590 2024-04-25T10:31:05.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.928+0000 7f9336177740 1 -- 172.21.15.106:0/3161405236 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5612e986b590 msgr2=0x5612e9c28dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:05.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.928+0000 7f9336177740 1 --2- 172.21.15.106:0/3161405236 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5612e986b590 0x5612e9c28dc0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f93200113d0 tx=0x7f9320009750 comp rx=0 tx=0).stop 2024-04-25T10:31:05.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.928+0000 7f9336177740 1 -- 172.21.15.106:0/3161405236 shutdown_connections 2024-04-25T10:31:05.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.928+0000 7f9336177740 1 --2- 172.21.15.106:0/3161405236 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5612e986b590 0x5612e9c28dc0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.928+0000 7f9336177740 1 --2- 172.21.15.106:0/3161405236 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612e98633b0 0x5612e9c28880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.928+0000 7f9336177740 1 --2- 172.21.15.106:0/3161405236 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5612e988c270 0x5612e9c2e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.928+0000 7f9336177740 1 -- 172.21.15.106:0/3161405236 >> 172.21.15.106:0/3161405236 conn(0x5612e98803c0 msgr2=0x5612e9c2cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:05.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.928+0000 7f9336177740 1 -- 172.21.15.106:0/3161405236 shutdown_connections 2024-04-25T10:31:05.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.928+0000 7f9336177740 1 -- 172.21.15.106:0/3161405236 wait complete. 2024-04-25T10:31:05.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.929+0000 7f9336177740 1 -- start start 2024-04-25T10:31:05.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.929+0000 7f9336177740 1 Processor -- start 2024-04-25T10:31:05.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.930+0000 7f9336177740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5612e98633b0 0x5612e9c66a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:05.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.931+0000 7f9336177740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5612e986b590 0x5612e9c68210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:05.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.931+0000 7f9336177740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612e988c270 0x5612e9c4b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:05.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.931+0000 7f9336177740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5612e9c6cd00 con 0x5612e986b590 2024-04-25T10:31:05.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.931+0000 7f9336177740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5612e9b19fe0 con 0x5612e988c270 2024-04-25T10:31:05.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.931+0000 7f9336177740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x5612e9c2cbe0 con 0x5612e98633b0 2024-04-25T10:31:05.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.931+0000 7f9336177740 10 client.0 _ll_register_callbacks cb 0x5612e9b30150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:31:05.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.931+0000 7f9327fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5612e986b590 0x5612e9c68210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:05.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.931+0000 7f9327fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5612e986b590 0x5612e9c68210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58954/0 (socket says 172.21.15.106:58954) 2024-04-25T10:31:05.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.931+0000 7f9327fff640 1 -- 172.21.15.106:0/4111193749 learned_addr learned my addr 172.21.15.106:0/4111193749 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:31:05.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.931+0000 7f932d78b640 1 --2- 172.21.15.106:0/4111193749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612e988c270 0x5612e9c4b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:05.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.931+0000 7f932cf8a640 1 --2- 172.21.15.106:0/4111193749 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5612e98633b0 0x5612e9c66a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:05.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.931+0000 7f9327fff640 1 -- 172.21.15.106:0/4111193749 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5612e98633b0 msgr2=0x5612e9c66a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:05.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.931+0000 7f9327fff640 1 --2- 172.21.15.106:0/4111193749 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5612e98633b0 0x5612e9c66a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.931+0000 7f9327fff640 1 -- 172.21.15.106:0/4111193749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612e988c270 msgr2=0x5612e9c4b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:05.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.931+0000 7f9327fff640 1 --2- 172.21.15.106:0/4111193749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612e988c270 0x5612e9c4b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.931+0000 7f9327fff640 1 -- 172.21.15.106:0/4111193749 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612e98a42d0 con 0x5612e986b590 2024-04-25T10:31:05.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.931+0000 7f9327fff640 1 --2- 172.21.15.106:0/4111193749 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5612e986b590 0x5612e9c68210 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f931407c5f0 tx=0x7f931407ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d87245b50ad0d8b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:05.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.931+0000 7f93077fe640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T10:31:05.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.931+0000 7f93077fe640 1 -- 172.21.15.106:0/4111193749 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f931407f7e0 con 0x5612e986b590 2024-04-25T10:31:05.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.931+0000 7f93077fe640 1 -- 172.21.15.106:0/4111193749 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9314087d50 con 0x5612e986b590 2024-04-25T10:31:05.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.931+0000 7f9336177740 1 -- 172.21.15.106:0/4111193749 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9320003d30 con 0x5612e986b590 2024-04-25T10:31:05.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.931+0000 7f93077fe640 1 -- 172.21.15.106:0/4111193749 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f931407f380 con 0x5612e986b590 2024-04-25T10:31:05.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.931+0000 7f9336177740 10 client.4734 fetch_fsmap learned FSMap version 17 2024-04-25T10:31:05.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.931+0000 7f9336177740 1 -- 172.21.15.106:0/4111193749 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x5612e996e6b0 con 0x5612e986b590 2024-04-25T10:31:05.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.931+0000 7f93077fe640 1 -- 172.21.15.106:0/4111193749 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9320003d30 con 0x5612e986b590 2024-04-25T10:31:05.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.932+0000 7f93077fe640 1 -- 172.21.15.106:0/4111193749 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7f931403a030 con 0x5612e986b590 2024-04-25T10:31:05.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.932+0000 7f9336177740 10 client.4734 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T10:31:05.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.932+0000 7f9336177740 10 client.4734 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T10:31:05.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.932+0000 7f9336177740 20 client.4734 populate_metadata read hostname 'smithi106' 2024-04-25T10:31:05.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.932+0000 7f9336177740 1 --2- 172.21.15.106:0/4111193749 >> [v2:172.21.15.134:6833/2781656864,v1:172.21.15.134:6836/2781656864] conn(0x5612e984ae40 0x5612e9c488e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:05.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.932+0000 7f9336177740 4 client.4734 mds_command: new command op to 4284 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:31:05.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.932+0000 7f9336177740 1 -- 172.21.15.106:0/4111193749 --> [v2:172.21.15.134:6833/2781656864,v1:172.21.15.134:6836/2781656864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612e9ba0350 con 0x5612e984ae40 2024-04-25T10:31:05.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.932+0000 7f932cf8a640 1 --2- 172.21.15.106:0/4111193749 >> [v2:172.21.15.134:6833/2781656864,v1:172.21.15.134:6836/2781656864] conn(0x5612e984ae40 0x5612e9c488e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:05.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.932+0000 7f93077fe640 10 client.4734 ms_handle_connect on v2:172.21.15.134:6833/2781656864 2024-04-25T10:31:05.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.932+0000 7f932cf8a640 1 --2- 172.21.15.106:0/4111193749 >> [v2:172.21.15.134:6833/2781656864,v1:172.21.15.134:6836/2781656864] conn(0x5612e984ae40 0x5612e9c488e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3ad213f080ee94a7 server_cookie=fd5e83cf9ac96028 in_seq=0 out_seq=0 2024-04-25T10:31:05.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.933+0000 7f93077fe640 1 -- 172.21.15.106:0/4111193749 <== mds.2 v2:172.21.15.134:6833/2781656864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5612e9ba0350 con 0x5612e984ae40 2024-04-25T10:31:05.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.933+0000 7f93077fe640 10 client.4734 handle_command_reply: tid=0 2024-04-25T10:31:05.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.933+0000 7f9336177740 1 client.4734 shutdown 2024-04-25T10:31:05.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.933+0000 7f9336177740 1 -- 172.21.15.106:0/4111193749 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5612e986b590 msgr2=0x5612e9c68210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:05.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.933+0000 7f9336177740 1 --2- 172.21.15.106:0/4111193749 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5612e986b590 0x5612e9c68210 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f931407c5f0 tx=0x7f931407ee90 comp rx=0 tx=0).stop 2024-04-25T10:31:05.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.933+0000 7f9336177740 1 -- 172.21.15.106:0/4111193749 shutdown_connections 2024-04-25T10:31:05.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.933+0000 7f9336177740 1 --2- 172.21.15.106:0/4111193749 >> [v2:172.21.15.134:6833/2781656864,v1:172.21.15.134:6836/2781656864] conn(0x5612e984ae40 0x5612e9c488e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.933+0000 7f9336177740 1 --2- 172.21.15.106:0/4111193749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612e988c270 0x5612e9c4b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.933+0000 7f93077fe640 0 client.4734 ms_handle_reset on v2:172.21.15.134:6833/2781656864 2024-04-25T10:31:05.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.933+0000 7f9336177740 1 --2- 172.21.15.106:0/4111193749 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5612e986b590 0x5612e9c68210 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.933+0000 7f9336177740 1 --2- 172.21.15.106:0/4111193749 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5612e98633b0 0x5612e9c66a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.933+0000 7f9336177740 1 -- 172.21.15.106:0/4111193749 >> 172.21.15.106:0/4111193749 conn(0x5612e98803c0 msgr2=0x5612e9c27b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:05.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.933+0000 7f9336177740 1 -- 172.21.15.106:0/4111193749 shutdown_connections 2024-04-25T10:31:05.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.933+0000 7f9336177740 1 -- 172.21.15.106:0/4111193749 wait complete. 2024-04-25T10:31:05.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.933+0000 7f9336177740 20 client.4734 trim_cache size 0 max 16384 2024-04-25T10:31:05.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.952+0000 7f9336177740 1 Processor -- start 2024-04-25T10:31:05.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.952+0000 7f9336177740 1 -- start start 2024-04-25T10:31:05.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.953+0000 7f9336177740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5612e98633b0 0x5612e9c2bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:05.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.953+0000 7f9336177740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612e986b590 0x5612e9c2c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:05.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.953+0000 7f9336177740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5612e988c270 0x5612e9c4a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:05.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.953+0000 7f9336177740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f93281e6a90 con 0x5612e988c270 2024-04-25T10:31:05.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.953+0000 7f9336177740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5612e9c65f30 con 0x5612e986b590 2024-04-25T10:31:05.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.953+0000 7f9336177740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x5612e9c664f0 con 0x5612e98633b0 2024-04-25T10:31:05.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.953+0000 7f932d78b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5612e988c270 0x5612e9c4a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:05.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.953+0000 7f932d78b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5612e988c270 0x5612e9c4a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58962/0 (socket says 172.21.15.106:58962) 2024-04-25T10:31:05.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.953+0000 7f932d78b640 1 -- 172.21.15.106:0/3710212206 learned_addr learned my addr 172.21.15.106:0/3710212206 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:31:05.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.953+0000 7f9327fff640 1 --2- 172.21.15.106:0/3710212206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612e986b590 0x5612e9c2c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:05.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.953+0000 7f932cf8a640 1 --2- 172.21.15.106:0/3710212206 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5612e98633b0 0x5612e9c2bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:05.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.954+0000 7f932d78b640 1 -- 172.21.15.106:0/3710212206 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5612e98633b0 msgr2=0x5612e9c2bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:05.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.954+0000 7f932d78b640 1 --2- 172.21.15.106:0/3710212206 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5612e98633b0 0x5612e9c2bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.954+0000 7f932d78b640 1 -- 172.21.15.106:0/3710212206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612e986b590 msgr2=0x5612e9c2c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:05.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.954+0000 7f932d78b640 1 --2- 172.21.15.106:0/3710212206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612e986b590 0x5612e9c2c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.954+0000 7f932d78b640 1 -- 172.21.15.106:0/3710212206 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612e9c4be30 con 0x5612e988c270 2024-04-25T10:31:05.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.954+0000 7f932d78b640 1 --2- 172.21.15.106:0/3710212206 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5612e988c270 0x5612e9c4a6f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f932000afd0 tx=0x7f9320016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29ece669d86386f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:05.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.954+0000 7f9311ffb640 1 -- 172.21.15.106:0/3710212206 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9320003990 con 0x5612e988c270 2024-04-25T10:31:05.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.954+0000 7f9311ffb640 1 -- 172.21.15.106:0/3710212206 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9320003b50 con 0x5612e988c270 2024-04-25T10:31:05.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.954+0000 7f9311ffb640 1 -- 172.21.15.106:0/3710212206 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9320015700 con 0x5612e988c270 2024-04-25T10:31:05.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.955+0000 7f9336177740 1 -- 172.21.15.106:0/3710212206 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5612e988c270 msgr2=0x5612e9c4a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:05.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.955+0000 7f9336177740 1 --2- 172.21.15.106:0/3710212206 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5612e988c270 0x5612e9c4a6f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f932000afd0 tx=0x7f9320016000 comp rx=0 tx=0).stop 2024-04-25T10:31:05.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.955+0000 7f9336177740 1 -- 172.21.15.106:0/3710212206 shutdown_connections 2024-04-25T10:31:05.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.955+0000 7f9336177740 1 --2- 172.21.15.106:0/3710212206 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5612e988c270 0x5612e9c4a6f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.955+0000 7f9336177740 1 --2- 172.21.15.106:0/3710212206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612e986b590 0x5612e9c2c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.955+0000 7f9336177740 1 --2- 172.21.15.106:0/3710212206 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5612e98633b0 0x5612e9c2bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.955+0000 7f9336177740 1 -- 172.21.15.106:0/3710212206 >> 172.21.15.106:0/3710212206 conn(0x5612e98803c0 msgr2=0x5612e9c4e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:05.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.955+0000 7f9336177740 1 -- 172.21.15.106:0/3710212206 shutdown_connections 2024-04-25T10:31:05.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.955+0000 7f9336177740 1 -- 172.21.15.106:0/3710212206 wait complete. 2024-04-25T10:31:05.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.955+0000 7f9336177740 1 -- start start 2024-04-25T10:31:05.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.956+0000 7f9336177740 1 Processor -- start 2024-04-25T10:31:05.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.957+0000 7f9336177740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612e98633b0 0x5612e9cecc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:05.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.957+0000 7f9336177740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5612e986b590 0x5612e9c2d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:05.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.957+0000 7f9336177740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5612e988c270 0x5612e9c3dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:05.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.957+0000 7f9336177740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5612e9c568d0 con 0x5612e988c270 2024-04-25T10:31:05.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.957+0000 7f9336177740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5612e9c4c740 con 0x5612e98633b0 2024-04-25T10:31:05.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.957+0000 7f9336177740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x5612e9c4bbe0 con 0x5612e986b590 2024-04-25T10:31:05.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.957+0000 7f9336177740 10 client.0 _ll_register_callbacks cb 0x7f9314009400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:31:05.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.957+0000 7f932d78b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5612e988c270 0x5612e9c3dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:05.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.957+0000 7f932d78b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5612e988c270 0x5612e9c3dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58976/0 (socket says 172.21.15.106:58976) 2024-04-25T10:31:05.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.957+0000 7f932d78b640 1 -- 172.21.15.106:0/2704350052 learned_addr learned my addr 172.21.15.106:0/2704350052 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:31:05.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.957+0000 7f932cf8a640 1 --2- 172.21.15.106:0/2704350052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612e98633b0 0x5612e9cecc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:05.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.957+0000 7f9327fff640 1 --2- 172.21.15.106:0/2704350052 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5612e986b590 0x5612e9c2d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:05.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.957+0000 7f932d78b640 1 -- 172.21.15.106:0/2704350052 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5612e986b590 msgr2=0x5612e9c2d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:05.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.957+0000 7f932d78b640 1 --2- 172.21.15.106:0/2704350052 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5612e986b590 0x5612e9c2d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.957+0000 7f932d78b640 1 -- 172.21.15.106:0/2704350052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612e98633b0 msgr2=0x5612e9cecc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:05.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.957+0000 7f932d78b640 1 --2- 172.21.15.106:0/2704350052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612e98633b0 0x5612e9cecc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.957+0000 7f932d78b640 1 -- 172.21.15.106:0/2704350052 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612e9c2d810 con 0x5612e988c270 2024-04-25T10:31:05.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.957+0000 7f932cf8a640 1 --2- 172.21.15.106:0/2704350052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612e98633b0 0x5612e9cecc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:31:05.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.957+0000 7f9327fff640 1 --2- 172.21.15.106:0/2704350052 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5612e986b590 0x5612e9c2d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:31:05.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.957+0000 7f932d78b640 1 --2- 172.21.15.106:0/2704350052 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5612e988c270 0x5612e9c3dd60 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9320009790 tx=0x7f9320004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9446d9311dcb0b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:05.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.957+0000 7f93077fe640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T10:31:05.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.957+0000 7f93077fe640 1 -- 172.21.15.106:0/2704350052 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9320003990 con 0x5612e988c270 2024-04-25T10:31:05.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.958+0000 7f93077fe640 1 -- 172.21.15.106:0/2704350052 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9320004030 con 0x5612e988c270 2024-04-25T10:31:05.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.958+0000 7f9336177740 1 -- 172.21.15.106:0/2704350052 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9320003b50 con 0x5612e988c270 2024-04-25T10:31:05.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.958+0000 7f93077fe640 1 -- 172.21.15.106:0/2704350052 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f9320004270 con 0x5612e988c270 2024-04-25T10:31:05.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.958+0000 7f9336177740 10 client.4740 fetch_fsmap learned FSMap version 17 2024-04-25T10:31:05.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.958+0000 7f9336177740 1 -- 172.21.15.106:0/2704350052 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x5612e9c4af30 con 0x5612e988c270 2024-04-25T10:31:05.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.958+0000 7f93077fe640 1 -- 172.21.15.106:0/2704350052 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9320003b50 con 0x5612e988c270 2024-04-25T10:31:05.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.958+0000 7f93077fe640 1 -- 172.21.15.106:0/2704350052 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7f9320025070 con 0x5612e988c270 2024-04-25T10:31:05.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.958+0000 7f9336177740 10 client.4740 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T10:31:05.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.958+0000 7f9336177740 10 client.4740 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T10:31:05.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.958+0000 7f9336177740 20 client.4740 populate_metadata read hostname 'smithi106' 2024-04-25T10:31:05.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.958+0000 7f9336177740 1 --2- 172.21.15.106:0/2704350052 >> [v2:172.21.15.134:6833/2781656864,v1:172.21.15.134:6836/2781656864] conn(0x5612e9c87cc0 0x5612e9c646e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:05.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.958+0000 7f9336177740 4 client.4740 mds_command: new command op to 4284 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:31:05.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.958+0000 7f9336177740 1 -- 172.21.15.106:0/2704350052 --> [v2:172.21.15.134:6833/2781656864,v1:172.21.15.134:6836/2781656864] -- command(tid 0: {"prefix": "status"}) -- 0x5612e9c6cb50 con 0x5612e9c87cc0 2024-04-25T10:31:05.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.958+0000 7f932cf8a640 1 --2- 172.21.15.106:0/2704350052 >> [v2:172.21.15.134:6833/2781656864,v1:172.21.15.134:6836/2781656864] conn(0x5612e9c87cc0 0x5612e9c646e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:05.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.959+0000 7f93077fe640 10 client.4740 ms_handle_connect on v2:172.21.15.134:6833/2781656864 2024-04-25T10:31:05.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.959+0000 7f932cf8a640 1 --2- 172.21.15.106:0/2704350052 >> [v2:172.21.15.134:6833/2781656864,v1:172.21.15.134:6836/2781656864] conn(0x5612e9c87cc0 0x5612e9c646e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3f8da06fd80a74dd server_cookie=92c2281b46a80c88 in_seq=0 out_seq=0 2024-04-25T10:31:05.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.959+0000 7f93077fe640 1 -- 172.21.15.106:0/2704350052 <== mds.2 v2:172.21.15.134:6833/2781656864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x5612e9c6cb50 con 0x5612e9c87cc0 2024-04-25T10:31:05.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.959+0000 7f93077fe640 10 client.4740 handle_command_reply: tid=0 2024-04-25T10:31:05.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.959+0000 7f9336177740 1 client.4740 shutdown 2024-04-25T10:31:05.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.959+0000 7f9336177740 1 -- 172.21.15.106:0/2704350052 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5612e988c270 msgr2=0x5612e9c3dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:05.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.959+0000 7f9336177740 1 --2- 172.21.15.106:0/2704350052 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5612e988c270 0x5612e9c3dd60 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9320009790 tx=0x7f9320004ab0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.959+0000 7f9336177740 1 -- 172.21.15.106:0/2704350052 shutdown_connections 2024-04-25T10:31:05.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.959+0000 7f9336177740 1 --2- 172.21.15.106:0/2704350052 >> [v2:172.21.15.134:6833/2781656864,v1:172.21.15.134:6836/2781656864] conn(0x5612e9c87cc0 0x5612e9c646e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.959+0000 7f9336177740 1 --2- 172.21.15.106:0/2704350052 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5612e988c270 0x5612e9c3dd60 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.959+0000 7f93077fe640 0 client.4740 ms_handle_reset on v2:172.21.15.134:6833/2781656864 2024-04-25T10:31:05.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.959+0000 7f9336177740 1 --2- 172.21.15.106:0/2704350052 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5612e986b590 0x5612e9c2d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.959+0000 7f9336177740 1 --2- 172.21.15.106:0/2704350052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612e98633b0 0x5612e9cecc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.959+0000 7f9336177740 1 -- 172.21.15.106:0/2704350052 >> 172.21.15.106:0/2704350052 conn(0x5612e98803c0 msgr2=0x5612e9c2bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:05.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.959+0000 7f9336177740 1 -- 172.21.15.106:0/2704350052 shutdown_connections 2024-04-25T10:31:05.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.959+0000 7f9336177740 1 -- 172.21.15.106:0/2704350052 wait complete. 2024-04-25T10:31:05.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.959+0000 7f9336177740 20 client.4740 trim_cache size 0 max 16384 2024-04-25T10:31:05.968 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-25T10:31:05.968 INFO:teuthology.orchestra.run.smithi106.stdout: "cluster_fsid": "3ee77985-e8e4-41cf-81e9-7c774cf7891f", 2024-04-25T10:31:05.968 INFO:teuthology.orchestra.run.smithi106.stdout: "whoami": 2, 2024-04-25T10:31:05.969 INFO:teuthology.orchestra.run.smithi106.stdout: "id": 4284, 2024-04-25T10:31:05.969 INFO:teuthology.orchestra.run.smithi106.stdout: "want_state": "up:active", 2024-04-25T10:31:05.969 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "up:active", 2024-04-25T10:31:05.969 INFO:teuthology.orchestra.run.smithi106.stdout: "fs_name": "cephfs", 2024-04-25T10:31:05.969 INFO:teuthology.orchestra.run.smithi106.stdout: "rank_uptime": 22.476275076, 2024-04-25T10:31:05.969 INFO:teuthology.orchestra.run.smithi106.stdout: "mdsmap_epoch": 17, 2024-04-25T10:31:05.969 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch": 20, 2024-04-25T10:31:05.969 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch_barrier": 20, 2024-04-25T10:31:05.969 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 40.139838359000002 2024-04-25T10:31:05.969 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-25T10:31:05.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.960+0000 7f932f1ee640 1 -- 172.21.15.106:0/3060512612 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f930c0744b0 msgr2=0x7f930c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:05.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.960+0000 7f932f1ee640 1 --2- 172.21.15.106:0/3060512612 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f930c0744b0 0x7f930c0769c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f931c000f60 tx=0x7f931c007e20 comp rx=0 tx=0).stop 2024-04-25T10:31:05.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.960+0000 7f932f1ee640 1 -- 172.21.15.106:0/3060512612 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9328154c60 msgr2=0x7f9328147680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:05.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.960+0000 7f932f1ee640 1 --2- 172.21.15.106:0/3060512612 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9328154c60 0x7f9328147680 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f93140098c0 tx=0x7f93140046c0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.960+0000 7f932f1ee640 1 -- 172.21.15.106:0/3060512612 shutdown_connections 2024-04-25T10:31:05.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.960+0000 7f932f1ee640 1 --2- 172.21.15.106:0/3060512612 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f930c0744b0 0x7f930c0769c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.960+0000 7f932f1ee640 1 --2- 172.21.15.106:0/3060512612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f932815e020 0x7f9328147bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.960+0000 7f932f1ee640 1 --2- 172.21.15.106:0/3060512612 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9328154c60 0x7f9328147680 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.960+0000 7f932f1ee640 1 --2- 172.21.15.106:0/3060512612 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f93280c5400 0x7f9328150ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.960+0000 7f932f1ee640 1 -- 172.21.15.106:0/3060512612 >> 172.21.15.106:0/3060512612 conn(0x7f93280cff60 msgr2=0x7f9328153a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:05.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.960+0000 7f932f1ee640 1 -- 172.21.15.106:0/3060512612 shutdown_connections 2024-04-25T10:31:05.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:05.960+0000 7f932f1ee640 1 -- 172.21.15.106:0/3060512612 wait complete. 2024-04-25T10:31:05.972 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-04-25T10:31:06.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.078+0000 7f001aba8640 1 Processor -- start 2024-04-25T10:31:06.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.078+0000 7f001aba8640 1 -- start start 2024-04-25T10:31:06.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.079+0000 7f001aba8640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f001406dc00 0x7f0014156f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.079+0000 7f001aba8640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f001415cbd0 con 0x7f001415d690 2024-04-25T10:31:06.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.080+0000 7f001aba8640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0014160990 con 0x7f00140c61a0 2024-04-25T10:31:06.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.080+0000 7f001aba8640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f001415d200 con 0x7f001406dc00 2024-04-25T10:31:06.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.080+0000 7f0019145640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f001415d690 0x7f00140cce20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:46652/0 (socket says 172.21.15.106:46652) 2024-04-25T10:31:06.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.080+0000 7f0019145640 1 -- 172.21.15.106:0/2098375240 learned_addr learned my addr 172.21.15.106:0/2098375240 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:31:06.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.080+0000 7f0013fff640 1 --2- 172.21.15.106:0/2098375240 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f001406dc00 0x7f0014156f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.080+0000 7f00137fe640 1 -- 172.21.15.106:0/2098375240 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 605352268 0 0) 0x7f001415cbd0 con 0x7f001415d690 2024-04-25T10:31:06.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.080+0000 7f00137fe640 1 -- 172.21.15.106:0/2098375240 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efffc003520 con 0x7f001415d690 2024-04-25T10:31:06.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.080+0000 7f00137fe640 1 -- 172.21.15.106:0/2098375240 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2120234178 0 0) 0x7f0014160990 con 0x7f00140c61a0 2024-04-25T10:31:06.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.080+0000 7f0013fff640 1 -- 172.21.15.106:0/2098375240 >> v1:172.21.15.134:6789/0 conn(0x7f00140c61a0 legacy=0x7f00141601c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:06.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.080+0000 7f0013fff640 1 -- 172.21.15.106:0/2098375240 >> v1:172.21.15.106:6789/0 conn(0x7f001415d690 legacy=0x7f00140cce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:06.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.080+0000 7f0013fff640 1 -- 172.21.15.106:0/2098375240 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0014160c70 con 0x7f001406dc00 2024-04-25T10:31:06.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.081+0000 7f0013fff640 1 --2- 172.21.15.106:0/2098375240 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f001406dc00 0x7f0014156f30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f00080097d0 tx=0x7f000802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9581f01eb5a6c02 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:06.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.081+0000 7f00137fe640 1 -- 172.21.15.106:0/2098375240 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0008037070 con 0x7f001406dc00 2024-04-25T10:31:06.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.081+0000 7f00137fe640 1 -- 172.21.15.106:0/2098375240 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0008032440 con 0x7f001406dc00 2024-04-25T10:31:06.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.081+0000 7f00137fe640 1 -- 172.21.15.106:0/2098375240 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0008032810 con 0x7f001406dc00 2024-04-25T10:31:06.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.082+0000 7f001aba8640 1 -- 172.21.15.106:0/2098375240 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f001406dc00 msgr2=0x7f0014156f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:06.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.082+0000 7f001aba8640 1 --2- 172.21.15.106:0/2098375240 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f001406dc00 0x7f0014156f30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f00080097d0 tx=0x7f000802a210 comp rx=0 tx=0).stop 2024-04-25T10:31:06.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.082+0000 7f001aba8640 1 -- 172.21.15.106:0/2098375240 shutdown_connections 2024-04-25T10:31:06.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.082+0000 7f001aba8640 1 --2- 172.21.15.106:0/2098375240 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f001406dc00 0x7f0014156f30 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.082+0000 7f001aba8640 1 -- 172.21.15.106:0/2098375240 >> 172.21.15.106:0/2098375240 conn(0x7f00140727d0 msgr2=0x7f0014072c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:06.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.082+0000 7f001aba8640 1 -- 172.21.15.106:0/2098375240 shutdown_connections 2024-04-25T10:31:06.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.082+0000 7f001aba8640 1 -- 172.21.15.106:0/2098375240 wait complete. 2024-04-25T10:31:06.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.083+0000 7f001aba8640 1 Processor -- start 2024-04-25T10:31:06.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.083+0000 7f001aba8640 1 -- start start 2024-04-25T10:31:06.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.085+0000 7f001aba8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f001406dc00 0x7f00141f63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.085+0000 7f001aba8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00140c61a0 0x7f00141f3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.085+0000 7f001aba8640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f001415d690 0x7f00141507a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.085+0000 7f001aba8640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f001415daa0 con 0x7f00140c61a0 2024-04-25T10:31:06.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.085+0000 7f001aba8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f001415ce90 con 0x7f001406dc00 2024-04-25T10:31:06.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.085+0000 7f001aba8640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f001406d6d0 con 0x7f001415d690 2024-04-25T10:31:06.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.085+0000 7f0013fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00140c61a0 0x7f00141f3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.085+0000 7f0013fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00140c61a0 0x7f00141f3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58992/0 (socket says 172.21.15.106:58992) 2024-04-25T10:31:06.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.085+0000 7f0013fff640 1 -- 172.21.15.106:0/36599401 learned_addr learned my addr 172.21.15.106:0/36599401 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:31:06.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.085+0000 7f0019145640 1 --2- 172.21.15.106:0/36599401 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f001415d690 0x7f00141507a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.085+0000 7f0018944640 1 --2- 172.21.15.106:0/36599401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f001406dc00 0x7f00141f63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.085+0000 7f0013fff640 1 -- 172.21.15.106:0/36599401 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f001415d690 msgr2=0x7f00141507a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:06.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.085+0000 7f0013fff640 1 --2- 172.21.15.106:0/36599401 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f001415d690 0x7f00141507a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.085+0000 7f0013fff640 1 -- 172.21.15.106:0/36599401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f001406dc00 msgr2=0x7f00141f63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:06.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.085+0000 7f0013fff640 1 --2- 172.21.15.106:0/36599401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f001406dc00 0x7f00141f63a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.085+0000 7f0013fff640 1 -- 172.21.15.106:0/36599401 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00141f6dc0 con 0x7f00140c61a0 2024-04-25T10:31:06.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.085+0000 7f0019145640 1 --2- 172.21.15.106:0/36599401 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f001415d690 0x7f00141507a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:31:06.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.085+0000 7f0013fff640 1 --2- 172.21.15.106:0/36599401 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00140c61a0 0x7f00141f3820 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0008035040 tx=0x7f00080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:06.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.085+0000 7f00117fa640 1 -- 172.21.15.106:0/36599401 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0008037040 con 0x7f00140c61a0 2024-04-25T10:31:06.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.085+0000 7f00117fa640 1 -- 172.21.15.106:0/36599401 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0008004230 con 0x7f00140c61a0 2024-04-25T10:31:06.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.085+0000 7f00117fa640 1 -- 172.21.15.106:0/36599401 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0008003610 con 0x7f00140c61a0 2024-04-25T10:31:06.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.085+0000 7f001aba8640 1 -- 172.21.15.106:0/36599401 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00141f3d60 con 0x7f00140c61a0 2024-04-25T10:31:06.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.085+0000 7f001aba8640 1 -- 172.21.15.106:0/36599401 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00141f3020 con 0x7f00140c61a0 2024-04-25T10:31:06.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.087+0000 7f00117fa640 1 -- 172.21.15.106:0/36599401 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f00080037b0 con 0x7f00140c61a0 2024-04-25T10:31:06.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.087+0000 7f00117fa640 1 --2- 172.21.15.106:0/36599401 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f00000744b0 0x7f00000769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.087+0000 7f00117fa640 1 -- 172.21.15.106:0/36599401 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f00080b5eb0 con 0x7f00140c61a0 2024-04-25T10:31:06.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.087+0000 7f0018944640 1 --2- 172.21.15.106:0/36599401 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f00000744b0 0x7f00000769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.088+0000 7f0018944640 1 --2- 172.21.15.106:0/36599401 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f00000744b0 0x7f00000769c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0004000f60 tx=0x7f0004009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:06.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.088+0000 7f0021a54740 1 Processor -- start 2024-04-25T10:31:06.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.088+0000 7f0021a54740 1 -- start start 2024-04-25T10:31:06.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.090+0000 7f0021a54740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d7a7e19270 0x55d7a81bb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.090+0000 7f0021a54740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d7a7df03b0 0x55d7a81b5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.090+0000 7f0021a54740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7a7df8590 0x55d7a81b5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.090+0000 7f0021a54740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55d7a811f740 con 0x55d7a7df03b0 2024-04-25T10:31:06.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.090+0000 7f0021a54740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d7a80bb3a0 con 0x55d7a7df8590 2024-04-25T10:31:06.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.090+0000 7f0021a54740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55d7a811b700 con 0x55d7a7e19270 2024-04-25T10:31:06.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.090+0000 7f0013fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d7a7df03b0 0x55d7a81b5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.090+0000 7f0013fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d7a7df03b0 0x55d7a81b5880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59008/0 (socket says 172.21.15.106:59008) 2024-04-25T10:31:06.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.090+0000 7f0013fff640 1 -- 172.21.15.106:0/163406386 learned_addr learned my addr 172.21.15.106:0/163406386 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:31:06.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.090+0000 7f0018944640 1 --2- 172.21.15.106:0/163406386 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d7a7e19270 0x55d7a81bb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.090+0000 7f0019145640 1 --2- 172.21.15.106:0/163406386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7a7df8590 0x55d7a81b5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.090+0000 7f0013fff640 1 -- 172.21.15.106:0/163406386 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d7a7e19270 msgr2=0x55d7a81bb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:06.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.090+0000 7f0013fff640 1 --2- 172.21.15.106:0/163406386 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d7a7e19270 0x55d7a81bb5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.090+0000 7f0013fff640 1 -- 172.21.15.106:0/163406386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7a7df8590 msgr2=0x55d7a81b5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:06.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.090+0000 7f0013fff640 1 --2- 172.21.15.106:0/163406386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7a7df8590 0x55d7a81b5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.090+0000 7f0013fff640 1 -- 172.21.15.106:0/163406386 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7a80d7750 con 0x55d7a7df03b0 2024-04-25T10:31:06.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.090+0000 7f0013fff640 1 --2- 172.21.15.106:0/163406386 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d7a7df03b0 0x55d7a81b5880 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f00080b4490 tx=0x7f000807f9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f10b8a329511937c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:06.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.090+0000 7efffa7fc640 1 -- 172.21.15.106:0/163406386 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f000807fb70 con 0x55d7a7df03b0 2024-04-25T10:31:06.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.090+0000 7efffa7fc640 1 -- 172.21.15.106:0/163406386 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0008080760 con 0x55d7a7df03b0 2024-04-25T10:31:06.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.090+0000 7efffa7fc640 1 -- 172.21.15.106:0/163406386 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0008080060 con 0x55d7a7df03b0 2024-04-25T10:31:06.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.091+0000 7f0021a54740 1 -- 172.21.15.106:0/163406386 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d7a7df03b0 msgr2=0x55d7a81b5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:06.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.091+0000 7f0021a54740 1 --2- 172.21.15.106:0/163406386 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d7a7df03b0 0x55d7a81b5880 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f00080b4490 tx=0x7f000807f9b0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.091+0000 7f0021a54740 1 -- 172.21.15.106:0/163406386 shutdown_connections 2024-04-25T10:31:06.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.091+0000 7f0021a54740 1 --2- 172.21.15.106:0/163406386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7a7df8590 0x55d7a81b5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.091+0000 7f0021a54740 1 --2- 172.21.15.106:0/163406386 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d7a7df03b0 0x55d7a81b5880 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.091+0000 7f0021a54740 1 --2- 172.21.15.106:0/163406386 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d7a7e19270 0x55d7a81bb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.091+0000 7f0021a54740 1 -- 172.21.15.106:0/163406386 >> 172.21.15.106:0/163406386 conn(0x55d7a7e0d3c0 msgr2=0x55d7a81b9d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:06.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.091+0000 7f0021a54740 1 -- 172.21.15.106:0/163406386 shutdown_connections 2024-04-25T10:31:06.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.091+0000 7f0021a54740 1 -- 172.21.15.106:0/163406386 wait complete. 2024-04-25T10:31:06.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.092+0000 7f0021a54740 1 -- start start 2024-04-25T10:31:06.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.092+0000 7f0021a54740 1 Processor -- start 2024-04-25T10:31:06.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.093+0000 7f0021a54740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d7a7df03b0 0x55d7a81fb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.093+0000 7f0021a54740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d7a7df8590 0x55d7a820b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.093+0000 7f0021a54740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7a7e19270 0x55d7a81d90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.093+0000 7f0021a54740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55d7a81f9d00 con 0x55d7a7df8590 2024-04-25T10:31:06.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.093+0000 7f0021a54740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d7a80a6fe0 con 0x55d7a7e19270 2024-04-25T10:31:06.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.093+0000 7f0021a54740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55d7a811f740 con 0x55d7a7df03b0 2024-04-25T10:31:06.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.093+0000 7f0021a54740 10 client.0 _ll_register_callbacks cb 0x55d7a80bd150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:31:06.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.093+0000 7f0013fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d7a7df8590 0x55d7a820b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.093+0000 7f0013fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d7a7df8590 0x55d7a820b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59016/0 (socket says 172.21.15.106:59016) 2024-04-25T10:31:06.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.093+0000 7f0013fff640 1 -- 172.21.15.106:0/925662790 learned_addr learned my addr 172.21.15.106:0/925662790 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:31:06.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.094+0000 7f0018944640 1 --2- 172.21.15.106:0/925662790 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d7a7df03b0 0x55d7a81fb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.094+0000 7f0019145640 1 --2- 172.21.15.106:0/925662790 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7a7e19270 0x55d7a81d90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.094+0000 7f0013fff640 1 -- 172.21.15.106:0/925662790 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d7a7df03b0 msgr2=0x55d7a81fb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:06.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.094+0000 7f0013fff640 1 --2- 172.21.15.106:0/925662790 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d7a7df03b0 0x55d7a81fb2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.094+0000 7f0013fff640 1 -- 172.21.15.106:0/925662790 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7a7e19270 msgr2=0x55d7a81d90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:06.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.094+0000 7f0013fff640 1 --2- 172.21.15.106:0/925662790 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7a7e19270 0x55d7a81d90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.094+0000 7f0013fff640 1 -- 172.21.15.106:0/925662790 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7a7e312d0 con 0x55d7a7df8590 2024-04-25T10:31:06.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.094+0000 7f0019145640 1 --2- 172.21.15.106:0/925662790 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7a7e19270 0x55d7a81d90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:31:06.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.094+0000 7f0013fff640 1 --2- 172.21.15.106:0/925662790 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d7a7df8590 0x55d7a820b9f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0008080910 tx=0x7f0008000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=690a2de2126d9d51 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:06.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.094+0000 7efff37fe640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T10:31:06.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.094+0000 7efff37fe640 1 -- 172.21.15.106:0/925662790 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0008084410 con 0x55d7a7df8590 2024-04-25T10:31:06.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.094+0000 7efff37fe640 1 -- 172.21.15.106:0/925662790 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f00080b4050 con 0x55d7a7df8590 2024-04-25T10:31:06.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.094+0000 7f0021a54740 1 -- 172.21.15.106:0/925662790 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0008080760 con 0x55d7a7df8590 2024-04-25T10:31:06.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.094+0000 7efff37fe640 1 -- 172.21.15.106:0/925662790 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f0008088e30 con 0x55d7a7df8590 2024-04-25T10:31:06.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.094+0000 7f0021a54740 10 client.4752 fetch_fsmap learned FSMap version 17 2024-04-25T10:31:06.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.094+0000 7f0021a54740 1 -- 172.21.15.106:0/925662790 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55d7a7efb6b0 con 0x55d7a7df8590 2024-04-25T10:31:06.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.094+0000 7efff37fe640 1 -- 172.21.15.106:0/925662790 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0008080760 con 0x55d7a7df8590 2024-04-25T10:31:06.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.094+0000 7efff37fe640 1 -- 172.21.15.106:0/925662790 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7f000804e060 con 0x55d7a7df8590 2024-04-25T10:31:06.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.094+0000 7f0021a54740 10 client.4752 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T10:31:06.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.094+0000 7f0021a54740 10 client.4752 resolve_mds: resolved name 'f' to daemon mds.f 2024-04-25T10:31:06.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.094+0000 7f0021a54740 20 client.4752 populate_metadata read hostname 'smithi106' 2024-04-25T10:31:06.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.094+0000 7f0021a54740 1 --2- 172.21.15.106:0/925662790 >> [v2:172.21.15.134:6832/1993918351,v1:172.21.15.134:6835/1993918351] conn(0x55d7a7dd7e40 0x55d7a820f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.094+0000 7f0021a54740 4 client.4752 mds_command: new command op to 4287 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:31:06.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.094+0000 7f0021a54740 1 -- 172.21.15.106:0/925662790 --> [v2:172.21.15.134:6832/1993918351,v1:172.21.15.134:6835/1993918351] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7a812d350 con 0x55d7a7dd7e40 2024-04-25T10:31:06.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.095+0000 7f0018944640 1 --2- 172.21.15.106:0/925662790 >> [v2:172.21.15.134:6832/1993918351,v1:172.21.15.134:6835/1993918351] conn(0x55d7a7dd7e40 0x55d7a820f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.095+0000 7f0018944640 1 --2- 172.21.15.106:0/925662790 >> [v2:172.21.15.134:6832/1993918351,v1:172.21.15.134:6835/1993918351] conn(0x55d7a7dd7e40 0x55d7a820f0f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=80fcf07ab82e82c3 server_cookie=e55771050a1ff7d4 in_seq=0 out_seq=0 2024-04-25T10:31:06.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.095+0000 7efff37fe640 10 client.4752 ms_handle_connect on v2:172.21.15.134:6832/1993918351 2024-04-25T10:31:06.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.096+0000 7efff37fe640 1 -- 172.21.15.106:0/925662790 <== mds.1 v2:172.21.15.134:6832/1993918351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d7a812d350 con 0x55d7a7dd7e40 2024-04-25T10:31:06.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.096+0000 7efff37fe640 10 client.4752 handle_command_reply: tid=0 2024-04-25T10:31:06.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.096+0000 7f0021a54740 1 client.4752 shutdown 2024-04-25T10:31:06.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.096+0000 7f0021a54740 1 -- 172.21.15.106:0/925662790 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d7a7df8590 msgr2=0x55d7a820b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:06.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.096+0000 7f0021a54740 1 --2- 172.21.15.106:0/925662790 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d7a7df8590 0x55d7a820b9f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0008080910 tx=0x7f0008000bc0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.096+0000 7f0021a54740 1 -- 172.21.15.106:0/925662790 shutdown_connections 2024-04-25T10:31:06.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.096+0000 7f0021a54740 1 --2- 172.21.15.106:0/925662790 >> [v2:172.21.15.134:6832/1993918351,v1:172.21.15.134:6835/1993918351] conn(0x55d7a7dd7e40 0x55d7a820f0f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.096+0000 7f0021a54740 1 --2- 172.21.15.106:0/925662790 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7a7e19270 0x55d7a81d90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.096+0000 7f0021a54740 1 --2- 172.21.15.106:0/925662790 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d7a7df8590 0x55d7a820b9f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.096+0000 7efff37fe640 0 client.4752 ms_handle_reset on v2:172.21.15.134:6832/1993918351 2024-04-25T10:31:06.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.096+0000 7f0021a54740 1 --2- 172.21.15.106:0/925662790 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d7a7df03b0 0x55d7a81fb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.096+0000 7f0021a54740 1 -- 172.21.15.106:0/925662790 >> 172.21.15.106:0/925662790 conn(0x55d7a7e0d3c0 msgr2=0x55d7a81ba9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:06.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.096+0000 7f0021a54740 1 -- 172.21.15.106:0/925662790 shutdown_connections 2024-04-25T10:31:06.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.096+0000 7f0021a54740 1 -- 172.21.15.106:0/925662790 wait complete. 2024-04-25T10:31:06.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.096+0000 7f0021a54740 20 client.4752 trim_cache size 0 max 16384 2024-04-25T10:31:06.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.115+0000 7f0021a54740 1 Processor -- start 2024-04-25T10:31:06.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.115+0000 7f0021a54740 1 -- start start 2024-04-25T10:31:06.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.116+0000 7f0021a54740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d7a7dd7e40 0x55d7a8207e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.116+0000 7f0021a54740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d7a7df03b0 0x55d7a8208380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.116+0000 7f0021a54740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7a7df8590 0x55d7a81f8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.116+0000 7f0021a54740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0014153310 con 0x55d7a7dd7e40 2024-04-25T10:31:06.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.116+0000 7f0021a54740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d7a81b9be0 con 0x55d7a7df8590 2024-04-25T10:31:06.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.116+0000 7f0021a54740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55d7a7efb6b0 con 0x55d7a7df03b0 2024-04-25T10:31:06.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.117+0000 7f0018944640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d7a7dd7e40 0x55d7a8207e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.117+0000 7f0018944640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d7a7dd7e40 0x55d7a8207e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59020/0 (socket says 172.21.15.106:59020) 2024-04-25T10:31:06.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.117+0000 7f0018944640 1 -- 172.21.15.106:0/1892911234 learned_addr learned my addr 172.21.15.106:0/1892911234 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:31:06.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.117+0000 7f0013fff640 1 --2- 172.21.15.106:0/1892911234 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d7a7df03b0 0x55d7a8208380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.117+0000 7f0019145640 1 --2- 172.21.15.106:0/1892911234 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7a7df8590 0x55d7a81f8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.117+0000 7f0018944640 1 -- 172.21.15.106:0/1892911234 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d7a7df03b0 msgr2=0x55d7a8208380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:06.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.117+0000 7f0018944640 1 --2- 172.21.15.106:0/1892911234 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d7a7df03b0 0x55d7a8208380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.117+0000 7f0018944640 1 -- 172.21.15.106:0/1892911234 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7a7df8590 msgr2=0x55d7a81f8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:06.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.117+0000 7f0018944640 1 --2- 172.21.15.106:0/1892911234 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7a7df8590 0x55d7a81f8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.117+0000 7f0018944640 1 -- 172.21.15.106:0/1892911234 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7a81f9540 con 0x55d7a7dd7e40 2024-04-25T10:31:06.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.117+0000 7f0018944640 1 --2- 172.21.15.106:0/1892911234 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d7a7dd7e40 0x55d7a8207e40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f00040054e0 tx=0x7f0004011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bb1cb2e1c5efc02 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:06.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.117+0000 7efff9ffb640 1 -- 172.21.15.106:0/1892911234 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0004025500 con 0x55d7a7dd7e40 2024-04-25T10:31:06.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.117+0000 7efff9ffb640 1 -- 172.21.15.106:0/1892911234 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0004029020 con 0x55d7a7dd7e40 2024-04-25T10:31:06.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.117+0000 7efff9ffb640 1 -- 172.21.15.106:0/1892911234 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00040256a0 con 0x55d7a7dd7e40 2024-04-25T10:31:06.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.118+0000 7f0021a54740 1 -- 172.21.15.106:0/1892911234 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d7a7dd7e40 msgr2=0x55d7a8207e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:06.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.118+0000 7f0021a54740 1 --2- 172.21.15.106:0/1892911234 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d7a7dd7e40 0x55d7a8207e40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f00040054e0 tx=0x7f0004011000 comp rx=0 tx=0).stop 2024-04-25T10:31:06.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.118+0000 7f0021a54740 1 -- 172.21.15.106:0/1892911234 shutdown_connections 2024-04-25T10:31:06.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.118+0000 7f0021a54740 1 --2- 172.21.15.106:0/1892911234 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7a7df8590 0x55d7a81f8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.118+0000 7f0021a54740 1 --2- 172.21.15.106:0/1892911234 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d7a7df03b0 0x55d7a8208380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.118+0000 7f0021a54740 1 --2- 172.21.15.106:0/1892911234 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d7a7dd7e40 0x55d7a8207e40 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.118+0000 7f0021a54740 1 -- 172.21.15.106:0/1892911234 >> 172.21.15.106:0/1892911234 conn(0x55d7a7e0d3c0 msgr2=0x55d7a820b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:06.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.118+0000 7f0021a54740 1 -- 172.21.15.106:0/1892911234 shutdown_connections 2024-04-25T10:31:06.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.118+0000 7f0021a54740 1 -- 172.21.15.106:0/1892911234 wait complete. 2024-04-25T10:31:06.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.119+0000 7f0021a54740 1 -- start start 2024-04-25T10:31:06.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.119+0000 7f0021a54740 1 Processor -- start 2024-04-25T10:31:06.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.120+0000 7f0021a54740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d7a7dd7e40 0x55d7a81d45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.120+0000 7f0021a54740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7a7df03b0 0x55d7a821e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.120+0000 7f0021a54740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d7a7df8590 0x55d7a81fa2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.120+0000 7f0021a54740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55d7a81d9620 con 0x55d7a7dd7e40 2024-04-25T10:31:06.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.120+0000 7f0021a54740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d7a820c140 con 0x55d7a7df03b0 2024-04-25T10:31:06.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.120+0000 7f0021a54740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55d7a81d7800 con 0x55d7a7df8590 2024-04-25T10:31:06.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.120+0000 7f0021a54740 10 client.0 _ll_register_callbacks cb 0x7f000807bff0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:31:06.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.120+0000 7f0018944640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d7a7dd7e40 0x55d7a81d45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.120+0000 7f0018944640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d7a7dd7e40 0x55d7a81d45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59032/0 (socket says 172.21.15.106:59032) 2024-04-25T10:31:06.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.120+0000 7f0018944640 1 -- 172.21.15.106:0/2881831350 learned_addr learned my addr 172.21.15.106:0/2881831350 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:31:06.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.120+0000 7f0019145640 1 --2- 172.21.15.106:0/2881831350 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d7a7df8590 0x55d7a81fa2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.120+0000 7f0013fff640 1 --2- 172.21.15.106:0/2881831350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7a7df03b0 0x55d7a821e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.121+0000 7f0018944640 1 -- 172.21.15.106:0/2881831350 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d7a7df8590 msgr2=0x55d7a81fa2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:06.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.121+0000 7f0018944640 1 --2- 172.21.15.106:0/2881831350 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d7a7df8590 0x55d7a81fa2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.121+0000 7f0018944640 1 -- 172.21.15.106:0/2881831350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7a7df03b0 msgr2=0x55d7a821e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:06.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.121+0000 7f0018944640 1 --2- 172.21.15.106:0/2881831350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7a7df03b0 0x55d7a821e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.121+0000 7f0018944640 1 -- 172.21.15.106:0/2881831350 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7a821e960 con 0x55d7a7dd7e40 2024-04-25T10:31:06.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.121+0000 7f0018944640 1 --2- 172.21.15.106:0/2881831350 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d7a7dd7e40 0x55d7a81d45e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0004009010 tx=0x7f0004000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32a5b3bdeee0e62c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:06.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.121+0000 7efff37fe640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T10:31:06.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.121+0000 7efff37fe640 1 -- 172.21.15.106:0/2881831350 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0004025950 con 0x55d7a7dd7e40 2024-04-25T10:31:06.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.121+0000 7efff37fe640 1 -- 172.21.15.106:0/2881831350 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f000400d490 con 0x55d7a7dd7e40 2024-04-25T10:31:06.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.121+0000 7f0021a54740 1 -- 172.21.15.106:0/2881831350 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0004029020 con 0x55d7a7dd7e40 2024-04-25T10:31:06.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.121+0000 7efff37fe640 1 -- 172.21.15.106:0/2881831350 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f000403e050 con 0x55d7a7dd7e40 2024-04-25T10:31:06.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.121+0000 7f0021a54740 10 client.4758 fetch_fsmap learned FSMap version 17 2024-04-25T10:31:06.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.121+0000 7f0021a54740 1 -- 172.21.15.106:0/2881831350 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55d7a821eb70 con 0x55d7a7dd7e40 2024-04-25T10:31:06.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.121+0000 7efff37fe640 1 -- 172.21.15.106:0/2881831350 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0004029020 con 0x55d7a7dd7e40 2024-04-25T10:31:06.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.121+0000 7efff37fe640 1 -- 172.21.15.106:0/2881831350 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7f0004041020 con 0x55d7a7dd7e40 2024-04-25T10:31:06.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.121+0000 7f0021a54740 10 client.4758 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T10:31:06.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.121+0000 7f0021a54740 10 client.4758 resolve_mds: resolved name 'f' to daemon mds.f 2024-04-25T10:31:06.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.121+0000 7f0021a54740 20 client.4758 populate_metadata read hostname 'smithi106' 2024-04-25T10:31:06.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.121+0000 7f0021a54740 1 --2- 172.21.15.106:0/2881831350 >> [v2:172.21.15.134:6832/1993918351,v1:172.21.15.134:6835/1993918351] conn(0x55d7a7dad280 0x55d7a81cd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.121+0000 7f0021a54740 4 client.4758 mds_command: new command op to 4287 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:31:06.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.121+0000 7f0021a54740 1 -- 172.21.15.106:0/2881831350 --> [v2:172.21.15.134:6832/1993918351,v1:172.21.15.134:6835/1993918351] -- command(tid 0: {"prefix": "status"}) -- 0x55d7a81271c0 con 0x55d7a7dad280 2024-04-25T10:31:06.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.122+0000 7f0013fff640 1 --2- 172.21.15.106:0/2881831350 >> [v2:172.21.15.134:6832/1993918351,v1:172.21.15.134:6835/1993918351] conn(0x55d7a7dad280 0x55d7a81cd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.122+0000 7efff37fe640 10 client.4758 ms_handle_connect on v2:172.21.15.134:6832/1993918351 2024-04-25T10:31:06.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.122+0000 7f0013fff640 1 --2- 172.21.15.106:0/2881831350 >> [v2:172.21.15.134:6832/1993918351,v1:172.21.15.134:6835/1993918351] conn(0x55d7a7dad280 0x55d7a81cd680 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6363498e8958dce2 server_cookie=df67f121c4738f7a in_seq=0 out_seq=0 2024-04-25T10:31:06.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.122+0000 7efff37fe640 1 -- 172.21.15.106:0/2881831350 <== mds.1 v2:172.21.15.134:6832/1993918351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+319 (crc 0 0 0) 0x55d7a81271c0 con 0x55d7a7dad280 2024-04-25T10:31:06.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.122+0000 7efff37fe640 10 client.4758 handle_command_reply: tid=0 2024-04-25T10:31:06.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.122+0000 7f0021a54740 1 client.4758 shutdown 2024-04-25T10:31:06.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.122+0000 7f0021a54740 1 -- 172.21.15.106:0/2881831350 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d7a7dd7e40 msgr2=0x55d7a81d45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:06.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.122+0000 7f0021a54740 1 --2- 172.21.15.106:0/2881831350 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d7a7dd7e40 0x55d7a81d45e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0004009010 tx=0x7f0004000ed0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.122+0000 7f0021a54740 1 -- 172.21.15.106:0/2881831350 shutdown_connections 2024-04-25T10:31:06.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.122+0000 7f0021a54740 1 --2- 172.21.15.106:0/2881831350 >> [v2:172.21.15.134:6832/1993918351,v1:172.21.15.134:6835/1993918351] conn(0x55d7a7dad280 0x55d7a81cd680 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.122+0000 7f0021a54740 1 --2- 172.21.15.106:0/2881831350 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d7a7df8590 0x55d7a81fa2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.122+0000 7f0021a54740 1 --2- 172.21.15.106:0/2881831350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7a7df03b0 0x55d7a821e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.122+0000 7f0021a54740 1 --2- 172.21.15.106:0/2881831350 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d7a7dd7e40 0x55d7a81d45e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.122+0000 7efff37fe640 0 client.4758 ms_handle_reset on v2:172.21.15.134:6832/1993918351 2024-04-25T10:31:06.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.122+0000 7f0021a54740 1 -- 172.21.15.106:0/2881831350 >> 172.21.15.106:0/2881831350 conn(0x55d7a7e0d3c0 msgr2=0x55d7a81b8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:06.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.123+0000 7f0021a54740 1 -- 172.21.15.106:0/2881831350 shutdown_connections 2024-04-25T10:31:06.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.123+0000 7f0021a54740 1 -- 172.21.15.106:0/2881831350 wait complete. 2024-04-25T10:31:06.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.123+0000 7f0021a54740 20 client.4758 trim_cache size 0 max 16384 2024-04-25T10:31:06.137 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-25T10:31:06.137 INFO:teuthology.orchestra.run.smithi106.stdout: "cluster_fsid": "3ee77985-e8e4-41cf-81e9-7c774cf7891f", 2024-04-25T10:31:06.138 INFO:teuthology.orchestra.run.smithi106.stdout: "whoami": 1, 2024-04-25T10:31:06.138 INFO:teuthology.orchestra.run.smithi106.stdout: "id": 4287, 2024-04-25T10:31:06.138 INFO:teuthology.orchestra.run.smithi106.stdout: "want_state": "up:active", 2024-04-25T10:31:06.138 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "up:active", 2024-04-25T10:31:06.138 INFO:teuthology.orchestra.run.smithi106.stdout: "fs_name": "cephfs", 2024-04-25T10:31:06.138 INFO:teuthology.orchestra.run.smithi106.stdout: "rank_uptime": 27.662379075, 2024-04-25T10:31:06.138 INFO:teuthology.orchestra.run.smithi106.stdout: "mdsmap_epoch": 17, 2024-04-25T10:31:06.138 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch": 20, 2024-04-25T10:31:06.138 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch_barrier": 20, 2024-04-25T10:31:06.138 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 40.303334866 2024-04-25T10:31:06.138 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-25T10:31:06.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.123+0000 7f001aba8640 1 -- 172.21.15.106:0/36599401 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f00000744b0 msgr2=0x7f00000769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:06.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.123+0000 7f001aba8640 1 --2- 172.21.15.106:0/36599401 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f00000744b0 0x7f00000769c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0004000f60 tx=0x7f0004009040 comp rx=0 tx=0).stop 2024-04-25T10:31:06.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.123+0000 7f001aba8640 1 -- 172.21.15.106:0/36599401 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00140c61a0 msgr2=0x7f00141f3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:06.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.123+0000 7f001aba8640 1 --2- 172.21.15.106:0/36599401 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00140c61a0 0x7f00141f3820 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0008035040 tx=0x7f00080046c0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.123+0000 7f001aba8640 1 -- 172.21.15.106:0/36599401 shutdown_connections 2024-04-25T10:31:06.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.123+0000 7f001aba8640 1 --2- 172.21.15.106:0/36599401 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f00000744b0 0x7f00000769c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.123+0000 7f001aba8640 1 --2- 172.21.15.106:0/36599401 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f001415d690 0x7f00141507a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.123+0000 7f001aba8640 1 --2- 172.21.15.106:0/36599401 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00140c61a0 0x7f00141f3820 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.123+0000 7f001aba8640 1 --2- 172.21.15.106:0/36599401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f001406dc00 0x7f00141f63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.123+0000 7f001aba8640 1 -- 172.21.15.106:0/36599401 >> 172.21.15.106:0/36599401 conn(0x7f00140727d0 msgr2=0x7f001406e010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:06.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.123+0000 7f001aba8640 1 -- 172.21.15.106:0/36599401 shutdown_connections 2024-04-25T10:31:06.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.123+0000 7f001aba8640 1 -- 172.21.15.106:0/36599401 wait complete. 2024-04-25T10:31:06.141 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T10:31:06.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.248+0000 7f787deb6640 1 Processor -- start 2024-04-25T10:31:06.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.248+0000 7f787deb6640 1 -- start start 2024-04-25T10:31:06.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.249+0000 7f787deb6640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f78780c61a0 0x7f7878156ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.249+0000 7f787deb6640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78780c65b0 con 0x7f787815d7c0 2024-04-25T10:31:06.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.249+0000 7f787deb6640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7878160ae0 con 0x7f787806dc00 2024-04-25T10:31:06.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.249+0000 7f787deb6640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f787815dbd0 con 0x7f78780c61a0 2024-04-25T10:31:06.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.249+0000 7f7876ffd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f787815d7c0 0x7f78780ccde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:46658/0 (socket says 172.21.15.106:46658) 2024-04-25T10:31:06.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.249+0000 7f7876ffd640 1 -- 172.21.15.106:0/164913628 learned_addr learned my addr 172.21.15.106:0/164913628 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:31:06.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.250+0000 7f7877fff640 1 --2- 172.21.15.106:0/164913628 >> v2:172.21.15.134:3300/0 conn(0x7f78780c61a0 0x7f7878156ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.250+0000 7f78767fc640 1 -- 172.21.15.106:0/164913628 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1058694609 0 0) 0x7f786c001110 con 0x7f787815d7c0 2024-04-25T10:31:06.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.250+0000 7f78767fc640 1 -- 172.21.15.106:0/164913628 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7860003520 con 0x7f787815d7c0 2024-04-25T10:31:06.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.250+0000 7f78767fc640 1 -- 172.21.15.106:0/164913628 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3243811168 0 0) 0x7f7878160ae0 con 0x7f787806dc00 2024-04-25T10:31:06.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.250+0000 7f78767fc640 1 -- 172.21.15.106:0/164913628 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f786c001110 con 0x7f787806dc00 2024-04-25T10:31:06.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.250+0000 7f78767fc640 1 -- 172.21.15.106:0/164913628 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3533896251 0 0) 0x7f7860003520 con 0x7f787815d7c0 2024-04-25T10:31:06.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.250+0000 7f78767fc640 1 -- 172.21.15.106:0/164913628 >> v2:172.21.15.134:3300/0 conn(0x7f78780c61a0 msgr2=0x7f7878156ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:06.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.250+0000 7f78767fc640 1 --2- 172.21.15.106:0/164913628 >> v2:172.21.15.134:3300/0 conn(0x7f78780c61a0 0x7f7878156ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.250+0000 7f78767fc640 1 -- 172.21.15.106:0/164913628 >> v1:172.21.15.134:6789/0 conn(0x7f787806dc00 legacy=0x7f78781601c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:06.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.250+0000 7f78767fc640 1 -- 172.21.15.106:0/164913628 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7878160dc0 con 0x7f787815d7c0 2024-04-25T10:31:06.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.250+0000 7f78767fc640 1 -- 172.21.15.106:0/164913628 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3776459350 0 0) 0x7f786c001840 con 0x7f787815d7c0 2024-04-25T10:31:06.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.250+0000 7f78767fc640 1 -- 172.21.15.106:0/164913628 >> v1:172.21.15.106:6789/0 conn(0x7f787815d7c0 legacy=0x7f78780ccde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:06.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.250+0000 7f78767fc640 1 --2- 172.21.15.106:0/164913628 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7860003b90 0x7f7860003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.250+0000 7f78767fc640 1 --2- 172.21.15.106:0/164913628 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7860004800 0x7f7860004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.250+0000 7f78767fc640 1 --2- 172.21.15.106:0/164913628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7860005210 0x7f786000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.250+0000 7f78767fc640 1 -- 172.21.15.106:0/164913628 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f787815dbd0 con 0x7f7860004800 2024-04-25T10:31:06.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.250+0000 7f78767fc640 1 -- 172.21.15.106:0/164913628 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f786000c0f0 con 0x7f7860005210 2024-04-25T10:31:06.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.250+0000 7f78767fc640 1 -- 172.21.15.106:0/164913628 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f786000c2f0 con 0x7f7860003b90 2024-04-25T10:31:06.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.250+0000 7f7876ffd640 1 --2- 172.21.15.106:0/164913628 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7860004800 0x7f7860004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.250+0000 7f78777fe640 1 --2- 172.21.15.106:0/164913628 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7860003b90 0x7f7860003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.250+0000 7f7877fff640 1 --2- 172.21.15.106:0/164913628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7860005210 0x7f786000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.250+0000 7f7876ffd640 1 -- 172.21.15.106:0/164913628 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7860003b90 msgr2=0x7f7860003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:06.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.250+0000 7f7876ffd640 1 --2- 172.21.15.106:0/164913628 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7860003b90 0x7f7860003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.250+0000 7f7876ffd640 1 -- 172.21.15.106:0/164913628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7860005210 msgr2=0x7f786000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:06.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.250+0000 7f7876ffd640 1 --2- 172.21.15.106:0/164913628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7860005210 0x7f786000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.251+0000 7f7876ffd640 1 -- 172.21.15.106:0/164913628 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7860003520 con 0x7f7860004800 2024-04-25T10:31:06.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.251+0000 7f7877fff640 1 -- 172.21.15.106:0/164913628 reap_dead start 2024-04-25T10:31:06.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.251+0000 7f7876ffd640 1 --2- 172.21.15.106:0/164913628 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7860004800 0x7f7860004c10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f78780748a0 tx=0x7f7860005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c18e3debbce0e78 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:06.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.251+0000 7f78767fc640 1 -- 172.21.15.106:0/164913628 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f786c007db0 con 0x7f7860004800 2024-04-25T10:31:06.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.251+0000 7f78767fc640 1 -- 172.21.15.106:0/164913628 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f786c005af0 con 0x7f7860004800 2024-04-25T10:31:06.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.251+0000 7f78767fc640 1 -- 172.21.15.106:0/164913628 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f786c007070 con 0x7f7860004800 2024-04-25T10:31:06.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.252+0000 7f787deb6640 1 -- 172.21.15.106:0/164913628 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7860004800 msgr2=0x7f7860004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:06.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.252+0000 7f787deb6640 1 --2- 172.21.15.106:0/164913628 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7860004800 0x7f7860004c10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f78780748a0 tx=0x7f7860005830 comp rx=0 tx=0).stop 2024-04-25T10:31:06.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.252+0000 7f787deb6640 1 -- 172.21.15.106:0/164913628 shutdown_connections 2024-04-25T10:31:06.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.252+0000 7f787deb6640 1 --2- 172.21.15.106:0/164913628 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7860004800 0x7f7860004c10 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.252+0000 7f787deb6640 1 -- 172.21.15.106:0/164913628 >> 172.21.15.106:0/164913628 conn(0x7f78780727d0 msgr2=0x7f7878072c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:06.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.252+0000 7f787deb6640 1 -- 172.21.15.106:0/164913628 shutdown_connections 2024-04-25T10:31:06.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.252+0000 7f787deb6640 1 -- 172.21.15.106:0/164913628 wait complete. 2024-04-25T10:31:06.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.254+0000 7f787deb6640 1 Processor -- start 2024-04-25T10:31:06.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.254+0000 7f787deb6640 1 -- start start 2024-04-25T10:31:06.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.255+0000 7f787deb6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7860004800 0x7f7878159070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.255+0000 7f787deb6640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f787815a810 0x7f7878148800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.255+0000 7f787deb6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7878148f90 0x7f7878150750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.255+0000 7f787deb6640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f787815cfc0 con 0x7f787815a810 2024-04-25T10:31:06.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.255+0000 7f787deb6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f787815cce0 con 0x7f7860004800 2024-04-25T10:31:06.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.255+0000 7f787deb6640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f787815d2a0 con 0x7f7878148f90 2024-04-25T10:31:06.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.255+0000 7f7876ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f787815a810 0x7f7878148800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.255+0000 7f7876ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f787815a810 0x7f7878148800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59054/0 (socket says 172.21.15.106:59054) 2024-04-25T10:31:06.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.255+0000 7f7876ffd640 1 -- 172.21.15.106:0/518113171 learned_addr learned my addr 172.21.15.106:0/518113171 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:31:06.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.255+0000 7f78777fe640 1 --2- 172.21.15.106:0/518113171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7860004800 0x7f7878159070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.255+0000 7f7877fff640 1 --2- 172.21.15.106:0/518113171 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7878148f90 0x7f7878150750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.256+0000 7f7876ffd640 1 -- 172.21.15.106:0/518113171 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7878148f90 msgr2=0x7f7878150750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:06.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.256+0000 7f7876ffd640 1 --2- 172.21.15.106:0/518113171 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7878148f90 0x7f7878150750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.256+0000 7f7876ffd640 1 -- 172.21.15.106:0/518113171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7860004800 msgr2=0x7f7878159070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:06.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.256+0000 7f7876ffd640 1 --2- 172.21.15.106:0/518113171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7860004800 0x7f7878159070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.256+0000 7f7876ffd640 1 -- 172.21.15.106:0/518113171 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78781595b0 con 0x7f787815a810 2024-04-25T10:31:06.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.256+0000 7f7876ffd640 1 --2- 172.21.15.106:0/518113171 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f787815a810 0x7f7878148800 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f786c00c330 tx=0x7f78780cc3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:06.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.256+0000 7f785ffff640 1 -- 172.21.15.106:0/518113171 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f786c041070 con 0x7f787815a810 2024-04-25T10:31:06.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.256+0000 7f785ffff640 1 -- 172.21.15.106:0/518113171 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f786c007070 con 0x7f787815a810 2024-04-25T10:31:06.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.256+0000 7f785ffff640 1 -- 172.21.15.106:0/518113171 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f786c007db0 con 0x7f787815a810 2024-04-25T10:31:06.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.256+0000 7f787deb6640 1 -- 172.21.15.106:0/518113171 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78781597c0 con 0x7f787815a810 2024-04-25T10:31:06.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.256+0000 7f787deb6640 1 -- 172.21.15.106:0/518113171 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f787814e850 con 0x7f787815a810 2024-04-25T10:31:06.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.258+0000 7f785ffff640 1 -- 172.21.15.106:0/518113171 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f786c044070 con 0x7f787815a810 2024-04-25T10:31:06.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.258+0000 7f785ffff640 1 --2- 172.21.15.106:0/518113171 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f78640744e0 0x7f78640769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.258+0000 7f785ffff640 1 -- 172.21.15.106:0/518113171 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f786c0b5660 con 0x7f787815a810 2024-04-25T10:31:06.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.258+0000 7f78777fe640 1 --2- 172.21.15.106:0/518113171 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f78640744e0 0x7f78640769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.259+0000 7f78777fe640 1 --2- 172.21.15.106:0/518113171 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f78640744e0 0x7f78640769f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7868000c00 tx=0x7f7868009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:06.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.259+0000 7f7884d6e740 1 Processor -- start 2024-04-25T10:31:06.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.259+0000 7f7884d6e740 1 -- start start 2024-04-25T10:31:06.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.260+0000 7f7884d6e740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x558d5b93a270 0x558d5bcdc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.260+0000 7f7884d6e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d5b9113b0 0x558d5bcd6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.260+0000 7f7884d6e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558d5b919590 0x558d5bcd6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.260+0000 7f7884d6e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x558d5bc40740 con 0x558d5b919590 2024-04-25T10:31:06.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.260+0000 7f7884d6e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558d5bbdc3a0 con 0x558d5b9113b0 2024-04-25T10:31:06.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.260+0000 7f7884d6e740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x558d5bc3c700 con 0x558d5b93a270 2024-04-25T10:31:06.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.260+0000 7f7877fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558d5b919590 0x558d5bcd6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.260+0000 7f7877fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558d5b919590 0x558d5bcd6dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59064/0 (socket says 172.21.15.106:59064) 2024-04-25T10:31:06.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.260+0000 7f7877fff640 1 -- 172.21.15.106:0/2351480920 learned_addr learned my addr 172.21.15.106:0/2351480920 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:31:06.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.261+0000 7f78777fe640 1 --2- 172.21.15.106:0/2351480920 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x558d5b93a270 0x558d5bcdc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.261+0000 7f7876ffd640 1 --2- 172.21.15.106:0/2351480920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d5b9113b0 0x558d5bcd6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.261+0000 7f7877fff640 1 -- 172.21.15.106:0/2351480920 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x558d5b93a270 msgr2=0x558d5bcdc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:06.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.261+0000 7f7877fff640 1 --2- 172.21.15.106:0/2351480920 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x558d5b93a270 0x558d5bcdc5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.261+0000 7f7877fff640 1 -- 172.21.15.106:0/2351480920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d5b9113b0 msgr2=0x558d5bcd6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:06.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.261+0000 7f7877fff640 1 --2- 172.21.15.106:0/2351480920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d5b9113b0 0x558d5bcd6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.261+0000 7f7877fff640 1 -- 172.21.15.106:0/2351480920 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d5bbf8750 con 0x558d5b919590 2024-04-25T10:31:06.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.261+0000 7f7876ffd640 1 --2- 172.21.15.106:0/2351480920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d5b9113b0 0x558d5bcd6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:31:06.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.261+0000 7f7877fff640 1 --2- 172.21.15.106:0/2351480920 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558d5b919590 0x558d5bcd6dc0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f78780cd900 tx=0x7f7870009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec9232dcb24d4e09 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:06.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.261+0000 7f785d7fa640 1 -- 172.21.15.106:0/2351480920 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f787001a030 con 0x558d5b919590 2024-04-25T10:31:06.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.261+0000 7f785d7fa640 1 -- 172.21.15.106:0/2351480920 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7870007e40 con 0x558d5b919590 2024-04-25T10:31:06.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.261+0000 7f785d7fa640 1 -- 172.21.15.106:0/2351480920 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7870016860 con 0x558d5b919590 2024-04-25T10:31:06.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.262+0000 7f7884d6e740 1 -- 172.21.15.106:0/2351480920 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558d5b919590 msgr2=0x558d5bcd6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:06.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.262+0000 7f7884d6e740 1 --2- 172.21.15.106:0/2351480920 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558d5b919590 0x558d5bcd6dc0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f78780cd900 tx=0x7f7870009750 comp rx=0 tx=0).stop 2024-04-25T10:31:06.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.262+0000 7f7884d6e740 1 -- 172.21.15.106:0/2351480920 shutdown_connections 2024-04-25T10:31:06.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.262+0000 7f7884d6e740 1 --2- 172.21.15.106:0/2351480920 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558d5b919590 0x558d5bcd6dc0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.262+0000 7f7884d6e740 1 --2- 172.21.15.106:0/2351480920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d5b9113b0 0x558d5bcd6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.262+0000 7f7884d6e740 1 --2- 172.21.15.106:0/2351480920 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x558d5b93a270 0x558d5bcdc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.262+0000 7f7884d6e740 1 -- 172.21.15.106:0/2351480920 >> 172.21.15.106:0/2351480920 conn(0x558d5b92e3c0 msgr2=0x558d5bcdad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:06.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.262+0000 7f7884d6e740 1 -- 172.21.15.106:0/2351480920 shutdown_connections 2024-04-25T10:31:06.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.262+0000 7f7884d6e740 1 -- 172.21.15.106:0/2351480920 wait complete. 2024-04-25T10:31:06.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.263+0000 7f7884d6e740 1 -- start start 2024-04-25T10:31:06.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.263+0000 7f7884d6e740 1 Processor -- start 2024-04-25T10:31:06.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.264+0000 7f7884d6e740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x558d5b9113b0 0x558d5bd1ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.264+0000 7f7884d6e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d5b919590 0x558d5bcf9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.264+0000 7f7884d6e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558d5b93a270 0x558d5bcfd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.264+0000 7f7884d6e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x558d5bd1ad00 con 0x558d5b93a270 2024-04-25T10:31:06.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.264+0000 7f7884d6e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558d5bbc7fe0 con 0x558d5b919590 2024-04-25T10:31:06.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.264+0000 7f7884d6e740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x558d5bcdabe0 con 0x558d5b9113b0 2024-04-25T10:31:06.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.264+0000 7f7884d6e740 10 client.0 _ll_register_callbacks cb 0x558d5bbde150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:31:06.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.264+0000 7f7877fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558d5b93a270 0x558d5bcfd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.264+0000 7f78777fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x558d5b9113b0 0x558d5bd1ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.264+0000 7f7877fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558d5b93a270 0x558d5bcfd430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59068/0 (socket says 172.21.15.106:59068) 2024-04-25T10:31:06.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.264+0000 7f7877fff640 1 -- 172.21.15.106:0/1842526625 learned_addr learned my addr 172.21.15.106:0/1842526625 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:31:06.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.264+0000 7f7876ffd640 1 --2- 172.21.15.106:0/1842526625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d5b919590 0x558d5bcf9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.264+0000 7f7877fff640 1 -- 172.21.15.106:0/1842526625 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x558d5b9113b0 msgr2=0x558d5bd1ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:06.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.264+0000 7f7877fff640 1 --2- 172.21.15.106:0/1842526625 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x558d5b9113b0 0x558d5bd1ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.264+0000 7f7877fff640 1 -- 172.21.15.106:0/1842526625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d5b919590 msgr2=0x558d5bcf9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:06.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.264+0000 7f7877fff640 1 --2- 172.21.15.106:0/1842526625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d5b919590 0x558d5bcf9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.264+0000 7f7877fff640 1 -- 172.21.15.106:0/1842526625 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d5b9522d0 con 0x558d5b93a270 2024-04-25T10:31:06.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.265+0000 7f7877fff640 1 --2- 172.21.15.106:0/1842526625 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558d5b93a270 0x558d5bcfd430 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7870007b90 tx=0x7f7870004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eea4dadf57ab6a08 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:06.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.265+0000 7f78567fc640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T10:31:06.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.265+0000 7f78567fc640 1 -- 172.21.15.106:0/1842526625 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7870003b90 con 0x558d5b93a270 2024-04-25T10:31:06.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.265+0000 7f78567fc640 1 -- 172.21.15.106:0/1842526625 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7870003d30 con 0x558d5b93a270 2024-04-25T10:31:06.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.265+0000 7f7884d6e740 1 -- 172.21.15.106:0/1842526625 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7870007e40 con 0x558d5b93a270 2024-04-25T10:31:06.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.265+0000 7f78567fc640 1 -- 172.21.15.106:0/1842526625 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f7870030050 con 0x558d5b93a270 2024-04-25T10:31:06.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.265+0000 7f7884d6e740 10 client.4770 fetch_fsmap learned FSMap version 17 2024-04-25T10:31:06.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.265+0000 7f7884d6e740 1 -- 172.21.15.106:0/1842526625 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x558d5ba1c6b0 con 0x558d5b93a270 2024-04-25T10:31:06.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.265+0000 7f78567fc640 1 -- 172.21.15.106:0/1842526625 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7870007e40 con 0x558d5b93a270 2024-04-25T10:31:06.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.265+0000 7f78567fc640 1 -- 172.21.15.106:0/1842526625 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7f7870033020 con 0x558d5b93a270 2024-04-25T10:31:06.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.265+0000 7f7884d6e740 10 client.4770 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T10:31:06.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.265+0000 7f7884d6e740 10 client.4770 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:31:06.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.265+0000 7f7884d6e740 20 client.4770 populate_metadata read hostname 'smithi106' 2024-04-25T10:31:06.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.265+0000 7f7884d6e740 1 --2- 172.21.15.106:0/1842526625 >> [v2:172.21.15.134:6834/2662003420,v1:172.21.15.134:6837/2662003420] conn(0x558d5b8f8e40 0x558d5bd305d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.265+0000 7f7884d6e740 4 client.4770 mds_command: new command op to 4290 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:31:06.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.265+0000 7f7884d6e740 1 -- 172.21.15.106:0/1842526625 --> [v2:172.21.15.134:6834/2662003420,v1:172.21.15.134:6837/2662003420] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d5bc4e350 con 0x558d5b8f8e40 2024-04-25T10:31:06.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.265+0000 7f78777fe640 1 --2- 172.21.15.106:0/1842526625 >> [v2:172.21.15.134:6834/2662003420,v1:172.21.15.134:6837/2662003420] conn(0x558d5b8f8e40 0x558d5bd305d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.266+0000 7f78567fc640 10 client.4770 ms_handle_connect on v2:172.21.15.134:6834/2662003420 2024-04-25T10:31:06.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.266+0000 7f78777fe640 1 --2- 172.21.15.106:0/1842526625 >> [v2:172.21.15.134:6834/2662003420,v1:172.21.15.134:6837/2662003420] conn(0x558d5b8f8e40 0x558d5bd305d0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8092da3f5023a2a3 server_cookie=57592d10e1d81a1c in_seq=0 out_seq=0 2024-04-25T10:31:06.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.266+0000 7f78567fc640 1 -- 172.21.15.106:0/1842526625 <== mds.0 v2:172.21.15.134:6834/2662003420 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558d5bc4e350 con 0x558d5b8f8e40 2024-04-25T10:31:06.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.267+0000 7f78567fc640 10 client.4770 handle_command_reply: tid=0 2024-04-25T10:31:06.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.267+0000 7f7884d6e740 1 client.4770 shutdown 2024-04-25T10:31:06.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.267+0000 7f7884d6e740 1 -- 172.21.15.106:0/1842526625 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558d5b93a270 msgr2=0x558d5bcfd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:06.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.267+0000 7f7884d6e740 1 --2- 172.21.15.106:0/1842526625 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558d5b93a270 0x558d5bcfd430 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7870007b90 tx=0x7f7870004a40 comp rx=0 tx=0).stop 2024-04-25T10:31:06.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.267+0000 7f7884d6e740 1 -- 172.21.15.106:0/1842526625 shutdown_connections 2024-04-25T10:31:06.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.267+0000 7f7884d6e740 1 --2- 172.21.15.106:0/1842526625 >> [v2:172.21.15.134:6834/2662003420,v1:172.21.15.134:6837/2662003420] conn(0x558d5b8f8e40 0x558d5bd305d0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.267+0000 7f7884d6e740 1 --2- 172.21.15.106:0/1842526625 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558d5b93a270 0x558d5bcfd430 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.267+0000 7f7884d6e740 1 --2- 172.21.15.106:0/1842526625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d5b919590 0x558d5bcf9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.267+0000 7f78567fc640 0 client.4770 ms_handle_reset on v2:172.21.15.134:6834/2662003420 2024-04-25T10:31:06.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.267+0000 7f7884d6e740 1 --2- 172.21.15.106:0/1842526625 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x558d5b9113b0 0x558d5bd1ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.267+0000 7f7884d6e740 1 -- 172.21.15.106:0/1842526625 >> 172.21.15.106:0/1842526625 conn(0x558d5b92e3c0 msgr2=0x558d5bcd5b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:06.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.267+0000 7f7884d6e740 1 -- 172.21.15.106:0/1842526625 shutdown_connections 2024-04-25T10:31:06.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.267+0000 7f7884d6e740 1 -- 172.21.15.106:0/1842526625 wait complete. 2024-04-25T10:31:06.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.267+0000 7f7884d6e740 20 client.4770 trim_cache size 0 max 16384 2024-04-25T10:31:06.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.286+0000 7f7884d6e740 1 Processor -- start 2024-04-25T10:31:06.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.286+0000 7f7884d6e740 1 -- start start 2024-04-25T10:31:06.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.287+0000 7f7884d6e740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x558d5b9113b0 0x558d5bcf7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.287+0000 7f7884d6e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558d5b919590 0x558d5bd1efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.287+0000 7f7884d6e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d5b93a270 0x558d5bd1f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.287+0000 7f7884d6e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f78781f6a10 con 0x558d5b919590 2024-04-25T10:31:06.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.287+0000 7f7884d6e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558d5bcf8dd0 con 0x558d5b93a270 2024-04-25T10:31:06.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.287+0000 7f7884d6e740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x558d5bcfde40 con 0x558d5b9113b0 2024-04-25T10:31:06.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.287+0000 7f7876ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558d5b919590 0x558d5bd1efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.287+0000 7f7876ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558d5b919590 0x558d5bd1efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59074/0 (socket says 172.21.15.106:59074) 2024-04-25T10:31:06.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.287+0000 7f7876ffd640 1 -- 172.21.15.106:0/1341836955 learned_addr learned my addr 172.21.15.106:0/1341836955 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:31:06.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.287+0000 7f78777fe640 1 --2- 172.21.15.106:0/1341836955 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x558d5b9113b0 0x558d5bcf7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.287+0000 7f7877fff640 1 --2- 172.21.15.106:0/1341836955 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d5b93a270 0x558d5bd1f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.287+0000 7f7876ffd640 1 -- 172.21.15.106:0/1341836955 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x558d5b9113b0 msgr2=0x558d5bcf7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:06.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.287+0000 7f7876ffd640 1 --2- 172.21.15.106:0/1341836955 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x558d5b9113b0 0x558d5bcf7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.287+0000 7f7876ffd640 1 -- 172.21.15.106:0/1341836955 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d5b93a270 msgr2=0x558d5bd1f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:06.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.287+0000 7f7876ffd640 1 --2- 172.21.15.106:0/1341836955 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d5b93a270 0x558d5bd1f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.287+0000 7f7876ffd640 1 -- 172.21.15.106:0/1341836955 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d5bcdb200 con 0x558d5b919590 2024-04-25T10:31:06.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.287+0000 7f7876ffd640 1 --2- 172.21.15.106:0/1341836955 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558d5b919590 0x558d5bd1efa0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f786c07bc50 tx=0x7f786c07bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82f0f9012915cf25 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:06.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.288+0000 7f785cff9640 1 -- 172.21.15.106:0/1341836955 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f786c07f6f0 con 0x558d5b919590 2024-04-25T10:31:06.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.288+0000 7f785cff9640 1 -- 172.21.15.106:0/1341836955 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f786c083aa0 con 0x558d5b919590 2024-04-25T10:31:06.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.288+0000 7f785cff9640 1 -- 172.21.15.106:0/1341836955 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f786c088300 con 0x558d5b919590 2024-04-25T10:31:06.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.288+0000 7f7884d6e740 1 -- 172.21.15.106:0/1341836955 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558d5b919590 msgr2=0x558d5bd1efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:06.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.288+0000 7f7884d6e740 1 --2- 172.21.15.106:0/1341836955 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558d5b919590 0x558d5bd1efa0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f786c07bc50 tx=0x7f786c07bd60 comp rx=0 tx=0).stop 2024-04-25T10:31:06.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.288+0000 7f7884d6e740 1 -- 172.21.15.106:0/1341836955 shutdown_connections 2024-04-25T10:31:06.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.288+0000 7f7884d6e740 1 --2- 172.21.15.106:0/1341836955 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d5b93a270 0x558d5bd1f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.288+0000 7f7884d6e740 1 --2- 172.21.15.106:0/1341836955 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558d5b919590 0x558d5bd1efa0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.288+0000 7f7884d6e740 1 --2- 172.21.15.106:0/1341836955 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x558d5b9113b0 0x558d5bcf7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.288+0000 7f7884d6e740 1 -- 172.21.15.106:0/1341836955 >> 172.21.15.106:0/1341836955 conn(0x558d5b92e3c0 msgr2=0x558d5bcff120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:06.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.288+0000 7f7884d6e740 1 -- 172.21.15.106:0/1341836955 shutdown_connections 2024-04-25T10:31:06.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.288+0000 7f7884d6e740 1 -- 172.21.15.106:0/1341836955 wait complete. 2024-04-25T10:31:06.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.289+0000 7f7884d6e740 1 -- start start 2024-04-25T10:31:06.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.289+0000 7f7884d6e740 1 Processor -- start 2024-04-25T10:31:06.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.290+0000 7f7884d6e740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x558d5b9113b0 0x558d5bd15ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.290+0000 7f7884d6e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558d5b919590 0x558d5bd2cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.290+0000 7f7884d6e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d5b93a270 0x558d5bd938b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.290+0000 7f7884d6e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x558d5bd30450 con 0x558d5b919590 2024-04-25T10:31:06.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.290+0000 7f7884d6e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558d5bd2fe00 con 0x558d5b93a270 2024-04-25T10:31:06.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.290+0000 7f7884d6e740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x558d5bd30b20 con 0x558d5b9113b0 2024-04-25T10:31:06.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.291+0000 7f7884d6e740 10 client.0 _ll_register_callbacks cb 0x7f7868005230 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:31:06.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.291+0000 7f7876ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558d5b919590 0x558d5bd2cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.291+0000 7f7876ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558d5b919590 0x558d5bd2cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59084/0 (socket says 172.21.15.106:59084) 2024-04-25T10:31:06.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.291+0000 7f7876ffd640 1 -- 172.21.15.106:0/2490918007 learned_addr learned my addr 172.21.15.106:0/2490918007 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:31:06.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.291+0000 7f78777fe640 1 --2- 172.21.15.106:0/2490918007 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x558d5b9113b0 0x558d5bd15ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.291+0000 7f7877fff640 1 --2- 172.21.15.106:0/2490918007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d5b93a270 0x558d5bd938b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.291+0000 7f7876ffd640 1 -- 172.21.15.106:0/2490918007 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x558d5b9113b0 msgr2=0x558d5bd15ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:06.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.291+0000 7f7876ffd640 1 --2- 172.21.15.106:0/2490918007 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x558d5b9113b0 0x558d5bd15ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.291+0000 7f7876ffd640 1 -- 172.21.15.106:0/2490918007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d5b93a270 msgr2=0x558d5bd938b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:06.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.291+0000 7f7876ffd640 1 --2- 172.21.15.106:0/2490918007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d5b93a270 0x558d5bd938b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.291+0000 7f7876ffd640 1 -- 172.21.15.106:0/2490918007 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d5bd2d330 con 0x558d5b919590 2024-04-25T10:31:06.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.291+0000 7f7876ffd640 1 --2- 172.21.15.106:0/2490918007 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558d5b919590 0x558d5bd2cdf0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f786c07f1b0 tx=0x7f786c048070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1171a128307c3b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:06.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.291+0000 7f78567fc640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T10:31:06.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.291+0000 7f78567fc640 1 -- 172.21.15.106:0/2490918007 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f786c0b2070 con 0x558d5b919590 2024-04-25T10:31:06.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.291+0000 7f78567fc640 1 -- 172.21.15.106:0/2490918007 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f786c0882f0 con 0x558d5b919590 2024-04-25T10:31:06.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.291+0000 7f7884d6e740 1 -- 172.21.15.106:0/2490918007 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f786c083aa0 con 0x558d5b919590 2024-04-25T10:31:06.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.291+0000 7f78567fc640 1 -- 172.21.15.106:0/2490918007 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f786c07f6f0 con 0x558d5b919590 2024-04-25T10:31:06.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.291+0000 7f7884d6e740 10 client.4776 fetch_fsmap learned FSMap version 17 2024-04-25T10:31:06.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.291+0000 7f7884d6e740 1 -- 172.21.15.106:0/2490918007 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x558d5bd40b10 con 0x558d5b919590 2024-04-25T10:31:06.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.291+0000 7f78567fc640 1 -- 172.21.15.106:0/2490918007 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f786c083aa0 con 0x558d5b919590 2024-04-25T10:31:06.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.292+0000 7f78567fc640 1 -- 172.21.15.106:0/2490918007 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7f786c094b40 con 0x558d5b919590 2024-04-25T10:31:06.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.292+0000 7f7884d6e740 10 client.4776 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T10:31:06.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.292+0000 7f7884d6e740 10 client.4776 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:31:06.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.292+0000 7f7884d6e740 20 client.4776 populate_metadata read hostname 'smithi106' 2024-04-25T10:31:06.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.292+0000 7f7884d6e740 1 --2- 172.21.15.106:0/2490918007 >> [v2:172.21.15.134:6834/2662003420,v1:172.21.15.134:6837/2662003420] conn(0x558d5bd12260 0x558d5bd12670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.292+0000 7f7884d6e740 4 client.4776 mds_command: new command op to 4290 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:31:06.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.292+0000 7f7884d6e740 1 -- 172.21.15.106:0/2490918007 --> [v2:172.21.15.134:6834/2662003420,v1:172.21.15.134:6837/2662003420] -- command(tid 0: {"prefix": "status"}) -- 0x558d5bc481c0 con 0x558d5bd12260 2024-04-25T10:31:06.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.292+0000 7f78777fe640 1 --2- 172.21.15.106:0/2490918007 >> [v2:172.21.15.134:6834/2662003420,v1:172.21.15.134:6837/2662003420] conn(0x558d5bd12260 0x558d5bd12670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.292+0000 7f78567fc640 10 client.4776 ms_handle_connect on v2:172.21.15.134:6834/2662003420 2024-04-25T10:31:06.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.292+0000 7f78777fe640 1 --2- 172.21.15.106:0/2490918007 >> [v2:172.21.15.134:6834/2662003420,v1:172.21.15.134:6837/2662003420] conn(0x558d5bd12260 0x558d5bd12670 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27261fe353570c12 server_cookie=4f75325970a5b279 in_seq=0 out_seq=0 2024-04-25T10:31:06.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.292+0000 7f78567fc640 1 -- 172.21.15.106:0/2490918007 <== mds.0 v2:172.21.15.134:6834/2662003420 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x558d5bc481c0 con 0x558d5bd12260 2024-04-25T10:31:06.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.292+0000 7f78567fc640 10 client.4776 handle_command_reply: tid=0 2024-04-25T10:31:06.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.292+0000 7f7884d6e740 1 client.4776 shutdown 2024-04-25T10:31:06.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.292+0000 7f7884d6e740 1 -- 172.21.15.106:0/2490918007 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558d5b919590 msgr2=0x558d5bd2cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:06.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.292+0000 7f7884d6e740 1 --2- 172.21.15.106:0/2490918007 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558d5b919590 0x558d5bd2cdf0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f786c07f1b0 tx=0x7f786c048070 comp rx=0 tx=0).stop 2024-04-25T10:31:06.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.293+0000 7f7884d6e740 1 -- 172.21.15.106:0/2490918007 shutdown_connections 2024-04-25T10:31:06.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.293+0000 7f7884d6e740 1 --2- 172.21.15.106:0/2490918007 >> [v2:172.21.15.134:6834/2662003420,v1:172.21.15.134:6837/2662003420] conn(0x558d5bd12260 0x558d5bd12670 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.293+0000 7f7884d6e740 1 --2- 172.21.15.106:0/2490918007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d5b93a270 0x558d5bd938b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.293+0000 7f7884d6e740 1 --2- 172.21.15.106:0/2490918007 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558d5b919590 0x558d5bd2cdf0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.293+0000 7f78567fc640 0 client.4776 ms_handle_reset on v2:172.21.15.134:6834/2662003420 2024-04-25T10:31:06.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.293+0000 7f7884d6e740 1 --2- 172.21.15.106:0/2490918007 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x558d5b9113b0 0x558d5bd15ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.293+0000 7f7884d6e740 1 -- 172.21.15.106:0/2490918007 >> 172.21.15.106:0/2490918007 conn(0x558d5b92e3c0 msgr2=0x558d5bcff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:06.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.293+0000 7f7884d6e740 1 -- 172.21.15.106:0/2490918007 shutdown_connections 2024-04-25T10:31:06.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.293+0000 7f7884d6e740 1 -- 172.21.15.106:0/2490918007 wait complete. 2024-04-25T10:31:06.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.293+0000 7f7884d6e740 20 client.4776 trim_cache size 0 max 16384 2024-04-25T10:31:06.302 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-25T10:31:06.302 INFO:teuthology.orchestra.run.smithi106.stdout: "cluster_fsid": "3ee77985-e8e4-41cf-81e9-7c774cf7891f", 2024-04-25T10:31:06.302 INFO:teuthology.orchestra.run.smithi106.stdout: "whoami": 0, 2024-04-25T10:31:06.302 INFO:teuthology.orchestra.run.smithi106.stdout: "id": 4290, 2024-04-25T10:31:06.302 INFO:teuthology.orchestra.run.smithi106.stdout: "want_state": "up:active", 2024-04-25T10:31:06.303 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "up:active", 2024-04-25T10:31:06.303 INFO:teuthology.orchestra.run.smithi106.stdout: "fs_name": "cephfs", 2024-04-25T10:31:06.303 INFO:teuthology.orchestra.run.smithi106.stdout: "rank_uptime": 33.867857372000003, 2024-04-25T10:31:06.303 INFO:teuthology.orchestra.run.smithi106.stdout: "mdsmap_epoch": 17, 2024-04-25T10:31:06.303 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch": 20, 2024-04-25T10:31:06.303 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch_barrier": 20, 2024-04-25T10:31:06.303 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 40.473292493999999 2024-04-25T10:31:06.303 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-25T10:31:06.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.293+0000 7f787deb6640 1 -- 172.21.15.106:0/518113171 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f78640744e0 msgr2=0x7f78640769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:06.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.293+0000 7f787deb6640 1 --2- 172.21.15.106:0/518113171 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f78640744e0 0x7f78640769f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7868000c00 tx=0x7f7868009040 comp rx=0 tx=0).stop 2024-04-25T10:31:06.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.293+0000 7f787deb6640 1 -- 172.21.15.106:0/518113171 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f787815a810 msgr2=0x7f7878148800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:06.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.293+0000 7f787deb6640 1 --2- 172.21.15.106:0/518113171 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f787815a810 0x7f7878148800 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f786c00c330 tx=0x7f78780cc3f0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.293+0000 7f787deb6640 1 -- 172.21.15.106:0/518113171 shutdown_connections 2024-04-25T10:31:06.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.293+0000 7f787deb6640 1 --2- 172.21.15.106:0/518113171 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f78640744e0 0x7f78640769f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.293+0000 7f787deb6640 1 --2- 172.21.15.106:0/518113171 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7878148f90 0x7f7878150750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.293+0000 7f787deb6640 1 --2- 172.21.15.106:0/518113171 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f787815a810 0x7f7878148800 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.293+0000 7f787deb6640 1 --2- 172.21.15.106:0/518113171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7860004800 0x7f7878159070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.293+0000 7f787deb6640 1 -- 172.21.15.106:0/518113171 >> 172.21.15.106:0/518113171 conn(0x7f78780727d0 msgr2=0x7f787815a080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:06.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.293+0000 7f787deb6640 1 -- 172.21.15.106:0/518113171 shutdown_connections 2024-04-25T10:31:06.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.293+0000 7f787deb6640 1 -- 172.21.15.106:0/518113171 wait complete. 2024-04-25T10:31:06.306 INFO:teuthology.run_tasks:Running task sequential... 2024-04-25T10:31:06.317 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-25T10:31:06.317 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-25T10:31:06.317 INFO:teuthology.task.print:Enabling mgr modules 2024-04-25T10:31:06.317 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-04-25T10:31:06.341 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-25T10:31:06.342 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-04-25T10:31:06.342 INFO:tasks.ceph_fuse:client.0 config is {} 2024-04-25T10:31:06.342 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T10:31:06.342 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:31:06.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:31:06.342 DEBUG:teuthology.orchestra.run.smithi106:> ip netns list 2024-04-25T10:31:06.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:31:06.358 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link delete ceph-brx 2024-04-25T10:31:06.422 INFO:teuthology.orchestra.run.smithi106.stderr:Cannot find device "ceph-brx" 2024-04-25T10:31:06.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:31:06.424 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-25T10:31:06.424 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:31:06.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.569+0000 7fcb9d134640 1 Processor -- start 2024-04-25T10:31:06.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.569+0000 7fcb9d134640 1 -- start start 2024-04-25T10:31:06.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.570+0000 7fcb9d134640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fcb980c61a0 0x7fcb98156f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.570+0000 7fcb9d134640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb9815d690 0x7fcb980cce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.570+0000 7fcb9d134640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb9815cbd0 con 0x7fcb9806dc00 2024-04-25T10:31:06.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.570+0000 7fcb9d134640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fcb9815d200 con 0x7fcb980c61a0 2024-04-25T10:31:06.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.570+0000 7fcb9d134640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fcb98160990 con 0x7fcb9815d690 2024-04-25T10:31:06.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.571+0000 7fcb9759e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb9815d690 0x7fcb980cce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.571+0000 7fcb96d9d640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fcb980c61a0 0x7fcb98156f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.571+0000 7fcb8ffff640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fcb9806dc00 0x7fcb981601c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.106:54898/0 (socket says 172.21.15.106:54898) 2024-04-25T10:31:06.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.571+0000 7fcb9759e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb9815d690 0x7fcb980cce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.106:47242/0 (socket says 172.21.15.106:47242) 2024-04-25T10:31:06.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.571+0000 7fcb8ffff640 1 -- 172.21.15.106:0/319134882 learned_addr learned my addr 172.21.15.106:0/319134882 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:31:06.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.571+0000 7fcb96d9d640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fcb980c61a0 0x7fcb98156f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.106:49888/0 (socket says 172.21.15.106:49888) 2024-04-25T10:31:06.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.571+0000 7fcb9759e640 1 -- 172.21.15.106:0/319134882 >> v2:172.21.15.134:3300/0 conn(0x7fcb980c61a0 msgr2=0x7fcb98156f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:06.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.571+0000 7fcb9759e640 1 --2- 172.21.15.106:0/319134882 >> v2:172.21.15.134:3300/0 conn(0x7fcb980c61a0 0x7fcb98156f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.571+0000 7fcb9759e640 1 -- 172.21.15.106:0/319134882 >> v1:172.21.15.134:6789/0 conn(0x7fcb9806dc00 legacy=0x7fcb981601c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:06.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.571+0000 7fcb9759e640 1 -- 172.21.15.106:0/319134882 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb98160c70 con 0x7fcb9815d690 2024-04-25T10:31:06.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.572+0000 7fcb9759e640 1 --2- 172.21.15.106:0/319134882 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb9815d690 0x7fcb980cce20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcb90009d90 tx=0x7fcb9002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11464315f2a3f9bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:06.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.572+0000 7fcb9659c640 1 -- 172.21.15.106:0/319134882 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb90007740 con 0x7fcb9815d690 2024-04-25T10:31:06.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.572+0000 7fcb9659c640 1 -- 172.21.15.106:0/319134882 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb900078e0 con 0x7fcb9815d690 2024-04-25T10:31:06.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.572+0000 7fcb9659c640 1 -- 172.21.15.106:0/319134882 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb90007cb0 con 0x7fcb9815d690 2024-04-25T10:31:06.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.573+0000 7fcb9d134640 1 -- 172.21.15.106:0/319134882 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb9815d690 msgr2=0x7fcb980cce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:06.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.573+0000 7fcb9d134640 1 --2- 172.21.15.106:0/319134882 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb9815d690 0x7fcb980cce20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcb90009d90 tx=0x7fcb9002a660 comp rx=0 tx=0).stop 2024-04-25T10:31:06.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.573+0000 7fcb9d134640 1 -- 172.21.15.106:0/319134882 shutdown_connections 2024-04-25T10:31:06.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.573+0000 7fcb9d134640 1 --2- 172.21.15.106:0/319134882 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb9815d690 0x7fcb980cce20 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.573+0000 7fcb9d134640 1 --2- 172.21.15.106:0/319134882 >> v2:172.21.15.134:3300/0 conn(0x7fcb980c61a0 0x7fcb98156f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.573+0000 7fcb9d134640 1 -- 172.21.15.106:0/319134882 >> 172.21.15.106:0/319134882 conn(0x7fcb980727d0 msgr2=0x7fcb98072c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:06.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.573+0000 7fcb9d134640 1 -- 172.21.15.106:0/319134882 shutdown_connections 2024-04-25T10:31:06.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.573+0000 7fcb9d134640 1 -- 172.21.15.106:0/319134882 wait complete. 2024-04-25T10:31:06.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.574+0000 7fcb9d134640 1 Processor -- start 2024-04-25T10:31:06.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.574+0000 7fcb9d134640 1 -- start start 2024-04-25T10:31:06.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.576+0000 7fcb9d134640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb9806dc00 0x7fcb980c88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.576+0000 7fcb9d134640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb980c61a0 0x7fcb98150940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.576+0000 7fcb9d134640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb9815d690 0x7fcb98150e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.576+0000 7fcb9d134640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9815daa0 con 0x7fcb980c61a0 2024-04-25T10:31:06.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.576+0000 7fcb9d134640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9815ce90 con 0x7fcb9815d690 2024-04-25T10:31:06.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.576+0000 7fcb9d134640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fcb9806d6d0 con 0x7fcb9806dc00 2024-04-25T10:31:06.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.576+0000 7fcb8ffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb980c61a0 0x7fcb98150940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.576+0000 7fcb8ffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb980c61a0 0x7fcb98150940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59086/0 (socket says 172.21.15.106:59086) 2024-04-25T10:31:06.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.576+0000 7fcb8ffff640 1 -- 172.21.15.106:0/1959001961 learned_addr learned my addr 172.21.15.106:0/1959001961 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:31:06.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.576+0000 7fcb96d9d640 1 --2- 172.21.15.106:0/1959001961 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb9806dc00 0x7fcb980c88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.576+0000 7fcb9759e640 1 --2- 172.21.15.106:0/1959001961 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb9815d690 0x7fcb98150e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.576+0000 7fcb8ffff640 1 -- 172.21.15.106:0/1959001961 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb9806dc00 msgr2=0x7fcb980c88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:06.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.576+0000 7fcb8ffff640 1 --2- 172.21.15.106:0/1959001961 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb9806dc00 0x7fcb980c88e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.576+0000 7fcb8ffff640 1 -- 172.21.15.106:0/1959001961 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb9815d690 msgr2=0x7fcb98150e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:06.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.576+0000 7fcb8ffff640 1 --2- 172.21.15.106:0/1959001961 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb9815d690 0x7fcb98150e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.576+0000 7fcb8ffff640 1 -- 172.21.15.106:0/1959001961 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb980c8e20 con 0x7fcb980c61a0 2024-04-25T10:31:06.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.576+0000 7fcb8ffff640 1 --2- 172.21.15.106:0/1959001961 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb980c61a0 0x7fcb98150940 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fcb8000c3f0 tx=0x7fcb8000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:06.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.576+0000 7fcb8f7fe640 1 -- 172.21.15.106:0/1959001961 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb80017070 con 0x7fcb980c61a0 2024-04-25T10:31:06.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.576+0000 7fcb8f7fe640 1 -- 172.21.15.106:0/1959001961 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb80005ce0 con 0x7fcb980c61a0 2024-04-25T10:31:06.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.576+0000 7fcb8f7fe640 1 -- 172.21.15.106:0/1959001961 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb800072f0 con 0x7fcb980c61a0 2024-04-25T10:31:06.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.576+0000 7fcb9d134640 1 -- 172.21.15.106:0/1959001961 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb980c7e30 con 0x7fcb980c61a0 2024-04-25T10:31:06.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.577+0000 7fcb9d134640 1 -- 172.21.15.106:0/1959001961 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb981f6d40 con 0x7fcb980c61a0 2024-04-25T10:31:06.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.578+0000 7fcb9d134640 1 -- 172.21.15.106:0/1959001961 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb981f6b50 con 0x7fcb980c61a0 2024-04-25T10:31:06.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.578+0000 7fcb8f7fe640 1 -- 172.21.15.106:0/1959001961 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fcb8001a070 con 0x7fcb980c61a0 2024-04-25T10:31:06.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.579+0000 7fcb8f7fe640 1 --2- 172.21.15.106:0/1959001961 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fcb780744b0 0x7fcb780769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.579+0000 7fcb8f7fe640 1 -- 172.21.15.106:0/1959001961 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fcb80094510 con 0x7fcb980c61a0 2024-04-25T10:31:06.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.579+0000 7fcb96d9d640 1 --2- 172.21.15.106:0/1959001961 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fcb780744b0 0x7fcb780769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.579+0000 7fcb96d9d640 1 --2- 172.21.15.106:0/1959001961 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fcb780744b0 0x7fcb780769c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcb88009aa0 tx=0x7fcb88009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:06.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.583+0000 7fcb8f7fe640 1 -- 172.21.15.106:0/1959001961 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcb8005e650 con 0x7fcb980c61a0 2024-04-25T10:31:06.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.798+0000 7fcb9d134640 1 -- 172.21.15.106:0/1959001961 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fcb9806a3d0 con 0x7fcb980c61a0 2024-04-25T10:31:06.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.799+0000 7fcb8f7fe640 1 -- 172.21.15.106:0/1959001961 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7fcb80062950 con 0x7fcb980c61a0 2024-04-25T10:31:06.801 INFO:teuthology.orchestra.run.smithi106.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:31:06.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.800+0000 7fcb9d134640 1 -- 172.21.15.106:0/1959001961 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fcb780744b0 msgr2=0x7fcb780769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:06.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.800+0000 7fcb9d134640 1 --2- 172.21.15.106:0/1959001961 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fcb780744b0 0x7fcb780769c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcb88009aa0 tx=0x7fcb88009190 comp rx=0 tx=0).stop 2024-04-25T10:31:06.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.800+0000 7fcb9d134640 1 -- 172.21.15.106:0/1959001961 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb980c61a0 msgr2=0x7fcb98150940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:06.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.800+0000 7fcb9d134640 1 --2- 172.21.15.106:0/1959001961 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb980c61a0 0x7fcb98150940 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fcb8000c3f0 tx=0x7fcb8000c8b0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.801+0000 7fcb9d134640 1 -- 172.21.15.106:0/1959001961 shutdown_connections 2024-04-25T10:31:06.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.801+0000 7fcb9d134640 1 --2- 172.21.15.106:0/1959001961 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fcb780744b0 0x7fcb780769c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.801+0000 7fcb9d134640 1 --2- 172.21.15.106:0/1959001961 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb9815d690 0x7fcb98150e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.801+0000 7fcb9d134640 1 --2- 172.21.15.106:0/1959001961 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb980c61a0 0x7fcb98150940 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.801+0000 7fcb9d134640 1 --2- 172.21.15.106:0/1959001961 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb9806dc00 0x7fcb980c88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.801+0000 7fcb9d134640 1 -- 172.21.15.106:0/1959001961 >> 172.21.15.106:0/1959001961 conn(0x7fcb980727d0 msgr2=0x7fcb9815c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:06.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.801+0000 7fcb9d134640 1 -- 172.21.15.106:0/1959001961 shutdown_connections 2024-04-25T10:31:06.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06.801+0000 7fcb9d134640 1 -- 172.21.15.106:0/1959001961 wait complete. 2024-04-25T10:31:06.814 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:31:06.814 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:31:06.814 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi106.front.sepia.ceph.com 2024-04-25T10:31:06.814 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:31:06.814 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:31:06.814 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:31:06.814 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:31:06.815 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:31:06.815 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:31:06.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:31:06.815 DEBUG:teuthology.orchestra.run.smithi106:> ip addr 2024-04-25T10:31:06.829 INFO:teuthology.orchestra.run.smithi106.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:31:06.830 INFO:teuthology.orchestra.run.smithi106.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:31:06.830 INFO:teuthology.orchestra.run.smithi106.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:31:06.830 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:31:06.830 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 ::1/128 scope host 2024-04-25T10:31:06.830 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:31:06.830 INFO:teuthology.orchestra.run.smithi106.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:31:06.830 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2a brd ff:ff:ff:ff:ff:ff 2024-04-25T10:31:06.830 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f0 2024-04-25T10:31:06.830 INFO:teuthology.orchestra.run.smithi106.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:31:06.830 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2b brd ff:ff:ff:ff:ff:ff 2024-04-25T10:31:06.831 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f1 2024-04-25T10:31:06.831 INFO:teuthology.orchestra.run.smithi106.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:31:06.831 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6c brd ff:ff:ff:ff:ff:ff 2024-04-25T10:31:06.831 INFO:teuthology.orchestra.run.smithi106.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:31:06.831 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6d brd ff:ff:ff:ff:ff:ff 2024-04-25T10:31:06.831 INFO:teuthology.orchestra.run.smithi106.stdout: inet 172.21.15.106/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T10:31:06.831 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft 42737sec preferred_lft 42737sec 2024-04-25T10:31:06.831 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::d124:11fc:35b9:5828/64 scope link noprefixroute 2024-04-25T10:31:06.831 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:31:06.831 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T10:31:06.832 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T10:31:06.832 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2024-04-25T10:31:06.832 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link add name ceph-brx type bridge 2024-04-25T10:31:06.832 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip addr flush dev ceph-brx 2024-04-25T10:31:06.832 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link set ceph-brx up 2024-04-25T10:31:06.832 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T10:31:06.832 DEBUG:teuthology.orchestra.run.smithi106:> ') 2024-04-25T10:31:06.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T10:31:06.997 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:31:07.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:31:07.004 DEBUG:teuthology.orchestra.run.smithi106:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:31:07.027 INFO:teuthology.orchestra.run.smithi106.stdout:1 2024-04-25T10:31:07.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:31:07.029 DEBUG:teuthology.orchestra.run.smithi106:> route 2024-04-25T10:31:07.099 INFO:teuthology.orchestra.run.smithi106.stdout:Kernel IP routing table 2024-04-25T10:31:07.100 INFO:teuthology.orchestra.run.smithi106.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:31:07.100 INFO:teuthology.orchestra.run.smithi106.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:31:07.100 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:31:07.100 INFO:teuthology.orchestra.run.smithi106.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:31:07.101 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T10:31:07.101 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2024-04-25T10:31:07.101 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:31:07.101 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:31:07.101 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:31:07.101 DEBUG:teuthology.orchestra.run.smithi106:> ') 2024-04-25T10:31:07.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T10:31:07.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:31:07.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:31:07.397 DEBUG:teuthology.orchestra.run.smithi106:> ip netns list 2024-04-25T10:31:07.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:31:07.412 DEBUG:teuthology.orchestra.run.smithi106:> ip netns list-id 2024-04-25T10:31:07.467 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T10:31:07.467 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2024-04-25T10:31:07.467 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:31:07.467 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T10:31:07.467 DEBUG:teuthology.orchestra.run.smithi106:> ') 2024-04-25T10:31:07.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T10:31:07.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:31:07.585 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T10:31:07.585 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T10:31:07.585 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2024-04-25T10:31:07.585 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T10:31:07.585 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T10:31:07.585 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T10:31:07.585 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T10:31:07.586 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T10:31:07.586 DEBUG:teuthology.orchestra.run.smithi106:> ') 2024-04-25T10:31:07.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T10:31:07.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:31:07.708 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T10:31:07.709 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2024-04-25T10:31:07.709 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link set brx.0 up 2024-04-25T10:31:07.709 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T10:31:07.709 DEBUG:teuthology.orchestra.run.smithi106:> ') 2024-04-25T10:31:07.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T10:31:07.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:31:07.783 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T10:31:07.783 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:31:07.783 DEBUG:teuthology.orchestra.run.smithi106:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:31:07.798 INFO:teuthology.orchestra.run.smithi106.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:31:07.798 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:31:07.798 DEBUG:teuthology.orchestra.run.smithi106:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:31:07.852 DEBUG:teuthology.orchestra.run.smithi106:> sudo modprobe fuse 2024-04-25T10:31:07.917 DEBUG:teuthology.orchestra.run.smithi106:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:31:07.973 INFO:teuthology.orchestra.run.smithi106.stdout:/proc 2024-04-25T10:31:07.973 INFO:teuthology.orchestra.run.smithi106.stdout:/sys 2024-04-25T10:31:07.974 INFO:teuthology.orchestra.run.smithi106.stdout:/dev 2024-04-25T10:31:07.974 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/security 2024-04-25T10:31:07.974 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/shm 2024-04-25T10:31:07.974 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/pts 2024-04-25T10:31:07.974 INFO:teuthology.orchestra.run.smithi106.stdout:/run 2024-04-25T10:31:07.974 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/cgroup 2024-04-25T10:31:07.974 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/pstore 2024-04-25T10:31:07.974 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/bpf 2024-04-25T10:31:07.974 INFO:teuthology.orchestra.run.smithi106.stdout:/ 2024-04-25T10:31:07.974 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/selinux 2024-04-25T10:31:07.974 INFO:teuthology.orchestra.run.smithi106.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:31:07.975 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/hugepages 2024-04-25T10:31:07.975 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/mqueue 2024-04-25T10:31:07.975 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/debug 2024-04-25T10:31:07.975 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/tracing 2024-04-25T10:31:07.975 INFO:teuthology.orchestra.run.smithi106.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:31:07.975 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/config 2024-04-25T10:31:07.975 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/fuse/connections 2024-04-25T10:31:07.975 INFO:teuthology.orchestra.run.smithi106.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T10:31:07.975 INFO:teuthology.orchestra.run.smithi106.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T10:31:07.975 INFO:teuthology.orchestra.run.smithi106.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T10:31:07.975 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T10:31:07.975 INFO:teuthology.orchestra.run.smithi106.stdout:/run/user/1000 2024-04-25T10:31:07.975 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph 2024-04-25T10:31:07.976 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:31:07.976 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:31:07.976 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:31:07.976 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:31:07.976 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns 2024-04-25T10:31:07.976 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:31:07.976 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:31:07.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:31:07.976 DEBUG:teuthology.orchestra.run.smithi106:> ls /sys/fs/fuse/connections 2024-04-25T10:31:08.028 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T10:31:08.028 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T10:31:08.072 DEBUG:teuthology.orchestra.run.smithi106:> sudo modprobe fuse 2024-04-25T10:31:08.095 DEBUG:teuthology.orchestra.run.smithi106:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:31:08.151 INFO:teuthology.orchestra.run.smithi106.stdout:/proc 2024-04-25T10:31:08.151 INFO:teuthology.orchestra.run.smithi106.stdout:/sys 2024-04-25T10:31:08.151 INFO:teuthology.orchestra.run.smithi106.stdout:/dev 2024-04-25T10:31:08.151 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/security 2024-04-25T10:31:08.151 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/shm 2024-04-25T10:31:08.152 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/pts 2024-04-25T10:31:08.152 INFO:teuthology.orchestra.run.smithi106.stdout:/run 2024-04-25T10:31:08.152 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/cgroup 2024-04-25T10:31:08.152 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/pstore 2024-04-25T10:31:08.152 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/bpf 2024-04-25T10:31:08.152 INFO:teuthology.orchestra.run.smithi106.stdout:/ 2024-04-25T10:31:08.152 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/selinux 2024-04-25T10:31:08.152 INFO:teuthology.orchestra.run.smithi106.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:31:08.152 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/hugepages 2024-04-25T10:31:08.152 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/mqueue 2024-04-25T10:31:08.152 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/debug 2024-04-25T10:31:08.152 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/tracing 2024-04-25T10:31:08.153 INFO:teuthology.orchestra.run.smithi106.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:31:08.153 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/config 2024-04-25T10:31:08.153 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/fuse/connections 2024-04-25T10:31:08.153 INFO:teuthology.orchestra.run.smithi106.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T10:31:08.153 INFO:teuthology.orchestra.run.smithi106.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T10:31:08.153 INFO:teuthology.orchestra.run.smithi106.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T10:31:08.153 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T10:31:08.153 INFO:teuthology.orchestra.run.smithi106.stdout:/run/user/1000 2024-04-25T10:31:08.153 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph 2024-04-25T10:31:08.153 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:31:08.153 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:31:08.153 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:31:08.154 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:31:08.154 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns 2024-04-25T10:31:08.154 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:31:08.154 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:31:08.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:31:08.154 DEBUG:teuthology.orchestra.run.smithi106:> ls /sys/fs/fuse/connections 2024-04-25T10:31:08.180 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi106.stderr:2024-04-25T10:31:08.178+0000 7f75ccf5e480 -1 init, newargv = 0x7f75b0001230 newargc=15 2024-04-25T10:31:08.181 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi106.stderr:ceph-fuse[43955]: starting ceph client 2024-04-25T10:31:08.196 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi106.stderr:ceph-fuse[43955]: starting fuse 2024-04-25T10:31:09.296 DEBUG:teuthology.orchestra.run.smithi106:> sudo modprobe fuse 2024-04-25T10:31:09.324 DEBUG:teuthology.orchestra.run.smithi106:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:31:09.381 INFO:teuthology.orchestra.run.smithi106.stdout:/proc 2024-04-25T10:31:09.381 INFO:teuthology.orchestra.run.smithi106.stdout:/sys 2024-04-25T10:31:09.381 INFO:teuthology.orchestra.run.smithi106.stdout:/dev 2024-04-25T10:31:09.381 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/security 2024-04-25T10:31:09.381 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/shm 2024-04-25T10:31:09.381 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/pts 2024-04-25T10:31:09.381 INFO:teuthology.orchestra.run.smithi106.stdout:/run 2024-04-25T10:31:09.381 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/cgroup 2024-04-25T10:31:09.382 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/pstore 2024-04-25T10:31:09.382 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/bpf 2024-04-25T10:31:09.382 INFO:teuthology.orchestra.run.smithi106.stdout:/ 2024-04-25T10:31:09.382 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/selinux 2024-04-25T10:31:09.382 INFO:teuthology.orchestra.run.smithi106.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:31:09.382 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/hugepages 2024-04-25T10:31:09.382 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/mqueue 2024-04-25T10:31:09.382 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/debug 2024-04-25T10:31:09.382 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/tracing 2024-04-25T10:31:09.382 INFO:teuthology.orchestra.run.smithi106.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:31:09.382 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/config 2024-04-25T10:31:09.383 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/fuse/connections 2024-04-25T10:31:09.383 INFO:teuthology.orchestra.run.smithi106.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T10:31:09.383 INFO:teuthology.orchestra.run.smithi106.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T10:31:09.383 INFO:teuthology.orchestra.run.smithi106.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T10:31:09.383 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T10:31:09.383 INFO:teuthology.orchestra.run.smithi106.stdout:/run/user/1000 2024-04-25T10:31:09.383 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph 2024-04-25T10:31:09.383 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:31:09.383 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:31:09.383 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:31:09.383 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:31:09.383 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns 2024-04-25T10:31:09.383 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:31:09.383 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:31:09.383 INFO:teuthology.orchestra.run.smithi106.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T10:31:09.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:31:09.384 DEBUG:teuthology.orchestra.run.smithi106:> ls /sys/fs/fuse/connections 2024-04-25T10:31:09.436 INFO:teuthology.orchestra.run.smithi106.stdout:43 2024-04-25T10:31:09.436 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-04-25T10:31:09.437 DEBUG:teuthology.orchestra.run.smithi106:> sudo stdin-killer -- python3 -c ' 2024-04-25T10:31:09.437 DEBUG:teuthology.orchestra.run.smithi106:> import glob 2024-04-25T10:31:09.437 DEBUG:teuthology.orchestra.run.smithi106:> import re 2024-04-25T10:31:09.437 DEBUG:teuthology.orchestra.run.smithi106:> import os 2024-04-25T10:31:09.437 DEBUG:teuthology.orchestra.run.smithi106:> import subprocess 2024-04-25T10:31:09.437 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-25T10:31:09.437 DEBUG:teuthology.orchestra.run.smithi106:> def _find_admin_socket(client_name): 2024-04-25T10:31:09.437 DEBUG:teuthology.orchestra.run.smithi106:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T10:31:09.438 DEBUG:teuthology.orchestra.run.smithi106:> files = glob.glob(asok_path) 2024-04-25T10:31:09.438 DEBUG:teuthology.orchestra.run.smithi106:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T10:31:09.438 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-25T10:31:09.438 DEBUG:teuthology.orchestra.run.smithi106:> # Given a non-glob path, it better be there 2024-04-25T10:31:09.438 DEBUG:teuthology.orchestra.run.smithi106:> if "*" not in asok_path: 2024-04-25T10:31:09.439 DEBUG:teuthology.orchestra.run.smithi106:> assert(len(files) == 1) 2024-04-25T10:31:09.439 DEBUG:teuthology.orchestra.run.smithi106:> return files[0] 2024-04-25T10:31:09.439 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-25T10:31:09.439 DEBUG:teuthology.orchestra.run.smithi106:> for f in files: 2024-04-25T10:31:09.439 DEBUG:teuthology.orchestra.run.smithi106:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T10:31:09.439 DEBUG:teuthology.orchestra.run.smithi106:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T10:31:09.439 DEBUG:teuthology.orchestra.run.smithi106:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T10:31:09.439 DEBUG:teuthology.orchestra.run.smithi106:> contents = proc_f.read() 2024-04-25T10:31:09.439 DEBUG:teuthology.orchestra.run.smithi106:> if mountpoint in contents: 2024-04-25T10:31:09.439 DEBUG:teuthology.orchestra.run.smithi106:> return f 2024-04-25T10:31:09.439 DEBUG:teuthology.orchestra.run.smithi106:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T10:31:09.439 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-25T10:31:09.439 DEBUG:teuthology.orchestra.run.smithi106:> print(_find_admin_socket("client.0")) 2024-04-25T10:31:09.440 DEBUG:teuthology.orchestra.run.smithi106:> ' 2024-04-25T10:31:09.554 INFO:teuthology.orchestra.run.smithi106.stdout:/var/run/ceph/ceph-client.0.43955.asok 2024-04-25T10:31:09.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:31:09.564 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.43955.asok 2024-04-25T10:31:09.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:31:09.564 DEBUG:teuthology.orchestra.run.smithi106:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.43955.asok status 2024-04-25T10:31:09.704 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-25T10:31:09.704 INFO:teuthology.orchestra.run.smithi106.stdout: "metadata": { 2024-04-25T10:31:09.705 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_sha1": "994e2f20102a20fa9a27966c14b625cc01884df8", 2024-04-25T10:31:09.705 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T10:31:09.705 INFO:teuthology.orchestra.run.smithi106.stdout: "entity_id": "0", 2024-04-25T10:31:09.705 INFO:teuthology.orchestra.run.smithi106.stdout: "hostname": "smithi106", 2024-04-25T10:31:09.705 INFO:teuthology.orchestra.run.smithi106.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T10:31:09.705 INFO:teuthology.orchestra.run.smithi106.stdout: "pid": "43955", 2024-04-25T10:31:09.706 INFO:teuthology.orchestra.run.smithi106.stdout: "root": "/" 2024-04-25T10:31:09.706 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:09.706 INFO:teuthology.orchestra.run.smithi106.stdout: "dentry_count": 0, 2024-04-25T10:31:09.706 INFO:teuthology.orchestra.run.smithi106.stdout: "dentry_pinned_count": 0, 2024-04-25T10:31:09.706 INFO:teuthology.orchestra.run.smithi106.stdout: "id": 4785, 2024-04-25T10:31:09.706 INFO:teuthology.orchestra.run.smithi106.stdout: "inst": { 2024-04-25T10:31:09.707 INFO:teuthology.orchestra.run.smithi106.stdout: "name": { 2024-04-25T10:31:09.707 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "client", 2024-04-25T10:31:09.707 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 4785 2024-04-25T10:31:09.707 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:09.707 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": { 2024-04-25T10:31:09.707 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:31:09.708 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:31:09.708 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 1852887618 2024-04-25T10:31:09.708 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:31:09.708 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:09.709 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": { 2024-04-25T10:31:09.709 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:31:09.709 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:31:09.709 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 1852887618 2024-04-25T10:31:09.709 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:09.709 INFO:teuthology.orchestra.run.smithi106.stdout: "inst_str": "client.4785 192.168.0.1:0/1852887618", 2024-04-25T10:31:09.710 INFO:teuthology.orchestra.run.smithi106.stdout: "addr_str": "192.168.0.1:0/1852887618", 2024-04-25T10:31:09.710 INFO:teuthology.orchestra.run.smithi106.stdout: "inode_count": 1, 2024-04-25T10:31:09.710 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_epoch": 17, 2024-04-25T10:31:09.710 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_epoch": 20, 2024-04-25T10:31:09.711 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_epoch_barrier": 0, 2024-04-25T10:31:09.711 INFO:teuthology.orchestra.run.smithi106.stdout: "blocklisted": false, 2024-04-25T10:31:09.711 INFO:teuthology.orchestra.run.smithi106.stdout: "fs_name": "cephfs" 2024-04-25T10:31:09.712 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-25T10:31:09.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:31:09.714 DEBUG:teuthology.orchestra.run.smithi106:> stat --file-system '--printf=%T 2024-04-25T10:31:09.714 DEBUG:teuthology.orchestra.run.smithi106:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T10:31:09.731 INFO:teuthology.orchestra.run.smithi106.stdout:fuseblk 2024-04-25T10:31:09.731 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T10:31:09.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:31:09.732 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:31:09.795 INFO:teuthology.run_tasks:Running task mon_thrash... 2024-04-25T10:31:09.807 INFO:tasks.mon_thrash:Beginning mon_thrash... 2024-04-25T10:31:09.807 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:31:09.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:09.942+0000 7f8ed1ee9640 1 Processor -- start 2024-04-25T10:31:09.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:09.942+0000 7f8ed1ee9640 1 -- start start 2024-04-25T10:31:09.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:09.943+0000 7f8ed1ee9640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f8ecc0c8d90 0x7f8ecc156ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:09.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:09.943+0000 7f8ed1ee9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ecc0b4ce0 0x7f8ecc15ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:09.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:09.943+0000 7f8ed1ee9640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ecc0ce630 con 0x7f8ecc0c7e30 2024-04-25T10:31:09.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:09.943+0000 7f8ed1ee9640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f8ecc0b50f0 con 0x7f8ecc0c8d90 2024-04-25T10:31:09.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:09.943+0000 7f8ed1ee9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f8ecc0c7c60 con 0x7f8ecc0b4ce0 2024-04-25T10:31:09.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:09.943+0000 7f8ecb7fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f8ecc0c7e30 0x7f8ecc06d000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:46668/0 (socket says 172.21.15.106:46668) 2024-04-25T10:31:09.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:09.943+0000 7f8ecb7fe640 1 -- 172.21.15.106:0/2448256749 learned_addr learned my addr 172.21.15.106:0/2448256749 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:31:09.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:09.943+0000 7f8ecaffd640 1 --2- 172.21.15.106:0/2448256749 >> v2:172.21.15.134:3300/0 conn(0x7f8ecc0c8d90 0x7f8ecc156ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:09.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:09.943+0000 7f8ecbfff640 1 --2- 172.21.15.106:0/2448256749 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ecc0b4ce0 0x7f8ecc15ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:09.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:09.943+0000 7f8eca7fc640 1 -- 172.21.15.106:0/2448256749 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 747138209 0 0) 0x7f8ecc0ce630 con 0x7f8ecc0c7e30 2024-04-25T10:31:09.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:09.943+0000 7f8eca7fc640 1 -- 172.21.15.106:0/2448256749 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8eb4003520 con 0x7f8ecc0c7e30 2024-04-25T10:31:09.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:09.944+0000 7f8ecaffd640 1 -- 172.21.15.106:0/2448256749 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ecc0b4ce0 msgr2=0x7f8ecc15ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:09.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:09.944+0000 7f8ecaffd640 1 --2- 172.21.15.106:0/2448256749 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ecc0b4ce0 0x7f8ecc15ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:09.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:09.944+0000 7f8ecaffd640 1 -- 172.21.15.106:0/2448256749 >> v1:172.21.15.106:6789/0 conn(0x7f8ecc0c7e30 legacy=0x7f8ecc06d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:09.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:09.944+0000 7f8ecaffd640 1 -- 172.21.15.106:0/2448256749 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ecc0c7980 con 0x7f8ecc0c8d90 2024-04-25T10:31:09.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:09.944+0000 7f8ecbfff640 1 --2- 172.21.15.106:0/2448256749 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ecc0b4ce0 0x7f8ecc15ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:31:09.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:09.944+0000 7f8ecaffd640 1 --2- 172.21.15.106:0/2448256749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ecc0c8d90 0x7f8ecc156ef0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0009de0 tx=0x7f8ec002a630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df4233646972be0c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:09.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:09.945+0000 7f8eca7fc640 1 -- 172.21.15.106:0/2448256749 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec0007740 con 0x7f8ecc0c8d90 2024-04-25T10:31:09.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:09.945+0000 7f8eca7fc640 1 -- 172.21.15.106:0/2448256749 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ec002ae50 con 0x7f8ecc0c8d90 2024-04-25T10:31:09.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:09.945+0000 7f8eca7fc640 1 -- 172.21.15.106:0/2448256749 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec0007ae0 con 0x7f8ecc0c8d90 2024-04-25T10:31:09.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:09.945+0000 7f8ed1ee9640 1 -- 172.21.15.106:0/2448256749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ecc0c8d90 msgr2=0x7f8ecc156ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:09.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:09.945+0000 7f8ed1ee9640 1 --2- 172.21.15.106:0/2448256749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ecc0c8d90 0x7f8ecc156ef0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0009de0 tx=0x7f8ec002a630 comp rx=0 tx=0).stop 2024-04-25T10:31:09.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:09.945+0000 7f8ed1ee9640 1 -- 172.21.15.106:0/2448256749 shutdown_connections 2024-04-25T10:31:09.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:09.945+0000 7f8ed1ee9640 1 --2- 172.21.15.106:0/2448256749 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ecc0b4ce0 0x7f8ecc15ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:09.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:09.945+0000 7f8ed1ee9640 1 --2- 172.21.15.106:0/2448256749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ecc0c8d90 0x7f8ecc156ef0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:09.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:09.945+0000 7f8ed1ee9640 1 -- 172.21.15.106:0/2448256749 >> 172.21.15.106:0/2448256749 conn(0x7f8ecc068b10 msgr2=0x7f8ecc0d1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:09.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:09.945+0000 7f8ed1ee9640 1 -- 172.21.15.106:0/2448256749 shutdown_connections 2024-04-25T10:31:09.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:09.945+0000 7f8ed1ee9640 1 -- 172.21.15.106:0/2448256749 wait complete. 2024-04-25T10:31:09.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:09.947+0000 7f8ed1ee9640 1 Processor -- start 2024-04-25T10:31:09.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:09.947+0000 7f8ed1ee9640 1 -- start start 2024-04-25T10:31:09.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:09.948+0000 7f8ed1ee9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ecc0b4ce0 0x7f8ecc14e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:09.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:09.948+0000 7f8ed1ee9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ecc0c7e30 0x7f8ecc1f71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:09.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:09.948+0000 7f8ed1ee9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8ecc0c8d90 0x7f8ecc1e8420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:09.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:09.948+0000 7f8ed1ee9640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8ecc06d710 con 0x7f8ecc0c8d90 2024-04-25T10:31:09.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:09.948+0000 7f8ed1ee9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8ecc0c72f0 con 0x7f8ecc0b4ce0 2024-04-25T10:31:09.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:09.948+0000 7f8ed1ee9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f8ecc06d890 con 0x7f8ecc0c7e30 2024-04-25T10:31:09.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:09.949+0000 7f8ecbfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8ecc0c8d90 0x7f8ecc1e8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:09.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:09.949+0000 7f8ecbfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8ecc0c8d90 0x7f8ecc1e8420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59094/0 (socket says 172.21.15.106:59094) 2024-04-25T10:31:09.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:09.949+0000 7f8ecbfff640 1 -- 172.21.15.106:0/2861846349 learned_addr learned my addr 172.21.15.106:0/2861846349 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:31:09.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:09.949+0000 7f8ecb7fe640 1 --2- 172.21.15.106:0/2861846349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ecc0b4ce0 0x7f8ecc14e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:09.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:09.949+0000 7f8ecaffd640 1 --2- 172.21.15.106:0/2861846349 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ecc0c7e30 0x7f8ecc1f71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:09.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:09.949+0000 7f8ecbfff640 1 -- 172.21.15.106:0/2861846349 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ecc0c7e30 msgr2=0x7f8ecc1f71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:09.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:09.949+0000 7f8ecbfff640 1 --2- 172.21.15.106:0/2861846349 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ecc0c7e30 0x7f8ecc1f71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:09.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:09.949+0000 7f8ecbfff640 1 -- 172.21.15.106:0/2861846349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ecc0b4ce0 msgr2=0x7f8ecc14e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:09.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:09.949+0000 7f8ecbfff640 1 --2- 172.21.15.106:0/2861846349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ecc0b4ce0 0x7f8ecc14e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:09.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:09.949+0000 7f8ecbfff640 1 -- 172.21.15.106:0/2861846349 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ecc1f3260 con 0x7f8ecc0c8d90 2024-04-25T10:31:09.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:09.949+0000 7f8ecaffd640 1 --2- 172.21.15.106:0/2861846349 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ecc0c7e30 0x7f8ecc1f71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:31:09.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:09.949+0000 7f8ecbfff640 1 --2- 172.21.15.106:0/2861846349 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8ecc0c8d90 0x7f8ecc1e8420 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8ec400e850 tx=0x7f8ec400ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:09.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:09.949+0000 7f8eb3fff640 1 -- 172.21.15.106:0/2861846349 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec4017070 con 0x7f8ecc0c8d90 2024-04-25T10:31:09.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:09.949+0000 7f8ed1ee9640 1 -- 172.21.15.106:0/2861846349 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ecc1f3520 con 0x7f8ecc0c8d90 2024-04-25T10:31:09.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:09.949+0000 7f8eb3fff640 1 -- 172.21.15.106:0/2861846349 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ec400c940 con 0x7f8ecc0c8d90 2024-04-25T10:31:09.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:09.949+0000 7f8eb3fff640 1 -- 172.21.15.106:0/2861846349 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec400cd10 con 0x7f8ecc0c8d90 2024-04-25T10:31:09.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:09.949+0000 7f8ed1ee9640 1 -- 172.21.15.106:0/2861846349 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ecc1f3ae0 con 0x7f8ecc0c8d90 2024-04-25T10:31:09.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:09.950+0000 7f8ed1ee9640 1 -- 172.21.15.106:0/2861846349 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ecc06a3d0 con 0x7f8ecc0c8d90 2024-04-25T10:31:09.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:09.951+0000 7f8eb3fff640 1 -- 172.21.15.106:0/2861846349 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8ec401a070 con 0x7f8ecc0c8d90 2024-04-25T10:31:09.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:09.951+0000 7f8eb3fff640 1 --2- 172.21.15.106:0/2861846349 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f8eb80744b0 0x7f8eb80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:09.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:09.952+0000 7f8ecb7fe640 1 --2- 172.21.15.106:0/2861846349 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f8eb80744b0 0x7f8eb80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:09.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:09.952+0000 7f8eb3fff640 1 -- 172.21.15.106:0/2861846349 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8ec4095460 con 0x7f8ecc0c8d90 2024-04-25T10:31:09.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:09.952+0000 7f8ecb7fe640 1 --2- 172.21.15.106:0/2861846349 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f8eb80744b0 0x7f8eb80769c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc007fd0 tx=0x7f8ebc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:09.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:09.956+0000 7f8eb3fff640 1 -- 172.21.15.106:0/2861846349 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8ec405fc00 con 0x7f8ecc0c8d90 2024-04-25T10:31:10.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.134+0000 7f8ed1ee9640 1 -- 172.21.15.106:0/2861846349 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8ecc06e200 con 0x7f8ecc0c8d90 2024-04-25T10:31:10.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.135+0000 7f8eb3fff640 1 -- 172.21.15.106:0/2861846349 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16641 (secure 0 0 0) 0x7f8ec405f500 con 0x7f8ecc0c8d90 2024-04-25T10:31:10.138 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:31:10.139 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":20,"fsid":"3ee77985-e8e4-41cf-81e9-7c774cf7891f","created":"2024-04-25T10:30:07.981054+0000","modified":"2024-04-25T10:30:33.409267+0000","last_up_change":"2024-04-25T10:30:19.184325+0000","last_in_change":"2024-04-25T10:30:10.217553+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:30:20.111046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:30:30.314914+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:30:30.715500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a5962cb1-dbbd-47b1-8d15-9bae114bfdf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1240717079},{"type":"v1","addr":"172.21.15.106:6801","nonce":1240717079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1240717079},{"type":"v1","addr":"172.21.15.106:6803","nonce":1240717079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1240717079},{"type":"v1","addr":"172.21.15.106:6807","nonce":1240717079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1240717079},{"type":"v1","addr":"172.21.15.106:6805","nonce":1240717079}]},"public_addr":"172.21.15.106:6801/1240717079","cluster_addr":"172.21.15.106:6803/1240717079","heartbeat_back_addr":"172.21.15.106:6807/1240717079","heartbeat_front_addr":"172.21.15.106:6805/1240717079","state":["exists","up"]},{"osd":1,"uuid":"94436e13-4769-4752-8397-b5e396a6f7ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4089653051},{"type":"v1","addr":"172.21.15.106:6825","nonce":4089653051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4089653051},{"type":"v1","addr":"172.21.15.106:6827","nonce":4089653051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4089653051},{"type":"v1","addr":"172.21.15.106:6831","nonce":4089653051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4089653051},{"type":"v1","addr":"172.21.15.106:6829","nonce":4089653051}]},"public_addr":"172.21.15.106:6825/4089653051","cluster_addr":"172.21.15.106:6827/4089653051","heartbeat_back_addr":"172.21.15.106:6831/4089653051","heartbeat_front_addr":"172.21.15.106:6829/4089653051","state":["exists","up"]},{"osd":2,"uuid":"429aae0b-70c6-4e7b-9043-6a11334c7efc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":1441018239},{"type":"v1","addr":"172.21.15.106:6809","nonce":1441018239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1441018239},{"type":"v1","addr":"172.21.15.106:6811","nonce":1441018239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1441018239},{"type":"v1","addr":"172.21.15.106:6815","nonce":1441018239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":1441018239},{"type":"v1","addr":"172.21.15.106:6813","nonce":1441018239}]},"public_addr":"172.21.15.106:6809/1441018239","cluster_addr":"172.21.15.106:6811/1441018239","heartbeat_back_addr":"172.21.15.106:6815/1441018239","heartbeat_front_addr":"172.21.15.106:6813/1441018239","state":["exists","up"]},{"osd":3,"uuid":"9593a4d1-772f-4667-ab0d-2982f841e342","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":1846968711},{"type":"v1","addr":"172.21.15.106:6817","nonce":1846968711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1846968711},{"type":"v1","addr":"172.21.15.106:6819","nonce":1846968711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":1846968711},{"type":"v1","addr":"172.21.15.106:6823","nonce":1846968711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":1846968711},{"type":"v1","addr":"172.21.15.106:6821","nonce":1846968711}]},"public_addr":"172.21.15.106:6817/1846968711","cluster_addr":"172.21.15.106:6819/1846968711","heartbeat_back_addr":"172.21.15.106:6823/1846968711","heartbeat_front_addr":"172.21.15.106:6821/1846968711","state":["exists","up"]},{"osd":4,"uuid":"1b61f16d-5c60-480a-b449-c745aa3f6b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":729224739},{"type":"v1","addr":"172.21.15.134:6825","nonce":729224739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":729224739},{"type":"v1","addr":"172.21.15.134:6827","nonce":729224739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":729224739},{"type":"v1","addr":"172.21.15.134:6831","nonce":729224739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":729224739},{"type":"v1","addr":"172.21.15.134:6829","nonce":729224739}]},"public_addr":"172.21.15.134:6825/729224739","cluster_addr":"172.21.15.134:6827/729224739","heartbeat_back_addr":"172.21.15.134:6831/729224739","heartbeat_front_addr":"172.21.15.134:6829/729224739","state":["exists","up"]},{"osd":5,"uuid":"944cdd7f-47c0-4b71-b7e0-d093542f4528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3132275652},{"type":"v1","addr":"172.21.15.134:6809","nonce":3132275652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3132275652},{"type":"v1","addr":"172.21.15.134:6811","nonce":3132275652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3132275652},{"type":"v1","addr":"172.21.15.134:6815","nonce":3132275652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3132275652},{"type":"v1","addr":"172.21.15.134:6813","nonce":3132275652}]},"public_addr":"172.21.15.134:6809/3132275652","cluster_addr":"172.21.15.134:6811/3132275652","heartbeat_back_addr":"172.21.15.134:6815/3132275652","heartbeat_front_addr":"172.21.15.134:6813/3132275652","state":["exists","up"]},{"osd":6,"uuid":"cc2446e2-1d21-4135-ac04-6af7c990058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":992702668},{"type":"v1","addr":"172.21.15.134:6801","nonce":992702668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":992702668},{"type":"v1","addr":"172.21.15.134:6803","nonce":992702668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":992702668},{"type":"v1","addr":"172.21.15.134:6807","nonce":992702668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":992702668},{"type":"v1","addr":"172.21.15.134:6805","nonce":992702668}]},"public_addr":"172.21.15.134:6801/992702668","cluster_addr":"172.21.15.134:6803/992702668","heartbeat_back_addr":"172.21.15.134:6807/992702668","heartbeat_front_addr":"172.21.15.134:6805/992702668","state":["exists","up"]},{"osd":7,"uuid":"3df1fc7a-82fc-4257-8629-98027b6e2aed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":3291062284},{"type":"v1","addr":"172.21.15.134:6817","nonce":3291062284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":3291062284},{"type":"v1","addr":"172.21.15.134:6819","nonce":3291062284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":3291062284},{"type":"v1","addr":"172.21.15.134:6823","nonce":3291062284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":3291062284},{"type":"v1","addr":"172.21.15.134:6821","nonce":3291062284}]},"public_addr":"172.21.15.134:6817/3291062284","cluster_addr":"172.21.15.134:6819/3291062284","heartbeat_back_addr":"172.21.15.134:6823/3291062284","heartbeat_front_addr":"172.21.15.134:6821/3291062284","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.403941+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.382563+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.411544+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.330444+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.405430+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.387737+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.408922+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.454450+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:31:10.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.137+0000 7f8ed1ee9640 1 -- 172.21.15.106:0/2861846349 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f8eb80744b0 msgr2=0x7f8eb80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:10.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.137+0000 7f8ed1ee9640 1 --2- 172.21.15.106:0/2861846349 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f8eb80744b0 0x7f8eb80769c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc007fd0 tx=0x7f8ebc009040 comp rx=0 tx=0).stop 2024-04-25T10:31:10.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.137+0000 7f8ed1ee9640 1 -- 172.21.15.106:0/2861846349 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8ecc0c8d90 msgr2=0x7f8ecc1e8420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:10.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.137+0000 7f8ed1ee9640 1 --2- 172.21.15.106:0/2861846349 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8ecc0c8d90 0x7f8ecc1e8420 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8ec400e850 tx=0x7f8ec400ed10 comp rx=0 tx=0).stop 2024-04-25T10:31:10.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.137+0000 7f8ed1ee9640 1 -- 172.21.15.106:0/2861846349 shutdown_connections 2024-04-25T10:31:10.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.137+0000 7f8ed1ee9640 1 --2- 172.21.15.106:0/2861846349 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f8eb80744b0 0x7f8eb80769c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.137+0000 7f8ed1ee9640 1 --2- 172.21.15.106:0/2861846349 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8ecc0c8d90 0x7f8ecc1e8420 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.137+0000 7f8ed1ee9640 1 --2- 172.21.15.106:0/2861846349 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ecc0c7e30 0x7f8ecc1f71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.137+0000 7f8ed1ee9640 1 --2- 172.21.15.106:0/2861846349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ecc0b4ce0 0x7f8ecc14e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.138+0000 7f8ed1ee9640 1 -- 172.21.15.106:0/2861846349 >> 172.21.15.106:0/2861846349 conn(0x7f8ecc068b10 msgr2=0x7f8ecc0b76a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:10.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.138+0000 7f8ed1ee9640 1 -- 172.21.15.106:0/2861846349 shutdown_connections 2024-04-25T10:31:10.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.138+0000 7f8ed1ee9640 1 -- 172.21.15.106:0/2861846349 wait complete. 2024-04-25T10:31:10.151 INFO:tasks.mon_thrash.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:30:20.111046+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:30:30.314914+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:30:30.715500+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:31:10.152 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:31:10.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.259+0000 7f4dffe84640 1 Processor -- start 2024-04-25T10:31:10.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.260+0000 7f4dffe84640 1 -- start start 2024-04-25T10:31:10.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.261+0000 7f4dffe84640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4df80c5810 0x7f4df815e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:10.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.261+0000 7f4dffe84640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4df80caa00 con 0x7f4df8154c60 2024-04-25T10:31:10.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.261+0000 7f4dffe84640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4df80cac70 con 0x7f4df80c5400 2024-04-25T10:31:10.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.261+0000 7f4dffe84640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f4df80cb880 con 0x7f4df80c5810 2024-04-25T10:31:10.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.261+0000 7f4dfdc20640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f4df8154c60 0x7f4df8070ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:46682/0 (socket says 172.21.15.106:46682) 2024-04-25T10:31:10.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.261+0000 7f4dfdc20640 1 -- 172.21.15.106:0/2778812995 learned_addr learned my addr 172.21.15.106:0/2778812995 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:31:10.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.261+0000 7f4dfcc1e640 1 -- 172.21.15.106:0/2778812995 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1423383867 0 0) 0x7f4df80caa00 con 0x7f4df8154c60 2024-04-25T10:31:10.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.261+0000 7f4dfe421640 1 --2- 172.21.15.106:0/2778812995 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4df80c5810 0x7f4df815e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:10.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.261+0000 7f4dfcc1e640 1 -- 172.21.15.106:0/2778812995 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4de4003520 con 0x7f4df8154c60 2024-04-25T10:31:10.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.262+0000 7f4dfcc1e640 1 -- 172.21.15.106:0/2778812995 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 474053545 0 0) 0x7f4de4003520 con 0x7f4df8154c60 2024-04-25T10:31:10.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.262+0000 7f4dfcc1e640 1 -- 172.21.15.106:0/2778812995 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4df80c5810 msgr2=0x7f4df815e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:10.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.262+0000 7f4dfcc1e640 1 --2- 172.21.15.106:0/2778812995 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4df80c5810 0x7f4df815e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.262+0000 7f4dfcc1e640 1 -- 172.21.15.106:0/2778812995 >> v1:172.21.15.134:6789/0 conn(0x7f4df80c5400 legacy=0x7f4df815e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:10.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.262+0000 7f4dfcc1e640 1 -- 172.21.15.106:0/2778812995 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4df80caf50 con 0x7f4df8154c60 2024-04-25T10:31:10.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.262+0000 7f4dfcc1e640 1 -- 172.21.15.106:0/2778812995 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3776459350 0 0) 0x7f4df40019f0 con 0x7f4df8154c60 2024-04-25T10:31:10.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.262+0000 7f4dfcc1e640 1 -- 172.21.15.106:0/2778812995 >> v1:172.21.15.106:6789/0 conn(0x7f4df8154c60 legacy=0x7f4df8070ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:10.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.262+0000 7f4dfcc1e640 1 --2- 172.21.15.106:0/2778812995 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4df80c5810 0x7f4de40039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:10.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.262+0000 7f4dfcc1e640 1 --2- 172.21.15.106:0/2778812995 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4de4003f60 0x7f4de4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:10.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.262+0000 7f4dfcc1e640 1 --2- 172.21.15.106:0/2778812995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4de4004d40 0x7f4de4005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:10.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.262+0000 7f4dfcc1e640 1 -- 172.21.15.106:0/2778812995 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4df80cb880 con 0x7f4de4003f60 2024-04-25T10:31:10.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.262+0000 7f4dfcc1e640 1 -- 172.21.15.106:0/2778812995 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4de400bbe0 con 0x7f4de4004d40 2024-04-25T10:31:10.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.262+0000 7f4dfcc1e640 1 -- 172.21.15.106:0/2778812995 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f4de400bde0 con 0x7f4df80c5810 2024-04-25T10:31:10.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.262+0000 7f4dfd41f640 1 --2- 172.21.15.106:0/2778812995 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4de4003f60 0x7f4de4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:10.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.262+0000 7f4dfdc20640 1 --2- 172.21.15.106:0/2778812995 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4df80c5810 0x7f4de40039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:10.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.262+0000 7f4dfe421640 1 --2- 172.21.15.106:0/2778812995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4de4004d40 0x7f4de4005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:10.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.262+0000 7f4dfd41f640 1 -- 172.21.15.106:0/2778812995 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4df80c5810 msgr2=0x7f4de40039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:10.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.262+0000 7f4dfd41f640 1 --2- 172.21.15.106:0/2778812995 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4df80c5810 0x7f4de40039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.262+0000 7f4dfd41f640 1 -- 172.21.15.106:0/2778812995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4de4004d40 msgr2=0x7f4de4005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:10.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.262+0000 7f4dfd41f640 1 --2- 172.21.15.106:0/2778812995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4de4004d40 0x7f4de4005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.262+0000 7f4dfd41f640 1 -- 172.21.15.106:0/2778812995 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4de4003520 con 0x7f4de4003f60 2024-04-25T10:31:10.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.263+0000 7f4dfd41f640 1 --2- 172.21.15.106:0/2778812995 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4de4003f60 0x7f4de4004800 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4de8001750 tx=0x7f4de4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66be144077508dc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:10.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.263+0000 7f4dfcc1e640 1 -- 172.21.15.106:0/2778812995 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de8001370 con 0x7f4de4003f60 2024-04-25T10:31:10.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.263+0000 7f4dfcc1e640 1 -- 172.21.15.106:0/2778812995 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4de80052d0 con 0x7f4de4003f60 2024-04-25T10:31:10.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.263+0000 7f4dfcc1e640 1 -- 172.21.15.106:0/2778812995 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de802abe0 con 0x7f4de4003f60 2024-04-25T10:31:10.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.264+0000 7f4dffe84640 1 -- 172.21.15.106:0/2778812995 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4de4003f60 msgr2=0x7f4de4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:10.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.264+0000 7f4dffe84640 1 --2- 172.21.15.106:0/2778812995 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4de4003f60 0x7f4de4004800 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4de8001750 tx=0x7f4de4005830 comp rx=0 tx=0).stop 2024-04-25T10:31:10.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.264+0000 7f4dfe421640 1 -- 172.21.15.106:0/2778812995 reap_dead start 2024-04-25T10:31:10.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.264+0000 7f4dffe84640 1 -- 172.21.15.106:0/2778812995 shutdown_connections 2024-04-25T10:31:10.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.264+0000 7f4dffe84640 1 -- 172.21.15.106:0/2778812995 >> 172.21.15.106:0/2778812995 conn(0x7f4df80cff60 msgr2=0x7f4df80d0370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:10.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.264+0000 7f4dffe84640 1 -- 172.21.15.106:0/2778812995 shutdown_connections 2024-04-25T10:31:10.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.264+0000 7f4dffe84640 1 -- 172.21.15.106:0/2778812995 wait complete. 2024-04-25T10:31:10.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.265+0000 7f4dffe84640 1 Processor -- start 2024-04-25T10:31:10.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.266+0000 7f4dffe84640 1 -- start start 2024-04-25T10:31:10.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.267+0000 7f4dffe84640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4de4003f60 0x7f4df81f4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:10.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.267+0000 7f4dffe84640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4df815e700 0x7f4df81f2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:10.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.267+0000 7f4dffe84640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4df80cc4e0 0x7f4df8146800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:10.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.267+0000 7f4dffe84640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4df80c5c20 con 0x7f4df80cc4e0 2024-04-25T10:31:10.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.267+0000 7f4dffe84640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4df80cb510 con 0x7f4de4003f60 2024-04-25T10:31:10.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.267+0000 7f4dffe84640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f4df80713f0 con 0x7f4df815e700 2024-04-25T10:31:10.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.267+0000 7f4dfe421640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4df80cc4e0 0x7f4df8146800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:10.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.267+0000 7f4dfe421640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4df80cc4e0 0x7f4df8146800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59118/0 (socket says 172.21.15.106:59118) 2024-04-25T10:31:10.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.267+0000 7f4dfe421640 1 -- 172.21.15.106:0/1874942910 learned_addr learned my addr 172.21.15.106:0/1874942910 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:31:10.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.267+0000 7f4dfdc20640 1 --2- 172.21.15.106:0/1874942910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4de4003f60 0x7f4df81f4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:10.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.267+0000 7f4dfd41f640 1 --2- 172.21.15.106:0/1874942910 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4df815e700 0x7f4df81f2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:10.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.267+0000 7f4dfe421640 1 -- 172.21.15.106:0/1874942910 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4df815e700 msgr2=0x7f4df81f2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:10.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.267+0000 7f4dfe421640 1 --2- 172.21.15.106:0/1874942910 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4df815e700 0x7f4df81f2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.267+0000 7f4dfe421640 1 -- 172.21.15.106:0/1874942910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4de4003f60 msgr2=0x7f4df81f4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:10.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.267+0000 7f4dfe421640 1 --2- 172.21.15.106:0/1874942910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4de4003f60 0x7f4df81f4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.267+0000 7f4dfe421640 1 -- 172.21.15.106:0/1874942910 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4df80b31b0 con 0x7f4df80cc4e0 2024-04-25T10:31:10.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.267+0000 7f4dfe421640 1 --2- 172.21.15.106:0/1874942910 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4df80cc4e0 0x7f4df8146800 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4df80ca620 tx=0x7f4df0011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:10.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.267+0000 7f4dee7fc640 1 -- 172.21.15.106:0/1874942910 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df0004240 con 0x7f4df80cc4e0 2024-04-25T10:31:10.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.267+0000 7f4dee7fc640 1 -- 172.21.15.106:0/1874942910 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4df001b070 con 0x7f4df80cc4e0 2024-04-25T10:31:10.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.268+0000 7f4dffe84640 1 -- 172.21.15.106:0/1874942910 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4df81f4bc0 con 0x7f4df80cc4e0 2024-04-25T10:31:10.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.268+0000 7f4dee7fc640 1 -- 172.21.15.106:0/1874942910 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df000f770 con 0x7f4df80cc4e0 2024-04-25T10:31:10.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.268+0000 7f4dffe84640 1 -- 172.21.15.106:0/1874942910 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4df81f0d90 con 0x7f4df80cc4e0 2024-04-25T10:31:10.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.268+0000 7f4dffe84640 1 -- 172.21.15.106:0/1874942910 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4df806a3d0 con 0x7f4df80cc4e0 2024-04-25T10:31:10.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.269+0000 7f4dee7fc640 1 -- 172.21.15.106:0/1874942910 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4df0012040 con 0x7f4df80cc4e0 2024-04-25T10:31:10.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.269+0000 7f4dee7fc640 1 --2- 172.21.15.106:0/1874942910 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f4ddc0744e0 0x7f4ddc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:10.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.270+0000 7f4dee7fc640 1 -- 172.21.15.106:0/1874942910 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4df0094ee0 con 0x7f4df80cc4e0 2024-04-25T10:31:10.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.270+0000 7f4dfdc20640 1 --2- 172.21.15.106:0/1874942910 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f4ddc0744e0 0x7f4ddc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:10.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.270+0000 7f4dfdc20640 1 --2- 172.21.15.106:0/1874942910 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f4ddc0744e0 0x7f4ddc0769f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4df4001560 tx=0x7f4df4006040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:10.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.274+0000 7f4dee7fc640 1 -- 172.21.15.106:0/1874942910 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4df80b31b0 con 0x7f4df80cc4e0 2024-04-25T10:31:10.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.449+0000 7f4dffe84640 1 -- 172.21.15.106:0/1874942910 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4df80cbe80 con 0x7f4df80cc4e0 2024-04-25T10:31:10.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.450+0000 7f4dee7fc640 1 -- 172.21.15.106:0/1874942910 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f4df005f0b0 con 0x7f4df80cc4e0 2024-04-25T10:31:10.541 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-25T10:31:10.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.452+0000 7f4dffe84640 1 -- 172.21.15.106:0/1874942910 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f4ddc0744e0 msgr2=0x7f4ddc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:10.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.452+0000 7f4dffe84640 1 --2- 172.21.15.106:0/1874942910 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f4ddc0744e0 0x7f4ddc0769f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4df4001560 tx=0x7f4df4006040 comp rx=0 tx=0).stop 2024-04-25T10:31:10.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.452+0000 7f4dffe84640 1 -- 172.21.15.106:0/1874942910 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4df80cc4e0 msgr2=0x7f4df8146800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:10.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.452+0000 7f4dffe84640 1 --2- 172.21.15.106:0/1874942910 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4df80cc4e0 0x7f4df8146800 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4df80ca620 tx=0x7f4df0011cf0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.452+0000 7f4dffe84640 1 -- 172.21.15.106:0/1874942910 shutdown_connections 2024-04-25T10:31:10.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.452+0000 7f4dffe84640 1 --2- 172.21.15.106:0/1874942910 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f4ddc0744e0 0x7f4ddc0769f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.452+0000 7f4dffe84640 1 --2- 172.21.15.106:0/1874942910 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4df80cc4e0 0x7f4df8146800 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.452+0000 7f4dffe84640 1 --2- 172.21.15.106:0/1874942910 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4df815e700 0x7f4df81f2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.452+0000 7f4dffe84640 1 --2- 172.21.15.106:0/1874942910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4de4003f60 0x7f4df81f4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.452+0000 7f4dffe84640 1 -- 172.21.15.106:0/1874942910 >> 172.21.15.106:0/1874942910 conn(0x7f4df80cff60 msgr2=0x7f4df8154150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:10.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.452+0000 7f4dffe84640 1 -- 172.21.15.106:0/1874942910 shutdown_connections 2024-04-25T10:31:10.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.452+0000 7f4dffe84640 1 -- 172.21.15.106:0/1874942910 wait complete. 2024-04-25T10:31:10.544 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:31:10.655 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.653+0000 7f1dfd135640 1 Processor -- start 2024-04-25T10:31:10.656 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.653+0000 7f1dfd135640 1 -- start start 2024-04-25T10:31:10.657 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.654+0000 7f1dfd135640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1df8156f30 0x7f1df815d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:10.657 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.654+0000 7f1dfd135640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1df815cbd0 con 0x7f1df80c61a0 2024-04-25T10:31:10.657 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.654+0000 7f1dfd135640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1df8160990 con 0x7f1df806dc00 2024-04-25T10:31:10.657 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.654+0000 7f1dfd135640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f1df8157340 con 0x7f1df8156f30 2024-04-25T10:31:10.657 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.654+0000 7f1df759e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1df8156f30 0x7f1df815d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:10.657 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.654+0000 7f1df6d9d640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f1df80c61a0 0x7f1df81601c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:46692/0 (socket says 172.21.15.106:46692) 2024-04-25T10:31:10.657 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.654+0000 7f1df6d9d640 1 -- 172.21.15.106:0/1822713322 learned_addr learned my addr 172.21.15.106:0/1822713322 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:31:10.657 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.655+0000 7f1df759e640 1 -- 172.21.15.106:0/1822713322 >> v1:172.21.15.134:6789/0 conn(0x7f1df806dc00 legacy=0x7f1df80cce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:10.657 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.655+0000 7f1df759e640 1 -- 172.21.15.106:0/1822713322 >> v1:172.21.15.106:6789/0 conn(0x7f1df80c61a0 legacy=0x7f1df81601c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:10.657 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.655+0000 7f1df759e640 1 -- 172.21.15.106:0/1822713322 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1df8160c70 con 0x7f1df8156f30 2024-04-25T10:31:10.657 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.655+0000 7f1df5d9b640 1 -- 172.21.15.106:0/1822713322 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1519159642 0 0) 0x7f1df815cbd0 con 0x7f1df80c61a0 2024-04-25T10:31:10.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.655+0000 7f1df759e640 1 --2- 172.21.15.106:0/1822713322 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1df8156f30 0x7f1df815d690 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1df0009d90 tx=0x7f1df002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2bdaf807ba6e54d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:10.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.656+0000 7f1df5d9b640 1 -- 172.21.15.106:0/1822713322 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df0007740 con 0x7f1df8156f30 2024-04-25T10:31:10.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.656+0000 7f1df5d9b640 1 -- 172.21.15.106:0/1822713322 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1df00078e0 con 0x7f1df8156f30 2024-04-25T10:31:10.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.656+0000 7f1df5d9b640 1 -- 172.21.15.106:0/1822713322 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df0007cb0 con 0x7f1df8156f30 2024-04-25T10:31:10.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.656+0000 7f1dfd135640 1 -- 172.21.15.106:0/1822713322 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1df8156f30 msgr2=0x7f1df815d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:10.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.656+0000 7f1dfd135640 1 --2- 172.21.15.106:0/1822713322 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1df8156f30 0x7f1df815d690 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1df0009d90 tx=0x7f1df002a6a0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.656+0000 7f1dfd135640 1 -- 172.21.15.106:0/1822713322 shutdown_connections 2024-04-25T10:31:10.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.656+0000 7f1dfd135640 1 --2- 172.21.15.106:0/1822713322 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1df8156f30 0x7f1df815d690 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.656+0000 7f1dfd135640 1 -- 172.21.15.106:0/1822713322 >> 172.21.15.106:0/1822713322 conn(0x7f1df80727d0 msgr2=0x7f1df8072c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:10.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.656+0000 7f1dfd135640 1 -- 172.21.15.106:0/1822713322 shutdown_connections 2024-04-25T10:31:10.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.656+0000 7f1dfd135640 1 -- 172.21.15.106:0/1822713322 wait complete. 2024-04-25T10:31:10.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.658+0000 7f1dfd135640 1 Processor -- start 2024-04-25T10:31:10.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.658+0000 7f1dfd135640 1 -- start start 2024-04-25T10:31:10.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.660+0000 7f1dfd135640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1df806dc00 0x7f1df8150370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:10.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.660+0000 7f1dfd135640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1df80c61a0 0x7f1df815f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:10.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.660+0000 7f1dfd135640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1df8156f30 0x7f1df814e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:10.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.660+0000 7f1dfd135640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1df815dbd0 con 0x7f1df8156f30 2024-04-25T10:31:10.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.660+0000 7f1dfd135640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1df815d200 con 0x7f1df80c61a0 2024-04-25T10:31:10.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.660+0000 7f1dfd135640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f1df815ce90 con 0x7f1df806dc00 2024-04-25T10:31:10.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.660+0000 7f1df759e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1df8156f30 0x7f1df814e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:10.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.660+0000 7f1df759e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1df8156f30 0x7f1df814e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59146/0 (socket says 172.21.15.106:59146) 2024-04-25T10:31:10.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.660+0000 7f1df759e640 1 -- 172.21.15.106:0/1347182646 learned_addr learned my addr 172.21.15.106:0/1347182646 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:31:10.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.660+0000 7f1df6d9d640 1 --2- 172.21.15.106:0/1347182646 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1df806dc00 0x7f1df8150370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:10.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.660+0000 7f1df659c640 1 --2- 172.21.15.106:0/1347182646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1df80c61a0 0x7f1df815f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:10.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.660+0000 7f1df759e640 1 -- 172.21.15.106:0/1347182646 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1df806dc00 msgr2=0x7f1df8150370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:10.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.660+0000 7f1df759e640 1 --2- 172.21.15.106:0/1347182646 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1df806dc00 0x7f1df8150370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.660+0000 7f1df759e640 1 -- 172.21.15.106:0/1347182646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1df80c61a0 msgr2=0x7f1df815f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:10.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.660+0000 7f1df759e640 1 --2- 172.21.15.106:0/1347182646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1df80c61a0 0x7f1df815f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.660+0000 7f1df759e640 1 -- 172.21.15.106:0/1347182646 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1df81508b0 con 0x7f1df8156f30 2024-04-25T10:31:10.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.660+0000 7f1df759e640 1 --2- 172.21.15.106:0/1347182646 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1df8156f30 0x7f1df814e280 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1df002ac90 tx=0x7f1df0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:10.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.660+0000 7f1ddf7fe640 1 -- 172.21.15.106:0/1347182646 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df0040070 con 0x7f1df8156f30 2024-04-25T10:31:10.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.660+0000 7f1ddf7fe640 1 -- 172.21.15.106:0/1347182646 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1df0031c90 con 0x7f1df8156f30 2024-04-25T10:31:10.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.660+0000 7f1ddf7fe640 1 -- 172.21.15.106:0/1347182646 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df0045680 con 0x7f1df8156f30 2024-04-25T10:31:10.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.660+0000 7f1dfd135640 1 -- 172.21.15.106:0/1347182646 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1df814f5f0 con 0x7f1df8156f30 2024-04-25T10:31:10.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.661+0000 7f1dfd135640 1 -- 172.21.15.106:0/1347182646 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1df814fb70 con 0x7f1df8156f30 2024-04-25T10:31:10.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.661+0000 7f1dfd135640 1 -- 172.21.15.106:0/1347182646 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1df806a3d0 con 0x7f1df8156f30 2024-04-25T10:31:10.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.662+0000 7f1ddf7fe640 1 -- 172.21.15.106:0/1347182646 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1df0039040 con 0x7f1df8156f30 2024-04-25T10:31:10.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.663+0000 7f1ddf7fe640 1 --2- 172.21.15.106:0/1347182646 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f1de40744e0 0x7f1de40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:10.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.663+0000 7f1df6d9d640 1 --2- 172.21.15.106:0/1347182646 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f1de40744e0 0x7f1de40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:10.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.663+0000 7f1ddf7fe640 1 -- 172.21.15.106:0/1347182646 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1df00b69f0 con 0x7f1df8156f30 2024-04-25T10:31:10.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.663+0000 7f1df6d9d640 1 --2- 172.21.15.106:0/1347182646 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f1de40744e0 0x7f1de40769f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1de8007fd0 tx=0x7f1de8007de0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:10.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.667+0000 7f1ddf7fe640 1 -- 172.21.15.106:0/1347182646 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1df0080c80 con 0x7f1df8156f30 2024-04-25T10:31:10.844 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.841+0000 7f1dfd135640 1 -- 172.21.15.106:0/1347182646 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1df8072e30 con 0x7f1df8156f30 2024-04-25T10:31:10.844 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.842+0000 7f1ddf7fe640 1 -- 172.21.15.106:0/1347182646 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f1df0084f80 con 0x7f1df8156f30 2024-04-25T10:31:10.844 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-25T10:31:10.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.844+0000 7f1dfd135640 1 -- 172.21.15.106:0/1347182646 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f1de40744e0 msgr2=0x7f1de40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:10.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.844+0000 7f1dfd135640 1 --2- 172.21.15.106:0/1347182646 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f1de40744e0 0x7f1de40769f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1de8007fd0 tx=0x7f1de8007de0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.844+0000 7f1dfd135640 1 -- 172.21.15.106:0/1347182646 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1df8156f30 msgr2=0x7f1df814e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:10.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.844+0000 7f1dfd135640 1 --2- 172.21.15.106:0/1347182646 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1df8156f30 0x7f1df814e280 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1df002ac90 tx=0x7f1df0004a40 comp rx=0 tx=0).stop 2024-04-25T10:31:10.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.844+0000 7f1dfd135640 1 -- 172.21.15.106:0/1347182646 shutdown_connections 2024-04-25T10:31:10.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.844+0000 7f1dfd135640 1 --2- 172.21.15.106:0/1347182646 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f1de40744e0 0x7f1de40769f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.844+0000 7f1dfd135640 1 --2- 172.21.15.106:0/1347182646 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1df8156f30 0x7f1df814e280 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.844+0000 7f1dfd135640 1 --2- 172.21.15.106:0/1347182646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1df80c61a0 0x7f1df815f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.844+0000 7f1dfd135640 1 --2- 172.21.15.106:0/1347182646 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1df806dc00 0x7f1df8150370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.844+0000 7f1dfd135640 1 -- 172.21.15.106:0/1347182646 >> 172.21.15.106:0/1347182646 conn(0x7f1df80727d0 msgr2=0x7f1df806ec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:10.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.844+0000 7f1dfd135640 1 -- 172.21.15.106:0/1347182646 shutdown_connections 2024-04-25T10:31:10.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.844+0000 7f1dfd135640 1 -- 172.21.15.106:0/1347182646 wait complete. 2024-04-25T10:31:10.857 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:31:10.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.964+0000 7f8384a51640 1 Processor -- start 2024-04-25T10:31:10.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.964+0000 7f8384a51640 1 -- start start 2024-04-25T10:31:10.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.966+0000 7f8384a51640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83800cbff0 0x7f83800cb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:10.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.966+0000 7f8384a51640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f8380156ec0 0x7f83800c61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:10.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.966+0000 7f8384a51640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f838015dae0 con 0x7f83800c66e0 2024-04-25T10:31:10.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.966+0000 7f8384a51640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f8380161270 con 0x7f8380156ec0 2024-04-25T10:31:10.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.966+0000 7f8384a51640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f83801604b0 con 0x7f83800cbff0 2024-04-25T10:31:10.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.966+0000 7f837dd9b640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f8380156ec0 0x7f83800c61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:10.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.966+0000 7f837e59c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83800cbff0 0x7f83800cb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:10.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.966+0000 7f837ed9d640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f83800c66e0 0x7f83800cce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.106:54910/0 (socket says 172.21.15.106:54910) 2024-04-25T10:31:10.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.966+0000 7f837ed9d640 1 -- 172.21.15.106:0/532552500 learned_addr learned my addr 172.21.15.106:0/532552500 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:31:10.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.967+0000 7f837d59a640 1 -- 172.21.15.106:0/532552500 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3903549629 0 0) 0x7f838015dae0 con 0x7f83800c66e0 2024-04-25T10:31:10.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.967+0000 7f837d59a640 1 -- 172.21.15.106:0/532552500 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8368003520 con 0x7f83800c66e0 2024-04-25T10:31:10.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.967+0000 7f837dd9b640 1 -- 172.21.15.106:0/532552500 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83800cbff0 msgr2=0x7f83800cb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:10.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.967+0000 7f837dd9b640 1 --2- 172.21.15.106:0/532552500 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83800cbff0 0x7f83800cb270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.967+0000 7f837dd9b640 1 -- 172.21.15.106:0/532552500 >> v1:172.21.15.134:6789/0 conn(0x7f83800c66e0 legacy=0x7f83800cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:10.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.967+0000 7f837dd9b640 1 -- 172.21.15.106:0/532552500 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8380160790 con 0x7f8380156ec0 2024-04-25T10:31:10.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.967+0000 7f837e59c640 1 --2- 172.21.15.106:0/532552500 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83800cbff0 0x7f83800cb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:31:10.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.968+0000 7f837dd9b640 1 --2- 172.21.15.106:0/532552500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8380156ec0 0x7f83800c61a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f83740097a0 tx=0x7f837402a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e46b4b74eea73d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:10.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.968+0000 7f837d59a640 1 -- 172.21.15.106:0/532552500 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8374037070 con 0x7f8380156ec0 2024-04-25T10:31:10.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.968+0000 7f837d59a640 1 -- 172.21.15.106:0/532552500 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8374032440 con 0x7f8380156ec0 2024-04-25T10:31:10.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.968+0000 7f837d59a640 1 -- 172.21.15.106:0/532552500 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83740327f0 con 0x7f8380156ec0 2024-04-25T10:31:10.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.968+0000 7f8384a51640 1 -- 172.21.15.106:0/532552500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8380156ec0 msgr2=0x7f83800c61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:10.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.968+0000 7f8384a51640 1 --2- 172.21.15.106:0/532552500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8380156ec0 0x7f83800c61a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f83740097a0 tx=0x7f837402a210 comp rx=0 tx=0).stop 2024-04-25T10:31:10.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.968+0000 7f8384a51640 1 -- 172.21.15.106:0/532552500 shutdown_connections 2024-04-25T10:31:10.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.968+0000 7f8384a51640 1 --2- 172.21.15.106:0/532552500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8380156ec0 0x7f83800c61a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.968+0000 7f8384a51640 1 --2- 172.21.15.106:0/532552500 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83800cbff0 0x7f83800cb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.969+0000 7f8384a51640 1 -- 172.21.15.106:0/532552500 >> 172.21.15.106:0/532552500 conn(0x7f83800727d0 msgr2=0x7f8380072c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:10.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.969+0000 7f8384a51640 1 -- 172.21.15.106:0/532552500 shutdown_connections 2024-04-25T10:31:10.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.969+0000 7f8384a51640 1 -- 172.21.15.106:0/532552500 wait complete. 2024-04-25T10:31:10.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.970+0000 7f8384a51640 1 Processor -- start 2024-04-25T10:31:10.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.970+0000 7f8384a51640 1 -- start start 2024-04-25T10:31:10.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.972+0000 7f8384a51640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83800c66e0 0x7f83800d0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:10.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.972+0000 7f8384a51640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8380156ec0 0x7f83801f5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:10.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.972+0000 7f8384a51640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f838015c9a0 0x7f83801f6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:10.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.972+0000 7f8384a51640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f83801572d0 con 0x7f838015c9a0 2024-04-25T10:31:10.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.972+0000 7f8384a51640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8380160d50 con 0x7f83800c66e0 2024-04-25T10:31:10.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.972+0000 7f8384a51640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f838015d730 con 0x7f8380156ec0 2024-04-25T10:31:10.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.972+0000 7f837ed9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f838015c9a0 0x7f83801f6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:10.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.972+0000 7f837ed9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f838015c9a0 0x7f83801f6320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59160/0 (socket says 172.21.15.106:59160) 2024-04-25T10:31:10.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.972+0000 7f837ed9d640 1 -- 172.21.15.106:0/292937773 learned_addr learned my addr 172.21.15.106:0/292937773 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:31:10.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.972+0000 7f837dd9b640 1 --2- 172.21.15.106:0/292937773 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8380156ec0 0x7f83801f5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:10.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.972+0000 7f837e59c640 1 --2- 172.21.15.106:0/292937773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83800c66e0 0x7f83800d0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:10.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.972+0000 7f837ed9d640 1 -- 172.21.15.106:0/292937773 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8380156ec0 msgr2=0x7f83801f5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:10.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.972+0000 7f837ed9d640 1 --2- 172.21.15.106:0/292937773 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8380156ec0 0x7f83801f5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.972+0000 7f837ed9d640 1 -- 172.21.15.106:0/292937773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83800c66e0 msgr2=0x7f83800d0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:10.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.972+0000 7f837ed9d640 1 --2- 172.21.15.106:0/292937773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83800c66e0 0x7f83800d0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.972+0000 7f837ed9d640 1 -- 172.21.15.106:0/292937773 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f838014f170 con 0x7f838015c9a0 2024-04-25T10:31:10.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.972+0000 7f837ed9d640 1 --2- 172.21.15.106:0/292937773 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f838015c9a0 0x7f83801f6320 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f837800c390 tx=0x7f837800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:10.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.972+0000 7f836effd640 1 -- 172.21.15.106:0/292937773 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8378017070 con 0x7f838015c9a0 2024-04-25T10:31:10.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.972+0000 7f836effd640 1 -- 172.21.15.106:0/292937773 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8378005ce0 con 0x7f838015c9a0 2024-04-25T10:31:10.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.972+0000 7f8384a51640 1 -- 172.21.15.106:0/292937773 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f838015c0d0 con 0x7f838015c9a0 2024-04-25T10:31:10.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.972+0000 7f836effd640 1 -- 172.21.15.106:0/292937773 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83780072d0 con 0x7f838015c9a0 2024-04-25T10:31:10.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.972+0000 7f8384a51640 1 -- 172.21.15.106:0/292937773 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f838015ad70 con 0x7f838015c9a0 2024-04-25T10:31:10.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.973+0000 7f8384a51640 1 -- 172.21.15.106:0/292937773 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f838015cf50 con 0x7f838015c9a0 2024-04-25T10:31:10.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.974+0000 7f836effd640 1 -- 172.21.15.106:0/292937773 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f837801a070 con 0x7f838015c9a0 2024-04-25T10:31:10.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.974+0000 7f836effd640 1 --2- 172.21.15.106:0/292937773 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f83600744e0 0x7f83600769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:10.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.974+0000 7f836effd640 1 -- 172.21.15.106:0/292937773 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8378094350 con 0x7f838015c9a0 2024-04-25T10:31:10.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.974+0000 7f837e59c640 1 --2- 172.21.15.106:0/292937773 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f83600744e0 0x7f83600769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:10.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.975+0000 7f837e59c640 1 --2- 172.21.15.106:0/292937773 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f83600744e0 0x7f83600769f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8370007440 tx=0x7f837000b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:10.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:10.979+0000 7f836effd640 1 -- 172.21.15.106:0/292937773 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f837805e5b0 con 0x7f838015c9a0 2024-04-25T10:31:11.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.152+0000 7f8384a51640 1 -- 172.21.15.106:0/292937773 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f83800723e0 con 0x7f838015c9a0 2024-04-25T10:31:11.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.153+0000 7f836effd640 1 -- 172.21.15.106:0/292937773 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f83800723e0 con 0x7f838015c9a0 2024-04-25T10:31:11.155 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-25T10:31:11.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.154+0000 7f8384a51640 1 -- 172.21.15.106:0/292937773 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f83600744e0 msgr2=0x7f83600769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:11.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.154+0000 7f8384a51640 1 --2- 172.21.15.106:0/292937773 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f83600744e0 0x7f83600769f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8370007440 tx=0x7f837000b040 comp rx=0 tx=0).stop 2024-04-25T10:31:11.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.154+0000 7f8384a51640 1 -- 172.21.15.106:0/292937773 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f838015c9a0 msgr2=0x7f83801f6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:11.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.154+0000 7f8384a51640 1 --2- 172.21.15.106:0/292937773 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f838015c9a0 0x7f83801f6320 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f837800c390 tx=0x7f837800c850 comp rx=0 tx=0).stop 2024-04-25T10:31:11.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.154+0000 7f8384a51640 1 -- 172.21.15.106:0/292937773 shutdown_connections 2024-04-25T10:31:11.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.155+0000 7f8384a51640 1 --2- 172.21.15.106:0/292937773 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f83600744e0 0x7f83600769f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.155+0000 7f8384a51640 1 --2- 172.21.15.106:0/292937773 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f838015c9a0 0x7f83801f6320 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.155+0000 7f8384a51640 1 --2- 172.21.15.106:0/292937773 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8380156ec0 0x7f83801f5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.155+0000 7f8384a51640 1 --2- 172.21.15.106:0/292937773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83800c66e0 0x7f83800d0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.155+0000 7f8384a51640 1 -- 172.21.15.106:0/292937773 >> 172.21.15.106:0/292937773 conn(0x7f83800727d0 msgr2=0x7f838006dad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:11.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.155+0000 7f8384a51640 1 -- 172.21.15.106:0/292937773 shutdown_connections 2024-04-25T10:31:11.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.155+0000 7f8384a51640 1 -- 172.21.15.106:0/292937773 wait complete. 2024-04-25T10:31:11.168 INFO:tasks.mon_thrash.ceph_manager:waiting for clean 2024-04-25T10:31:11.168 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:31:11.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.273+0000 7fab60c92640 1 Processor -- start 2024-04-25T10:31:11.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.273+0000 7fab60c92640 1 -- start start 2024-04-25T10:31:11.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.274+0000 7fab60c92640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fab5c0c7e30 0x7fab5c0c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:11.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.274+0000 7fab60c92640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fab5c156ef0 0x7fab5c0b4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:11.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.274+0000 7fab60c92640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fab5c06d000 0x7fab5c06d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:11.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.274+0000 7fab60c92640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fab5c0c7c60 con 0x7fab5c06d000 2024-04-25T10:31:11.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.274+0000 7fab60c92640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fab5c0ce630 con 0x7fab5c156ef0 2024-04-25T10:31:11.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.274+0000 7fab60c92640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fab5c0c7980 con 0x7fab5c0c7e30 2024-04-25T10:31:11.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.274+0000 7fab5ad9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fab5c06d000 0x7fab5c06d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:11.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.274+0000 7fab5ad9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fab5c06d000 0x7fab5c06d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59176/0 (socket says 172.21.15.106:59176) 2024-04-25T10:31:11.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.274+0000 7fab5ad9d640 1 -- 172.21.15.106:0/882357230 learned_addr learned my addr 172.21.15.106:0/882357230 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:31:11.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.275+0000 7fab5a59c640 1 --2- 172.21.15.106:0/882357230 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fab5c0c7e30 0x7fab5c0c8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:11.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.275+0000 7fab53fff640 1 --2- 172.21.15.106:0/882357230 >> v2:172.21.15.134:3300/0 conn(0x7fab5c156ef0 0x7fab5c0b4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:11.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.275+0000 7fab5ad9d640 1 -- 172.21.15.106:0/882357230 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fab5c0c7e30 msgr2=0x7fab5c0c8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:11.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.275+0000 7fab5ad9d640 1 --2- 172.21.15.106:0/882357230 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fab5c0c7e30 0x7fab5c0c8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.275+0000 7fab5ad9d640 1 -- 172.21.15.106:0/882357230 >> v2:172.21.15.134:3300/0 conn(0x7fab5c156ef0 msgr2=0x7fab5c0b4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:11.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.275+0000 7fab5ad9d640 1 --2- 172.21.15.106:0/882357230 >> v2:172.21.15.134:3300/0 conn(0x7fab5c156ef0 0x7fab5c0b4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.275+0000 7fab5ad9d640 1 -- 172.21.15.106:0/882357230 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab5c0c7010 con 0x7fab5c06d000 2024-04-25T10:31:11.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.275+0000 7fab53fff640 1 --2- 172.21.15.106:0/882357230 >> v2:172.21.15.134:3300/0 conn(0x7fab5c156ef0 0x7fab5c0b4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:31:11.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.275+0000 7fab5ad9d640 1 --2- 172.21.15.106:0/882357230 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab5c06d000 0x7fab5c06d410 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fab5c0cfaf0 tx=0x7fab5402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c023c2add7015df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:11.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.275+0000 7fab59d9b640 1 -- 172.21.15.106:0/882357230 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab54036070 con 0x7fab5c06d000 2024-04-25T10:31:11.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.276+0000 7fab59d9b640 1 -- 172.21.15.106:0/882357230 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab54031600 con 0x7fab5c06d000 2024-04-25T10:31:11.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.276+0000 7fab59d9b640 1 -- 172.21.15.106:0/882357230 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab540319d0 con 0x7fab5c06d000 2024-04-25T10:31:11.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.276+0000 7fab60c92640 1 -- 172.21.15.106:0/882357230 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab5c06d000 msgr2=0x7fab5c06d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:11.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.276+0000 7fab60c92640 1 --2- 172.21.15.106:0/882357230 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab5c06d000 0x7fab5c06d410 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fab5c0cfaf0 tx=0x7fab5402a610 comp rx=0 tx=0).stop 2024-04-25T10:31:11.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.276+0000 7fab60c92640 1 -- 172.21.15.106:0/882357230 shutdown_connections 2024-04-25T10:31:11.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.276+0000 7fab60c92640 1 --2- 172.21.15.106:0/882357230 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab5c06d000 0x7fab5c06d410 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.276+0000 7fab60c92640 1 --2- 172.21.15.106:0/882357230 >> v2:172.21.15.134:3300/0 conn(0x7fab5c156ef0 0x7fab5c0b4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.276+0000 7fab60c92640 1 --2- 172.21.15.106:0/882357230 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fab5c0c7e30 0x7fab5c0c8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.276+0000 7fab60c92640 1 -- 172.21.15.106:0/882357230 >> 172.21.15.106:0/882357230 conn(0x7fab5c068b10 msgr2=0x7fab5c0d1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:11.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.276+0000 7fab60c92640 1 -- 172.21.15.106:0/882357230 shutdown_connections 2024-04-25T10:31:11.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.276+0000 7fab60c92640 1 -- 172.21.15.106:0/882357230 wait complete. 2024-04-25T10:31:11.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.278+0000 7fab60c92640 1 Processor -- start 2024-04-25T10:31:11.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.278+0000 7fab60c92640 1 -- start start 2024-04-25T10:31:11.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.279+0000 7fab60c92640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fab5c06d000 0x7fab5c1f3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:11.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.279+0000 7fab60c92640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab5c0c7e30 0x7fab5c1f6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:11.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.279+0000 7fab60c92640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab5c156ef0 0x7fab5c148ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:11.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.279+0000 7fab60c92640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fab5c15cfb0 con 0x7fab5c156ef0 2024-04-25T10:31:11.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.279+0000 7fab60c92640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fab5c15ce30 con 0x7fab5c0c7e30 2024-04-25T10:31:11.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.279+0000 7fab60c92640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fab5c15d130 con 0x7fab5c06d000 2024-04-25T10:31:11.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.279+0000 7fab5ad9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab5c156ef0 0x7fab5c148ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:11.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.279+0000 7fab5ad9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab5c156ef0 0x7fab5c148ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59184/0 (socket says 172.21.15.106:59184) 2024-04-25T10:31:11.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.279+0000 7fab5ad9d640 1 -- 172.21.15.106:0/3241012000 learned_addr learned my addr 172.21.15.106:0/3241012000 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:31:11.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.280+0000 7fab5a59c640 1 --2- 172.21.15.106:0/3241012000 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fab5c06d000 0x7fab5c1f3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:11.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.280+0000 7fab53fff640 1 --2- 172.21.15.106:0/3241012000 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab5c0c7e30 0x7fab5c1f6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:11.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.280+0000 7fab5ad9d640 1 -- 172.21.15.106:0/3241012000 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fab5c06d000 msgr2=0x7fab5c1f3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:11.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.280+0000 7fab5ad9d640 1 --2- 172.21.15.106:0/3241012000 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fab5c06d000 0x7fab5c1f3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.280+0000 7fab5ad9d640 1 -- 172.21.15.106:0/3241012000 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab5c0c7e30 msgr2=0x7fab5c1f6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:11.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.280+0000 7fab5ad9d640 1 --2- 172.21.15.106:0/3241012000 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab5c0c7e30 0x7fab5c1f6f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.280+0000 7fab5ad9d640 1 -- 172.21.15.106:0/3241012000 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab5c0d0890 con 0x7fab5c156ef0 2024-04-25T10:31:11.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.280+0000 7fab5ad9d640 1 --2- 172.21.15.106:0/3241012000 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab5c156ef0 0x7fab5c148ae0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fab54009f40 tx=0x7fab54004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:11.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.280+0000 7fab52ffd640 1 -- 172.21.15.106:0/3241012000 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab54036050 con 0x7fab5c156ef0 2024-04-25T10:31:11.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.280+0000 7fab52ffd640 1 -- 172.21.15.106:0/3241012000 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab54042070 con 0x7fab5c156ef0 2024-04-25T10:31:11.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.280+0000 7fab60c92640 1 -- 172.21.15.106:0/3241012000 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab5c1f3010 con 0x7fab5c156ef0 2024-04-25T10:31:11.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.280+0000 7fab52ffd640 1 -- 172.21.15.106:0/3241012000 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab54009510 con 0x7fab5c156ef0 2024-04-25T10:31:11.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.280+0000 7fab60c92640 1 -- 172.21.15.106:0/3241012000 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab5c1f32d0 con 0x7fab5c156ef0 2024-04-25T10:31:11.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.281+0000 7fab60c92640 1 -- 172.21.15.106:0/3241012000 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab5c06a3d0 con 0x7fab5c156ef0 2024-04-25T10:31:11.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.282+0000 7fab52ffd640 1 -- 172.21.15.106:0/3241012000 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fab54007bb0 con 0x7fab5c156ef0 2024-04-25T10:31:11.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.282+0000 7fab52ffd640 1 --2- 172.21.15.106:0/3241012000 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fab3c0744b0 0x7fab3c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:11.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.282+0000 7fab5a59c640 1 --2- 172.21.15.106:0/3241012000 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fab3c0744b0 0x7fab3c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:11.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.282+0000 7fab52ffd640 1 -- 172.21.15.106:0/3241012000 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fab540b5f50 con 0x7fab5c156ef0 2024-04-25T10:31:11.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.283+0000 7fab5a59c640 1 --2- 172.21.15.106:0/3241012000 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fab3c0744b0 0x7fab3c0769c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fab4c003610 tx=0x7fab4c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:11.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.287+0000 7fab52ffd640 1 -- 172.21.15.106:0/3241012000 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fab54080120 con 0x7fab5c156ef0 2024-04-25T10:31:11.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.461+0000 7fab60c92640 1 -- 172.21.15.106:0/3241012000 --> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fab5c0d1ca0 con 0x7fab3c0744b0 2024-04-25T10:31:11.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.467+0000 7fab52ffd640 1 -- 172.21.15.106:0/3241012000 <== mgr.4107 v2:172.21.15.106:6832/1744210167 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251915 (secure 0 0 0) 0x7fab5c0d1ca0 con 0x7fab3c0744b0 2024-04-25T10:31:11.473 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:31:11.474 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-25T10:31:11.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.470+0000 7fab60c92640 1 -- 172.21.15.106:0/3241012000 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fab3c0744b0 msgr2=0x7fab3c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:11.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.470+0000 7fab60c92640 1 --2- 172.21.15.106:0/3241012000 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fab3c0744b0 0x7fab3c0769c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fab4c003610 tx=0x7fab4c008040 comp rx=0 tx=0).stop 2024-04-25T10:31:11.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.470+0000 7fab60c92640 1 -- 172.21.15.106:0/3241012000 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab5c156ef0 msgr2=0x7fab5c148ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:11.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.470+0000 7fab60c92640 1 --2- 172.21.15.106:0/3241012000 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab5c156ef0 0x7fab5c148ae0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fab54009f40 tx=0x7fab54004a40 comp rx=0 tx=0).stop 2024-04-25T10:31:11.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.470+0000 7fab60c92640 1 -- 172.21.15.106:0/3241012000 shutdown_connections 2024-04-25T10:31:11.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.470+0000 7fab60c92640 1 --2- 172.21.15.106:0/3241012000 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fab3c0744b0 0x7fab3c0769c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.470+0000 7fab60c92640 1 --2- 172.21.15.106:0/3241012000 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab5c156ef0 0x7fab5c148ae0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.470+0000 7fab60c92640 1 --2- 172.21.15.106:0/3241012000 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab5c0c7e30 0x7fab5c1f6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.470+0000 7fab60c92640 1 --2- 172.21.15.106:0/3241012000 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fab5c06d000 0x7fab5c1f3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.470+0000 7fab60c92640 1 -- 172.21.15.106:0/3241012000 >> 172.21.15.106:0/3241012000 conn(0x7fab5c068b10 msgr2=0x7fab5c0d1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:11.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.470+0000 7fab60c92640 1 -- 172.21.15.106:0/3241012000 shutdown_connections 2024-04-25T10:31:11.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.470+0000 7fab60c92640 1 -- 172.21.15.106:0/3241012000 wait complete. 2024-04-25T10:31:11.484 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2024-04-25T10:31:10.105789+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":598402,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":305,"num_write_kb":1690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":65,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":250,"ondisk_log_size":250,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219144,"kb_used_data":3016,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755576,"statfs":{"total":773094113280,"available":772869709824,"internally_reserved":0,"allocated":3088384,"data_stored":1764276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1406,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002715"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795275+0000","last_change":"2024-04-25T10:30:32.440983+0000","last_active":"2024-04-25T10:30:34.795275+0000","last_peered":"2024-04-25T10:30:34.795275+0000","last_clean":"2024-04-25T10:30:34.795275+0000","last_became_active":"2024-04-25T10:30:32.440552+0000","last_became_peered":"2024-04-25T10:30:32.440552+0000","last_unstale":"2024-04-25T10:30:34.795275+0000","last_undegraded":"2024-04-25T10:30:34.795275+0000","last_fullsized":"2024-04-25T10:30:34.795275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:47.963061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.412686+0000","last_change":"2024-04-25T10:30:32.425695+0000","last_active":"2024-04-25T10:30:33.412686+0000","last_peered":"2024-04-25T10:30:33.412686+0000","last_clean":"2024-04-25T10:30:33.412686+0000","last_became_active":"2024-04-25T10:30:32.425368+0000","last_became_peered":"2024-04-25T10:30:32.425368+0000","last_unstale":"2024-04-25T10:30:33.412686+0000","last_undegraded":"2024-04-25T10:30:33.412686+0000","last_fullsized":"2024-04-25T10:30:33.412686+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:39.120522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.196620+0000","last_change":"2024-04-25T10:30:32.430144+0000","last_active":"2024-04-25T10:30:34.196620+0000","last_peered":"2024-04-25T10:30:34.196620+0000","last_clean":"2024-04-25T10:30:34.196620+0000","last_became_active":"2024-04-25T10:30:32.429754+0000","last_became_peered":"2024-04-25T10:30:32.429754+0000","last_unstale":"2024-04-25T10:30:34.196620+0000","last_undegraded":"2024-04-25T10:30:34.196620+0000","last_fullsized":"2024-04-25T10:30:34.196620+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:11.448907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794948+0000","last_change":"2024-04-25T10:30:32.442478+0000","last_active":"2024-04-25T10:30:34.794948+0000","last_peered":"2024-04-25T10:30:34.794948+0000","last_clean":"2024-04-25T10:30:34.794948+0000","last_became_active":"2024-04-25T10:30:32.442182+0000","last_became_peered":"2024-04-25T10:30:32.442182+0000","last_unstale":"2024-04-25T10:30:34.794948+0000","last_undegraded":"2024-04-25T10:30:34.794948+0000","last_fullsized":"2024-04-25T10:30:34.794948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:22.185978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794683+0000","last_change":"2024-04-25T10:30:32.438231+0000","last_active":"2024-04-25T10:30:34.794683+0000","last_peered":"2024-04-25T10:30:34.794683+0000","last_clean":"2024-04-25T10:30:34.794683+0000","last_became_active":"2024-04-25T10:30:32.437656+0000","last_became_peered":"2024-04-25T10:30:32.437656+0000","last_unstale":"2024-04-25T10:30:34.794683+0000","last_undegraded":"2024-04-25T10:30:34.794683+0000","last_fullsized":"2024-04-25T10:30:34.794683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:56.915999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795645+0000","last_change":"2024-04-25T10:30:32.439221+0000","last_active":"2024-04-25T10:30:34.795645+0000","last_peered":"2024-04-25T10:30:34.795645+0000","last_clean":"2024-04-25T10:30:34.795645+0000","last_became_active":"2024-04-25T10:30:32.438923+0000","last_became_peered":"2024-04-25T10:30:32.438923+0000","last_unstale":"2024-04-25T10:30:34.795645+0000","last_undegraded":"2024-04-25T10:30:34.795645+0000","last_fullsized":"2024-04-25T10:30:34.795645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:40.469409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794157+0000","last_change":"2024-04-25T10:30:32.429453+0000","last_active":"2024-04-25T10:30:34.794157+0000","last_peered":"2024-04-25T10:30:34.794157+0000","last_clean":"2024-04-25T10:30:34.794157+0000","last_became_active":"2024-04-25T10:30:32.429131+0000","last_became_peered":"2024-04-25T10:30:32.429131+0000","last_unstale":"2024-04-25T10:30:34.794157+0000","last_undegraded":"2024-04-25T10:30:34.794157+0000","last_fullsized":"2024-04-25T10:30:34.794157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:26.785922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415260+0000","last_change":"2024-04-25T10:30:32.438860+0000","last_active":"2024-04-25T10:30:33.415260+0000","last_peered":"2024-04-25T10:30:33.415260+0000","last_clean":"2024-04-25T10:30:33.415260+0000","last_became_active":"2024-04-25T10:30:32.438491+0000","last_became_peered":"2024-04-25T10:30:32.438491+0000","last_unstale":"2024-04-25T10:30:33.415260+0000","last_undegraded":"2024-04-25T10:30:33.415260+0000","last_fullsized":"2024-04-25T10:30:33.415260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:08.610707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795467+0000","last_change":"2024-04-25T10:30:32.440331+0000","last_active":"2024-04-25T10:30:34.795467+0000","last_peered":"2024-04-25T10:30:34.795467+0000","last_clean":"2024-04-25T10:30:34.795467+0000","last_became_active":"2024-04-25T10:30:32.440043+0000","last_became_peered":"2024-04-25T10:30:32.440043+0000","last_unstale":"2024-04-25T10:30:34.795467+0000","last_undegraded":"2024-04-25T10:30:34.795467+0000","last_fullsized":"2024-04-25T10:30:34.795467+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:17.684685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794636+0000","last_change":"2024-04-25T10:30:32.435388+0000","last_active":"2024-04-25T10:30:34.794636+0000","last_peered":"2024-04-25T10:30:34.794636+0000","last_clean":"2024-04-25T10:30:34.794636+0000","last_became_active":"2024-04-25T10:30:32.434983+0000","last_became_peered":"2024-04-25T10:30:32.434983+0000","last_unstale":"2024-04-25T10:30:34.794636+0000","last_undegraded":"2024-04-25T10:30:34.794636+0000","last_fullsized":"2024-04-25T10:30:34.794636+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:39.256604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.793928+0000","last_change":"2024-04-25T10:30:32.435884+0000","last_active":"2024-04-25T10:30:34.793928+0000","last_peered":"2024-04-25T10:30:34.793928+0000","last_clean":"2024-04-25T10:30:34.793928+0000","last_became_active":"2024-04-25T10:30:32.434645+0000","last_became_peered":"2024-04-25T10:30:32.434645+0000","last_unstale":"2024-04-25T10:30:34.793928+0000","last_undegraded":"2024-04-25T10:30:34.793928+0000","last_fullsized":"2024-04-25T10:30:34.793928+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:07.778437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795720+0000","last_change":"2024-04-25T10:30:32.436403+0000","last_active":"2024-04-25T10:30:34.795720+0000","last_peered":"2024-04-25T10:30:34.795720+0000","last_clean":"2024-04-25T10:30:34.795720+0000","last_became_active":"2024-04-25T10:30:32.435908+0000","last_became_peered":"2024-04-25T10:30:32.435908+0000","last_unstale":"2024-04-25T10:30:34.795720+0000","last_undegraded":"2024-04-25T10:30:34.795720+0000","last_fullsized":"2024-04-25T10:30:34.795720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:03.339316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415872+0000","last_change":"2024-04-25T10:30:32.432110+0000","last_active":"2024-04-25T10:30:33.415872+0000","last_peered":"2024-04-25T10:30:33.415872+0000","last_clean":"2024-04-25T10:30:33.415872+0000","last_became_active":"2024-04-25T10:30:32.431792+0000","last_became_peered":"2024-04-25T10:30:32.431792+0000","last_unstale":"2024-04-25T10:30:33.415872+0000","last_undegraded":"2024-04-25T10:30:33.415872+0000","last_fullsized":"2024-04-25T10:30:33.415872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:34.185989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.196476+0000","last_change":"2024-04-25T10:30:32.433676+0000","last_active":"2024-04-25T10:30:34.196476+0000","last_peered":"2024-04-25T10:30:34.196476+0000","last_clean":"2024-04-25T10:30:34.196476+0000","last_became_active":"2024-04-25T10:30:32.433329+0000","last_became_peered":"2024-04-25T10:30:32.433329+0000","last_unstale":"2024-04-25T10:30:34.196476+0000","last_undegraded":"2024-04-25T10:30:34.196476+0000","last_fullsized":"2024-04-25T10:30:34.196476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:59.942368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.412754+0000","last_change":"2024-04-25T10:30:32.423582+0000","last_active":"2024-04-25T10:30:33.412754+0000","last_peered":"2024-04-25T10:30:33.412754+0000","last_clean":"2024-04-25T10:30:33.412754+0000","last_became_active":"2024-04-25T10:30:32.422991+0000","last_became_peered":"2024-04-25T10:30:32.422991+0000","last_unstale":"2024-04-25T10:30:33.412754+0000","last_undegraded":"2024-04-25T10:30:33.412754+0000","last_fullsized":"2024-04-25T10:30:33.412754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:11:47.132928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794545+0000","last_change":"2024-04-25T10:30:32.440235+0000","last_active":"2024-04-25T10:30:34.794545+0000","last_peered":"2024-04-25T10:30:34.794545+0000","last_clean":"2024-04-25T10:30:34.794545+0000","last_became_active":"2024-04-25T10:30:32.439191+0000","last_became_peered":"2024-04-25T10:30:32.439191+0000","last_unstale":"2024-04-25T10:30:34.794545+0000","last_undegraded":"2024-04-25T10:30:34.794545+0000","last_fullsized":"2024-04-25T10:30:34.794545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:51.423086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795170+0000","last_change":"2024-04-25T10:30:32.442418+0000","last_active":"2024-04-25T10:30:34.795170+0000","last_peered":"2024-04-25T10:30:34.795170+0000","last_clean":"2024-04-25T10:30:34.795170+0000","last_became_active":"2024-04-25T10:30:32.442125+0000","last_became_peered":"2024-04-25T10:30:32.442125+0000","last_unstale":"2024-04-25T10:30:34.795170+0000","last_undegraded":"2024-04-25T10:30:34.795170+0000","last_fullsized":"2024-04-25T10:30:34.795170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:14.213809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415344+0000","last_change":"2024-04-25T10:30:32.434758+0000","last_active":"2024-04-25T10:30:33.415344+0000","last_peered":"2024-04-25T10:30:33.415344+0000","last_clean":"2024-04-25T10:30:33.415344+0000","last_became_active":"2024-04-25T10:30:32.434492+0000","last_became_peered":"2024-04-25T10:30:32.434492+0000","last_unstale":"2024-04-25T10:30:33.415344+0000","last_undegraded":"2024-04-25T10:30:33.415344+0000","last_fullsized":"2024-04-25T10:30:33.415344+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:41.454407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.796066+0000","last_change":"2024-04-25T10:30:32.440399+0000","last_active":"2024-04-25T10:30:34.796066+0000","last_peered":"2024-04-25T10:30:34.796066+0000","last_clean":"2024-04-25T10:30:34.796066+0000","last_became_active":"2024-04-25T10:30:32.438227+0000","last_became_peered":"2024-04-25T10:30:32.438227+0000","last_unstale":"2024-04-25T10:30:34.796066+0000","last_undegraded":"2024-04-25T10:30:34.796066+0000","last_fullsized":"2024-04-25T10:30:34.796066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:49.571671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794149+0000","last_change":"2024-04-25T10:30:32.431609+0000","last_active":"2024-04-25T10:30:34.794149+0000","last_peered":"2024-04-25T10:30:34.794149+0000","last_clean":"2024-04-25T10:30:34.794149+0000","last_became_active":"2024-04-25T10:30:32.431314+0000","last_became_peered":"2024-04-25T10:30:32.431314+0000","last_unstale":"2024-04-25T10:30:34.794149+0000","last_undegraded":"2024-04-25T10:30:34.794149+0000","last_fullsized":"2024-04-25T10:30:34.794149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:33.421989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.196668+0000","last_change":"2024-04-25T10:30:32.432612+0000","last_active":"2024-04-25T10:30:34.196668+0000","last_peered":"2024-04-25T10:30:34.196668+0000","last_clean":"2024-04-25T10:30:34.196668+0000","last_became_active":"2024-04-25T10:30:32.432271+0000","last_became_peered":"2024-04-25T10:30:32.432271+0000","last_unstale":"2024-04-25T10:30:34.196668+0000","last_undegraded":"2024-04-25T10:30:34.196668+0000","last_fullsized":"2024-04-25T10:30:34.196668+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:14.281493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795827+0000","last_change":"2024-04-25T10:30:32.444618+0000","last_active":"2024-04-25T10:30:34.795827+0000","last_peered":"2024-04-25T10:30:34.795827+0000","last_clean":"2024-04-25T10:30:34.795827+0000","last_became_active":"2024-04-25T10:30:32.444273+0000","last_became_peered":"2024-04-25T10:30:32.444273+0000","last_unstale":"2024-04-25T10:30:34.795827+0000","last_undegraded":"2024-04-25T10:30:34.795827+0000","last_fullsized":"2024-04-25T10:30:34.795827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:09.295251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415348+0000","last_change":"2024-04-25T10:30:32.426578+0000","last_active":"2024-04-25T10:30:33.415348+0000","last_peered":"2024-04-25T10:30:33.415348+0000","last_clean":"2024-04-25T10:30:33.415348+0000","last_became_active":"2024-04-25T10:30:32.426223+0000","last_became_peered":"2024-04-25T10:30:32.426223+0000","last_unstale":"2024-04-25T10:30:33.415348+0000","last_undegraded":"2024-04-25T10:30:33.415348+0000","last_fullsized":"2024-04-25T10:30:33.415348+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:53.700585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794721+0000","last_change":"2024-04-25T10:30:32.439767+0000","last_active":"2024-04-25T10:30:34.794721+0000","last_peered":"2024-04-25T10:30:34.794721+0000","last_clean":"2024-04-25T10:30:34.794721+0000","last_became_active":"2024-04-25T10:30:32.438801+0000","last_became_peered":"2024-04-25T10:30:32.438801+0000","last_unstale":"2024-04-25T10:30:34.794721+0000","last_undegraded":"2024-04-25T10:30:34.794721+0000","last_fullsized":"2024-04-25T10:30:34.794721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:58.231512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415544+0000","last_change":"2024-04-25T10:30:32.427424+0000","last_active":"2024-04-25T10:30:33.415544+0000","last_peered":"2024-04-25T10:30:33.415544+0000","last_clean":"2024-04-25T10:30:33.415544+0000","last_became_active":"2024-04-25T10:30:32.427081+0000","last_became_peered":"2024-04-25T10:30:32.427081+0000","last_unstale":"2024-04-25T10:30:33.415544+0000","last_undegraded":"2024-04-25T10:30:33.415544+0000","last_fullsized":"2024-04-25T10:30:33.415544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:47.372451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794727+0000","last_change":"2024-04-25T10:30:32.432472+0000","last_active":"2024-04-25T10:30:34.794727+0000","last_peered":"2024-04-25T10:30:34.794727+0000","last_clean":"2024-04-25T10:30:34.794727+0000","last_became_active":"2024-04-25T10:30:32.432121+0000","last_became_peered":"2024-04-25T10:30:32.432121+0000","last_unstale":"2024-04-25T10:30:34.794727+0000","last_undegraded":"2024-04-25T10:30:34.794727+0000","last_fullsized":"2024-04-25T10:30:34.794727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:34.191339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.412571+0000","last_change":"2024-04-25T10:30:32.425108+0000","last_active":"2024-04-25T10:30:33.412571+0000","last_peered":"2024-04-25T10:30:33.412571+0000","last_clean":"2024-04-25T10:30:33.412571+0000","last_became_active":"2024-04-25T10:30:32.424794+0000","last_became_peered":"2024-04-25T10:30:32.424794+0000","last_unstale":"2024-04-25T10:30:33.412571+0000","last_undegraded":"2024-04-25T10:30:33.412571+0000","last_fullsized":"2024-04-25T10:30:33.412571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:01.282758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415734+0000","last_change":"2024-04-25T10:30:32.426672+0000","last_active":"2024-04-25T10:30:33.415734+0000","last_peered":"2024-04-25T10:30:33.415734+0000","last_clean":"2024-04-25T10:30:33.415734+0000","last_became_active":"2024-04-25T10:30:32.426348+0000","last_became_peered":"2024-04-25T10:30:32.426348+0000","last_unstale":"2024-04-25T10:30:33.415734+0000","last_undegraded":"2024-04-25T10:30:33.415734+0000","last_fullsized":"2024-04-25T10:30:33.415734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:57.645772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.196609+0000","last_change":"2024-04-25T10:30:32.427989+0000","last_active":"2024-04-25T10:30:34.196609+0000","last_peered":"2024-04-25T10:30:34.196609+0000","last_clean":"2024-04-25T10:30:34.196609+0000","last_became_active":"2024-04-25T10:30:32.427676+0000","last_became_peered":"2024-04-25T10:30:32.427676+0000","last_unstale":"2024-04-25T10:30:34.196609+0000","last_undegraded":"2024-04-25T10:30:34.196609+0000","last_fullsized":"2024-04-25T10:30:34.196609+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:15:02.332186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795935+0000","last_change":"2024-04-25T10:30:32.441112+0000","last_active":"2024-04-25T10:30:34.795935+0000","last_peered":"2024-04-25T10:30:34.795935+0000","last_clean":"2024-04-25T10:30:34.795935+0000","last_became_active":"2024-04-25T10:30:32.440815+0000","last_became_peered":"2024-04-25T10:30:32.440815+0000","last_unstale":"2024-04-25T10:30:34.795935+0000","last_undegraded":"2024-04-25T10:30:34.795935+0000","last_fullsized":"2024-04-25T10:30:34.795935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:02.956759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415178+0000","last_change":"2024-04-25T10:30:32.430902+0000","last_active":"2024-04-25T10:30:33.415178+0000","last_peered":"2024-04-25T10:30:33.415178+0000","last_clean":"2024-04-25T10:30:33.415178+0000","last_became_active":"2024-04-25T10:30:32.430615+0000","last_became_peered":"2024-04-25T10:30:32.430615+0000","last_unstale":"2024-04-25T10:30:33.415178+0000","last_undegraded":"2024-04-25T10:30:33.415178+0000","last_fullsized":"2024-04-25T10:30:33.415178+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:12.747595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795247+0000","last_change":"2024-04-25T10:30:32.435379+0000","last_active":"2024-04-25T10:30:34.795247+0000","last_peered":"2024-04-25T10:30:34.795247+0000","last_clean":"2024-04-25T10:30:34.795247+0000","last_became_active":"2024-04-25T10:30:32.435055+0000","last_became_peered":"2024-04-25T10:30:32.435055+0000","last_unstale":"2024-04-25T10:30:34.795247+0000","last_undegraded":"2024-04-25T10:30:34.795247+0000","last_fullsized":"2024-04-25T10:30:34.795247+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:13.623762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794490+0000","last_change":"2024-04-25T10:30:32.422102+0000","last_active":"2024-04-25T10:30:34.794490+0000","last_peered":"2024-04-25T10:30:34.794490+0000","last_clean":"2024-04-25T10:30:34.794490+0000","last_became_active":"2024-04-25T10:30:32.421766+0000","last_became_peered":"2024-04-25T10:30:32.421766+0000","last_unstale":"2024-04-25T10:30:34.794490+0000","last_undegraded":"2024-04-25T10:30:34.794490+0000","last_fullsized":"2024-04-25T10:30:34.794490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:30.300572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:43.470932+0000","last_change":"2024-04-25T10:30:31.418065+0000","last_active":"2024-04-25T10:30:43.470932+0000","last_peered":"2024-04-25T10:30:43.470932+0000","last_clean":"2024-04-25T10:30:43.470932+0000","last_became_active":"2024-04-25T10:30:31.417451+0000","last_became_peered":"2024-04-25T10:30:31.417451+0000","last_unstale":"2024-04-25T10:30:43.470932+0000","last_undegraded":"2024-04-25T10:30:43.470932+0000","last_fullsized":"2024-04-25T10:30:43.470932+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:17.262861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.196777+0000","last_change":"2024-04-25T10:30:32.426913+0000","last_active":"2024-04-25T10:30:34.196777+0000","last_peered":"2024-04-25T10:30:34.196777+0000","last_clean":"2024-04-25T10:30:34.196777+0000","last_became_active":"2024-04-25T10:30:32.426616+0000","last_became_peered":"2024-04-25T10:30:32.426616+0000","last_unstale":"2024-04-25T10:30:34.196777+0000","last_undegraded":"2024-04-25T10:30:34.196777+0000","last_fullsized":"2024-04-25T10:30:34.196777+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:50.614932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'7","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:53.568128+0000","last_change":"2024-04-25T10:30:31.416432+0000","last_active":"2024-04-25T10:30:53.568128+0000","last_peered":"2024-04-25T10:30:53.568128+0000","last_clean":"2024-04-25T10:30:53.568128+0000","last_became_active":"2024-04-25T10:30:31.415112+0000","last_became_peered":"2024-04-25T10:30:31.415112+0000","last_unstale":"2024-04-25T10:30:53.568128+0000","last_undegraded":"2024-04-25T10:30:53.568128+0000","last_fullsized":"2024-04-25T10:30:53.568128+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:22.674365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"20'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:48.494314+0000","last_change":"2024-04-25T10:30:31.418059+0000","last_active":"2024-04-25T10:30:48.494314+0000","last_peered":"2024-04-25T10:30:48.494314+0000","last_clean":"2024-04-25T10:30:48.494314+0000","last_became_active":"2024-04-25T10:30:31.417716+0000","last_became_peered":"2024-04-25T10:30:31.417716+0000","last_unstale":"2024-04-25T10:30:48.494314+0000","last_undegraded":"2024-04-25T10:30:48.494314+0000","last_fullsized":"2024-04-25T10:30:48.494314+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:57.752218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415428+0000","last_change":"2024-04-25T10:30:32.428078+0000","last_active":"2024-04-25T10:30:33.415428+0000","last_peered":"2024-04-25T10:30:33.415428+0000","last_clean":"2024-04-25T10:30:33.415428+0000","last_became_active":"2024-04-25T10:30:32.427700+0000","last_became_peered":"2024-04-25T10:30:32.427700+0000","last_unstale":"2024-04-25T10:30:33.415428+0000","last_undegraded":"2024-04-25T10:30:33.415428+0000","last_fullsized":"2024-04-25T10:30:33.415428+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:26.931791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"20'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:53.568903+0000","last_change":"2024-04-25T10:30:31.415030+0000","last_active":"2024-04-25T10:30:53.568903+0000","last_peered":"2024-04-25T10:30:53.568903+0000","last_clean":"2024-04-25T10:30:53.568903+0000","last_became_active":"2024-04-25T10:30:31.414542+0000","last_became_peered":"2024-04-25T10:30:31.414542+0000","last_unstale":"2024-04-25T10:30:53.568903+0000","last_undegraded":"2024-04-25T10:30:53.568903+0000","last_fullsized":"2024-04-25T10:30:53.568903+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:20.082420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.412471+0000","last_change":"2024-04-25T10:30:32.425207+0000","last_active":"2024-04-25T10:30:33.412471+0000","last_peered":"2024-04-25T10:30:33.412471+0000","last_clean":"2024-04-25T10:30:33.412471+0000","last_became_active":"2024-04-25T10:30:32.424849+0000","last_became_peered":"2024-04-25T10:30:32.424849+0000","last_unstale":"2024-04-25T10:30:33.412471+0000","last_undegraded":"2024-04-25T10:30:33.412471+0000","last_fullsized":"2024-04-25T10:30:33.412471+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:11.533686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"20'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:43.473310+0000","last_change":"2024-04-25T10:30:31.407700+0000","last_active":"2024-04-25T10:30:43.473310+0000","last_peered":"2024-04-25T10:30:43.473310+0000","last_clean":"2024-04-25T10:30:43.473310+0000","last_became_active":"2024-04-25T10:30:31.407376+0000","last_became_peered":"2024-04-25T10:30:31.407376+0000","last_unstale":"2024-04-25T10:30:43.473310+0000","last_undegraded":"2024-04-25T10:30:43.473310+0000","last_fullsized":"2024-04-25T10:30:43.473310+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:50.112742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794157+0000","last_change":"2024-04-25T10:30:32.444599+0000","last_active":"2024-04-25T10:30:34.794157+0000","last_peered":"2024-04-25T10:30:34.794157+0000","last_clean":"2024-04-25T10:30:34.794157+0000","last_became_active":"2024-04-25T10:30:32.444250+0000","last_became_peered":"2024-04-25T10:30:32.444250+0000","last_unstale":"2024-04-25T10:30:34.794157+0000","last_undegraded":"2024-04-25T10:30:34.794157+0000","last_fullsized":"2024-04-25T10:30:34.794157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:55.711525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"20'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:59.589106+0000","last_change":"2024-04-25T10:30:31.405051+0000","last_active":"2024-04-25T10:30:59.589106+0000","last_peered":"2024-04-25T10:30:59.589106+0000","last_clean":"2024-04-25T10:30:59.589106+0000","last_became_active":"2024-04-25T10:30:31.404669+0000","last_became_peered":"2024-04-25T10:30:31.404669+0000","last_unstale":"2024-04-25T10:30:59.589106+0000","last_undegraded":"2024-04-25T10:30:59.589106+0000","last_fullsized":"2024-04-25T10:30:59.589106+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:45.260847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.196763+0000","last_change":"2024-04-25T10:30:32.435804+0000","last_active":"2024-04-25T10:30:34.196763+0000","last_peered":"2024-04-25T10:30:34.196763+0000","last_clean":"2024-04-25T10:30:34.196763+0000","last_became_active":"2024-04-25T10:30:32.435471+0000","last_became_peered":"2024-04-25T10:30:32.435471+0000","last_unstale":"2024-04-25T10:30:34.196763+0000","last_undegraded":"2024-04-25T10:30:34.196763+0000","last_fullsized":"2024-04-25T10:30:34.196763+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:48.213846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:43.470877+0000","last_change":"2024-04-25T10:30:31.403560+0000","last_active":"2024-04-25T10:30:43.470877+0000","last_peered":"2024-04-25T10:30:43.470877+0000","last_clean":"2024-04-25T10:30:43.470877+0000","last_became_active":"2024-04-25T10:30:31.403167+0000","last_became_peered":"2024-04-25T10:30:31.403167+0000","last_unstale":"2024-04-25T10:30:43.470877+0000","last_undegraded":"2024-04-25T10:30:43.470877+0000","last_fullsized":"2024-04-25T10:30:43.470877+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:54.592832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795908+0000","last_change":"2024-04-25T10:30:32.424101+0000","last_active":"2024-04-25T10:30:34.795908+0000","last_peered":"2024-04-25T10:30:34.795908+0000","last_clean":"2024-04-25T10:30:34.795908+0000","last_became_active":"2024-04-25T10:30:32.423780+0000","last_became_peered":"2024-04-25T10:30:32.423780+0000","last_unstale":"2024-04-25T10:30:34.795908+0000","last_undegraded":"2024-04-25T10:30:34.795908+0000","last_fullsized":"2024-04-25T10:30:34.795908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:29.175348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'10","reported_seq":24,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:59.783452+0000","last_change":"2024-04-25T10:30:31.418201+0000","last_active":"2024-04-25T10:30:59.783452+0000","last_peered":"2024-04-25T10:30:59.783452+0000","last_clean":"2024-04-25T10:30:59.783452+0000","last_became_active":"2024-04-25T10:30:31.417824+0000","last_became_peered":"2024-04-25T10:30:31.417824+0000","last_unstale":"2024-04-25T10:30:59.783452+0000","last_undegraded":"2024-04-25T10:30:59.783452+0000","last_fullsized":"2024-04-25T10:30:59.783452+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:38.584225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795830+0000","last_change":"2024-04-25T10:30:32.431139+0000","last_active":"2024-04-25T10:30:34.795830+0000","last_peered":"2024-04-25T10:30:34.795830+0000","last_clean":"2024-04-25T10:30:34.795830+0000","last_became_active":"2024-04-25T10:30:32.429622+0000","last_became_peered":"2024-04-25T10:30:32.429622+0000","last_unstale":"2024-04-25T10:30:34.795830+0000","last_undegraded":"2024-04-25T10:30:34.795830+0000","last_fullsized":"2024-04-25T10:30:34.795830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:39.993647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'9","reported_seq":23,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:59.586443+0000","last_change":"2024-04-25T10:30:31.414775+0000","last_active":"2024-04-25T10:30:59.586443+0000","last_peered":"2024-04-25T10:30:59.586443+0000","last_clean":"2024-04-25T10:30:59.586443+0000","last_became_active":"2024-04-25T10:30:31.414422+0000","last_became_peered":"2024-04-25T10:30:31.414422+0000","last_unstale":"2024-04-25T10:30:59.586443+0000","last_undegraded":"2024-04-25T10:30:59.586443+0000","last_fullsized":"2024-04-25T10:30:59.586443+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:37.803505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.196648+0000","last_change":"2024-04-25T10:30:32.432884+0000","last_active":"2024-04-25T10:30:34.196648+0000","last_peered":"2024-04-25T10:30:34.196648+0000","last_clean":"2024-04-25T10:30:34.196648+0000","last_became_active":"2024-04-25T10:30:32.431743+0000","last_became_peered":"2024-04-25T10:30:32.431743+0000","last_unstale":"2024-04-25T10:30:34.196648+0000","last_undegraded":"2024-04-25T10:30:34.196648+0000","last_fullsized":"2024-04-25T10:30:34.196648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:15:06.737631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:53.568904+0000","last_change":"2024-04-25T10:30:31.419497+0000","last_active":"2024-04-25T10:30:53.568904+0000","last_peered":"2024-04-25T10:30:53.568904+0000","last_clean":"2024-04-25T10:30:53.568904+0000","last_became_active":"2024-04-25T10:30:31.419135+0000","last_became_peered":"2024-04-25T10:30:31.419135+0000","last_unstale":"2024-04-25T10:30:53.568904+0000","last_undegraded":"2024-04-25T10:30:53.568904+0000","last_fullsized":"2024-04-25T10:30:53.568904+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:59.403927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794867+0000","last_change":"2024-04-25T10:30:32.434197+0000","last_active":"2024-04-25T10:30:34.794867+0000","last_peered":"2024-04-25T10:30:34.794867+0000","last_clean":"2024-04-25T10:30:34.794867+0000","last_became_active":"2024-04-25T10:30:32.433623+0000","last_became_peered":"2024-04-25T10:30:32.433623+0000","last_unstale":"2024-04-25T10:30:34.794867+0000","last_undegraded":"2024-04-25T10:30:34.794867+0000","last_fullsized":"2024-04-25T10:30:34.794867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:08.027472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:53.567835+0000","last_change":"2024-04-25T10:30:31.416901+0000","last_active":"2024-04-25T10:30:53.567835+0000","last_peered":"2024-04-25T10:30:53.567835+0000","last_clean":"2024-04-25T10:30:53.567835+0000","last_became_active":"2024-04-25T10:30:31.415706+0000","last_became_peered":"2024-04-25T10:30:31.415706+0000","last_unstale":"2024-04-25T10:30:53.567835+0000","last_undegraded":"2024-04-25T10:30:53.567835+0000","last_fullsized":"2024-04-25T10:30:53.567835+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:23.243039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795344+0000","last_change":"2024-04-25T10:30:32.435089+0000","last_active":"2024-04-25T10:30:34.795344+0000","last_peered":"2024-04-25T10:30:34.795344+0000","last_clean":"2024-04-25T10:30:34.795344+0000","last_became_active":"2024-04-25T10:30:32.434737+0000","last_became_peered":"2024-04-25T10:30:32.434737+0000","last_unstale":"2024-04-25T10:30:34.795344+0000","last_undegraded":"2024-04-25T10:30:34.795344+0000","last_fullsized":"2024-04-25T10:30:34.795344+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:19.738850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'17","reported_seq":31,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:53.570024+0000","last_change":"2024-04-25T10:30:31.405163+0000","last_active":"2024-04-25T10:30:53.570024+0000","last_peered":"2024-04-25T10:30:53.570024+0000","last_clean":"2024-04-25T10:30:53.570024+0000","last_became_active":"2024-04-25T10:30:31.404775+0000","last_became_peered":"2024-04-25T10:30:31.404775+0000","last_unstale":"2024-04-25T10:30:53.570024+0000","last_undegraded":"2024-04-25T10:30:53.570024+0000","last_fullsized":"2024-04-25T10:30:53.570024+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:10:21.595653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794446+0000","last_change":"2024-04-25T10:30:32.432061+0000","last_active":"2024-04-25T10:30:34.794446+0000","last_peered":"2024-04-25T10:30:34.794446+0000","last_clean":"2024-04-25T10:30:34.794446+0000","last_became_active":"2024-04-25T10:30:32.431651+0000","last_became_peered":"2024-04-25T10:30:32.431651+0000","last_unstale":"2024-04-25T10:30:34.794446+0000","last_undegraded":"2024-04-25T10:30:34.794446+0000","last_fullsized":"2024-04-25T10:30:34.794446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:39.062490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:37.434698+0000","last_change":"2024-04-25T10:30:31.420258+0000","last_active":"2024-04-25T10:30:37.434698+0000","last_peered":"2024-04-25T10:30:37.434698+0000","last_clean":"2024-04-25T10:30:37.434698+0000","last_became_active":"2024-04-25T10:30:31.419902+0000","last_became_peered":"2024-04-25T10:30:31.419902+0000","last_unstale":"2024-04-25T10:30:37.434698+0000","last_undegraded":"2024-04-25T10:30:37.434698+0000","last_fullsized":"2024-04-25T10:30:37.434698+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:43.919867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795443+0000","last_change":"2024-04-25T10:30:32.433568+0000","last_active":"2024-04-25T10:30:34.795443+0000","last_peered":"2024-04-25T10:30:34.795443+0000","last_clean":"2024-04-25T10:30:34.795443+0000","last_became_active":"2024-04-25T10:30:32.432572+0000","last_became_peered":"2024-04-25T10:30:32.432572+0000","last_unstale":"2024-04-25T10:30:34.795443+0000","last_undegraded":"2024-04-25T10:30:34.795443+0000","last_fullsized":"2024-04-25T10:30:34.795443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:31.771919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:59.583380+0000","last_change":"2024-04-25T10:30:31.419308+0000","last_active":"2024-04-25T10:30:59.583380+0000","last_peered":"2024-04-25T10:30:59.583380+0000","last_clean":"2024-04-25T10:30:59.583380+0000","last_became_active":"2024-04-25T10:30:31.418978+0000","last_became_peered":"2024-04-25T10:30:31.418978+0000","last_unstale":"2024-04-25T10:30:59.583380+0000","last_undegraded":"2024-04-25T10:30:59.583380+0000","last_fullsized":"2024-04-25T10:30:59.583380+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:33.784804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795670+0000","last_change":"2024-04-25T10:30:32.436109+0000","last_active":"2024-04-25T10:30:34.795670+0000","last_peered":"2024-04-25T10:30:34.795670+0000","last_clean":"2024-04-25T10:30:34.795670+0000","last_became_active":"2024-04-25T10:30:32.434701+0000","last_became_peered":"2024-04-25T10:30:32.434701+0000","last_unstale":"2024-04-25T10:30:34.795670+0000","last_undegraded":"2024-04-25T10:30:34.795670+0000","last_fullsized":"2024-04-25T10:30:34.795670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:19.471867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:53.569755+0000","last_change":"2024-04-25T10:30:31.416044+0000","last_active":"2024-04-25T10:30:53.569755+0000","last_peered":"2024-04-25T10:30:53.569755+0000","last_clean":"2024-04-25T10:30:53.569755+0000","last_became_active":"2024-04-25T10:30:31.415707+0000","last_became_peered":"2024-04-25T10:30:31.415707+0000","last_unstale":"2024-04-25T10:30:53.569755+0000","last_undegraded":"2024-04-25T10:30:53.569755+0000","last_fullsized":"2024-04-25T10:30:53.569755+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:31.424135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795321+0000","last_change":"2024-04-25T10:30:32.442971+0000","last_active":"2024-04-25T10:30:34.795321+0000","last_peered":"2024-04-25T10:30:34.795321+0000","last_clean":"2024-04-25T10:30:34.795321+0000","last_became_active":"2024-04-25T10:30:32.442656+0000","last_became_peered":"2024-04-25T10:30:32.442656+0000","last_unstale":"2024-04-25T10:30:34.795321+0000","last_undegraded":"2024-04-25T10:30:34.795321+0000","last_fullsized":"2024-04-25T10:30:34.795321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:29.951142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:53.566613+0000","last_change":"2024-04-25T10:30:31.404793+0000","last_active":"2024-04-25T10:30:53.566613+0000","last_peered":"2024-04-25T10:30:53.566613+0000","last_clean":"2024-04-25T10:30:53.566613+0000","last_became_active":"2024-04-25T10:30:31.404392+0000","last_became_peered":"2024-04-25T10:30:31.404392+0000","last_unstale":"2024-04-25T10:30:53.566613+0000","last_undegraded":"2024-04-25T10:30:53.566613+0000","last_fullsized":"2024-04-25T10:30:53.566613+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:00.965450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795922+0000","last_change":"2024-04-25T10:30:32.441438+0000","last_active":"2024-04-25T10:30:34.795922+0000","last_peered":"2024-04-25T10:30:34.795922+0000","last_clean":"2024-04-25T10:30:34.795922+0000","last_became_active":"2024-04-25T10:30:32.440011+0000","last_became_peered":"2024-04-25T10:30:32.440011+0000","last_unstale":"2024-04-25T10:30:34.795922+0000","last_undegraded":"2024-04-25T10:30:34.795922+0000","last_fullsized":"2024-04-25T10:30:34.795922+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:09.127517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":110,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.793865+0000","last_change":"2024-04-25T10:30:21.429098+0000","last_active":"2024-04-25T10:30:34.793865+0000","last_peered":"2024-04-25T10:30:34.793865+0000","last_clean":"2024-04-25T10:30:34.793865+0000","last_became_active":"2024-04-25T10:30:21.427958+0000","last_became_peered":"2024-04-25T10:30:21.427958+0000","last_unstale":"2024-04-25T10:30:34.793865+0000","last_undegraded":"2024-04-25T10:30:34.793865+0000","last_fullsized":"2024-04-25T10:30:34.793865+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:20.186206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:20.186206+0000","last_clean_scrub_stamp":"2024-04-25T10:30:20.186206+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:46.046488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"20'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:53.569669+0000","last_change":"2024-04-25T10:30:31.404974+0000","last_active":"2024-04-25T10:30:53.569669+0000","last_peered":"2024-04-25T10:30:53.569669+0000","last_clean":"2024-04-25T10:30:53.569669+0000","last_became_active":"2024-04-25T10:30:31.404549+0000","last_became_peered":"2024-04-25T10:30:31.404549+0000","last_unstale":"2024-04-25T10:30:53.569669+0000","last_undegraded":"2024-04-25T10:30:53.569669+0000","last_fullsized":"2024-04-25T10:30:53.569669+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:06.191321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415412+0000","last_change":"2024-04-25T10:30:32.434409+0000","last_active":"2024-04-25T10:30:33.415412+0000","last_peered":"2024-04-25T10:30:33.415412+0000","last_clean":"2024-04-25T10:30:33.415412+0000","last_became_active":"2024-04-25T10:30:32.433830+0000","last_became_peered":"2024-04-25T10:30:32.433830+0000","last_unstale":"2024-04-25T10:30:33.415412+0000","last_undegraded":"2024-04-25T10:30:33.415412+0000","last_fullsized":"2024-04-25T10:30:33.415412+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:57.864627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"20'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:48.490759+0000","last_change":"2024-04-25T10:30:31.416025+0000","last_active":"2024-04-25T10:30:48.490759+0000","last_peered":"2024-04-25T10:30:48.490759+0000","last_clean":"2024-04-25T10:30:48.490759+0000","last_became_active":"2024-04-25T10:30:31.415675+0000","last_became_peered":"2024-04-25T10:30:31.415675+0000","last_unstale":"2024-04-25T10:30:48.490759+0000","last_undegraded":"2024-04-25T10:30:48.490759+0000","last_fullsized":"2024-04-25T10:30:48.490759+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:28:48.185715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.196505+0000","last_change":"2024-04-25T10:30:32.424630+0000","last_active":"2024-04-25T10:30:34.196505+0000","last_peered":"2024-04-25T10:30:34.196505+0000","last_clean":"2024-04-25T10:30:34.196505+0000","last_became_active":"2024-04-25T10:30:32.424303+0000","last_became_peered":"2024-04-25T10:30:32.424303+0000","last_unstale":"2024-04-25T10:30:34.196505+0000","last_undegraded":"2024-04-25T10:30:34.196505+0000","last_fullsized":"2024-04-25T10:30:34.196505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:55.535319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:48.498662+0000","last_change":"2024-04-25T10:30:31.419145+0000","last_active":"2024-04-25T10:30:48.498662+0000","last_peered":"2024-04-25T10:30:48.498662+0000","last_clean":"2024-04-25T10:30:48.498662+0000","last_became_active":"2024-04-25T10:30:31.418817+0000","last_became_peered":"2024-04-25T10:30:31.418817+0000","last_unstale":"2024-04-25T10:30:48.498662+0000","last_undegraded":"2024-04-25T10:30:48.498662+0000","last_fullsized":"2024-04-25T10:30:48.498662+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:01.608205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.412646+0000","last_change":"2024-04-25T10:30:32.419911+0000","last_active":"2024-04-25T10:30:33.412646+0000","last_peered":"2024-04-25T10:30:33.412646+0000","last_clean":"2024-04-25T10:30:33.412646+0000","last_became_active":"2024-04-25T10:30:32.419486+0000","last_became_peered":"2024-04-25T10:30:32.419486+0000","last_unstale":"2024-04-25T10:30:33.412646+0000","last_undegraded":"2024-04-25T10:30:33.412646+0000","last_fullsized":"2024-04-25T10:30:33.412646+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:54.967873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:59.589197+0000","last_change":"2024-04-25T10:30:31.405646+0000","last_active":"2024-04-25T10:30:59.589197+0000","last_peered":"2024-04-25T10:30:59.589197+0000","last_clean":"2024-04-25T10:30:59.589197+0000","last_became_active":"2024-04-25T10:30:31.405238+0000","last_became_peered":"2024-04-25T10:30:31.405238+0000","last_unstale":"2024-04-25T10:30:59.589197+0000","last_undegraded":"2024-04-25T10:30:59.589197+0000","last_fullsized":"2024-04-25T10:30:59.589197+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:13.159787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795730+0000","last_change":"2024-04-25T10:30:32.426130+0000","last_active":"2024-04-25T10:30:34.795730+0000","last_peered":"2024-04-25T10:30:34.795730+0000","last_clean":"2024-04-25T10:30:34.795730+0000","last_became_active":"2024-04-25T10:30:32.425783+0000","last_became_peered":"2024-04-25T10:30:32.425783+0000","last_unstale":"2024-04-25T10:30:34.795730+0000","last_undegraded":"2024-04-25T10:30:34.795730+0000","last_fullsized":"2024-04-25T10:30:34.795730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:30.844640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.119286+0000","last_change":"2024-04-25T10:30:31.412736+0000","last_active":"2024-04-25T10:30:35.119286+0000","last_peered":"2024-04-25T10:30:35.119286+0000","last_clean":"2024-04-25T10:30:35.119286+0000","last_became_active":"2024-04-25T10:30:31.412413+0000","last_became_peered":"2024-04-25T10:30:31.412413+0000","last_unstale":"2024-04-25T10:30:35.119286+0000","last_undegraded":"2024-04-25T10:30:35.119286+0000","last_fullsized":"2024-04-25T10:30:35.119286+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:19:56.236327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795688+0000","last_change":"2024-04-25T10:30:32.440858+0000","last_active":"2024-04-25T10:30:34.795688+0000","last_peered":"2024-04-25T10:30:34.795688+0000","last_clean":"2024-04-25T10:30:34.795688+0000","last_became_active":"2024-04-25T10:30:32.440519+0000","last_became_peered":"2024-04-25T10:30:32.440519+0000","last_unstale":"2024-04-25T10:30:34.795688+0000","last_undegraded":"2024-04-25T10:30:34.795688+0000","last_fullsized":"2024-04-25T10:30:34.795688+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:21.288748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"20'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:53.569891+0000","last_change":"2024-04-25T10:30:31.417958+0000","last_active":"2024-04-25T10:30:53.569891+0000","last_peered":"2024-04-25T10:30:53.569891+0000","last_clean":"2024-04-25T10:30:53.569891+0000","last_became_active":"2024-04-25T10:30:31.417541+0000","last_became_peered":"2024-04-25T10:30:31.417541+0000","last_unstale":"2024-04-25T10:30:53.569891+0000","last_undegraded":"2024-04-25T10:30:53.569891+0000","last_fullsized":"2024-04-25T10:30:53.569891+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:35.373351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.412553+0000","last_change":"2024-04-25T10:30:32.420451+0000","last_active":"2024-04-25T10:30:33.412553+0000","last_peered":"2024-04-25T10:30:33.412553+0000","last_clean":"2024-04-25T10:30:33.412553+0000","last_became_active":"2024-04-25T10:30:32.420109+0000","last_became_peered":"2024-04-25T10:30:32.420109+0000","last_unstale":"2024-04-25T10:30:33.412553+0000","last_undegraded":"2024-04-25T10:30:33.412553+0000","last_fullsized":"2024-04-25T10:30:33.412553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:22.067828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'7","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:53.569765+0000","last_change":"2024-04-25T10:30:31.406005+0000","last_active":"2024-04-25T10:30:53.569765+0000","last_peered":"2024-04-25T10:30:53.569765+0000","last_clean":"2024-04-25T10:30:53.569765+0000","last_became_active":"2024-04-25T10:30:31.404739+0000","last_became_peered":"2024-04-25T10:30:31.404739+0000","last_unstale":"2024-04-25T10:30:53.569765+0000","last_undegraded":"2024-04-25T10:30:53.569765+0000","last_fullsized":"2024-04-25T10:30:53.569765+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:31.771123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415711+0000","last_change":"2024-04-25T10:30:32.437513+0000","last_active":"2024-04-25T10:30:33.415711+0000","last_peered":"2024-04-25T10:30:33.415711+0000","last_clean":"2024-04-25T10:30:33.415711+0000","last_became_active":"2024-04-25T10:30:32.437171+0000","last_became_peered":"2024-04-25T10:30:32.437171+0000","last_unstale":"2024-04-25T10:30:33.415711+0000","last_undegraded":"2024-04-25T10:30:33.415711+0000","last_fullsized":"2024-04-25T10:30:33.415711+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:50.924843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'7","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:59.786001+0000","last_change":"2024-04-25T10:30:31.413140+0000","last_active":"2024-04-25T10:30:59.786001+0000","last_peered":"2024-04-25T10:30:59.786001+0000","last_clean":"2024-04-25T10:30:59.786001+0000","last_became_active":"2024-04-25T10:30:31.412821+0000","last_became_peered":"2024-04-25T10:30:31.412821+0000","last_unstale":"2024-04-25T10:30:59.786001+0000","last_undegraded":"2024-04-25T10:30:59.786001+0000","last_fullsized":"2024-04-25T10:30:59.786001+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:37.177254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.196799+0000","last_change":"2024-04-25T10:30:32.434333+0000","last_active":"2024-04-25T10:30:34.196799+0000","last_peered":"2024-04-25T10:30:34.196799+0000","last_clean":"2024-04-25T10:30:34.196799+0000","last_became_active":"2024-04-25T10:30:32.434000+0000","last_became_peered":"2024-04-25T10:30:32.434000+0000","last_unstale":"2024-04-25T10:30:34.196799+0000","last_undegraded":"2024-04-25T10:30:34.196799+0000","last_fullsized":"2024-04-25T10:30:34.196799+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:54.057457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:43.468255+0000","last_change":"2024-04-25T10:30:31.419475+0000","last_active":"2024-04-25T10:30:43.468255+0000","last_peered":"2024-04-25T10:30:43.468255+0000","last_clean":"2024-04-25T10:30:43.468255+0000","last_became_active":"2024-04-25T10:30:31.417605+0000","last_became_peered":"2024-04-25T10:30:31.417605+0000","last_unstale":"2024-04-25T10:30:43.468255+0000","last_undegraded":"2024-04-25T10:30:43.468255+0000","last_fullsized":"2024-04-25T10:30:43.468255+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:00.948269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794414+0000","last_change":"2024-04-25T10:30:32.419693+0000","last_active":"2024-04-25T10:30:34.794414+0000","last_peered":"2024-04-25T10:30:34.794414+0000","last_clean":"2024-04-25T10:30:34.794414+0000","last_became_active":"2024-04-25T10:30:32.419370+0000","last_became_peered":"2024-04-25T10:30:32.419370+0000","last_unstale":"2024-04-25T10:30:34.794414+0000","last_undegraded":"2024-04-25T10:30:34.794414+0000","last_fullsized":"2024-04-25T10:30:34.794414+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:31.199065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"20'7","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:59.589760+0000","last_change":"2024-04-25T10:30:31.410767+0000","last_active":"2024-04-25T10:30:59.589760+0000","last_peered":"2024-04-25T10:30:59.589760+0000","last_clean":"2024-04-25T10:30:59.589760+0000","last_became_active":"2024-04-25T10:30:31.410404+0000","last_became_peered":"2024-04-25T10:30:31.410404+0000","last_unstale":"2024-04-25T10:30:59.589760+0000","last_undegraded":"2024-04-25T10:30:59.589760+0000","last_fullsized":"2024-04-25T10:30:59.589760+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:37.431532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415585+0000","last_change":"2024-04-25T10:30:32.433687+0000","last_active":"2024-04-25T10:30:33.415585+0000","last_peered":"2024-04-25T10:30:33.415585+0000","last_clean":"2024-04-25T10:30:33.415585+0000","last_became_active":"2024-04-25T10:30:32.433389+0000","last_became_peered":"2024-04-25T10:30:32.433389+0000","last_unstale":"2024-04-25T10:30:33.415585+0000","last_undegraded":"2024-04-25T10:30:33.415585+0000","last_fullsized":"2024-04-25T10:30:33.415585+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:39.155317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"20'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:53.569498+0000","last_change":"2024-04-25T10:30:31.419118+0000","last_active":"2024-04-25T10:30:53.569498+0000","last_peered":"2024-04-25T10:30:53.569498+0000","last_clean":"2024-04-25T10:30:53.569498+0000","last_became_active":"2024-04-25T10:30:31.418822+0000","last_became_peered":"2024-04-25T10:30:31.418822+0000","last_unstale":"2024-04-25T10:30:53.569498+0000","last_undegraded":"2024-04-25T10:30:53.569498+0000","last_fullsized":"2024-04-25T10:30:53.569498+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:22.755042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415890+0000","last_change":"2024-04-25T10:30:32.427894+0000","last_active":"2024-04-25T10:30:33.415890+0000","last_peered":"2024-04-25T10:30:33.415890+0000","last_clean":"2024-04-25T10:30:33.415890+0000","last_became_active":"2024-04-25T10:30:32.427504+0000","last_became_peered":"2024-04-25T10:30:32.427504+0000","last_unstale":"2024-04-25T10:30:33.415890+0000","last_undegraded":"2024-04-25T10:30:33.415890+0000","last_fullsized":"2024-04-25T10:30:33.415890+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:53.877484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:43.469382+0000","last_change":"2024-04-25T10:30:31.417115+0000","last_active":"2024-04-25T10:30:43.469382+0000","last_peered":"2024-04-25T10:30:43.469382+0000","last_clean":"2024-04-25T10:30:43.469382+0000","last_became_active":"2024-04-25T10:30:31.416766+0000","last_became_peered":"2024-04-25T10:30:31.416766+0000","last_unstale":"2024-04-25T10:30:43.469382+0000","last_undegraded":"2024-04-25T10:30:43.469382+0000","last_fullsized":"2024-04-25T10:30:43.469382+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:06.021270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795407+0000","last_change":"2024-04-25T10:30:32.431674+0000","last_active":"2024-04-25T10:30:34.795407+0000","last_peered":"2024-04-25T10:30:34.795407+0000","last_clean":"2024-04-25T10:30:34.795407+0000","last_became_active":"2024-04-25T10:30:32.431395+0000","last_became_peered":"2024-04-25T10:30:32.431395+0000","last_unstale":"2024-04-25T10:30:34.795407+0000","last_undegraded":"2024-04-25T10:30:34.795407+0000","last_fullsized":"2024-04-25T10:30:34.795407+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:49.726323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795508+0000","last_change":"2024-04-25T10:30:32.437700+0000","last_active":"2024-04-25T10:30:34.795508+0000","last_peered":"2024-04-25T10:30:34.795508+0000","last_clean":"2024-04-25T10:30:34.795508+0000","last_became_active":"2024-04-25T10:30:32.437242+0000","last_became_peered":"2024-04-25T10:30:32.437242+0000","last_unstale":"2024-04-25T10:30:34.795508+0000","last_undegraded":"2024-04-25T10:30:34.795508+0000","last_fullsized":"2024-04-25T10:30:34.795508+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:06.499414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:59.589520+0000","last_change":"2024-04-25T10:30:31.419678+0000","last_active":"2024-04-25T10:30:59.589520+0000","last_peered":"2024-04-25T10:30:59.589520+0000","last_clean":"2024-04-25T10:30:59.589520+0000","last_became_active":"2024-04-25T10:30:31.419269+0000","last_became_peered":"2024-04-25T10:30:31.419269+0000","last_unstale":"2024-04-25T10:30:59.589520+0000","last_undegraded":"2024-04-25T10:30:59.589520+0000","last_fullsized":"2024-04-25T10:30:59.589520+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:33.034260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795138+0000","last_change":"2024-04-25T10:30:32.434410+0000","last_active":"2024-04-25T10:30:34.795138+0000","last_peered":"2024-04-25T10:30:34.795138+0000","last_clean":"2024-04-25T10:30:34.795138+0000","last_became_active":"2024-04-25T10:30:32.434093+0000","last_became_peered":"2024-04-25T10:30:32.434093+0000","last_unstale":"2024-04-25T10:30:34.795138+0000","last_undegraded":"2024-04-25T10:30:34.795138+0000","last_fullsized":"2024-04-25T10:30:34.795138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:46.964704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'11","reported_seq":25,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:59.589428+0000","last_change":"2024-04-25T10:30:31.418360+0000","last_active":"2024-04-25T10:30:59.589428+0000","last_peered":"2024-04-25T10:30:59.589428+0000","last_clean":"2024-04-25T10:30:59.589428+0000","last_became_active":"2024-04-25T10:30:31.418005+0000","last_became_peered":"2024-04-25T10:30:31.418005+0000","last_unstale":"2024-04-25T10:30:59.589428+0000","last_undegraded":"2024-04-25T10:30:59.589428+0000","last_fullsized":"2024-04-25T10:30:59.589428+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:15.556859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.196705+0000","last_change":"2024-04-25T10:30:32.439515+0000","last_active":"2024-04-25T10:30:34.196705+0000","last_peered":"2024-04-25T10:30:34.196705+0000","last_clean":"2024-04-25T10:30:34.196705+0000","last_became_active":"2024-04-25T10:30:32.439101+0000","last_became_peered":"2024-04-25T10:30:32.439101+0000","last_unstale":"2024-04-25T10:30:34.196705+0000","last_undegraded":"2024-04-25T10:30:34.196705+0000","last_fullsized":"2024-04-25T10:30:34.196705+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:33.797903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:37.434282+0000","last_change":"2024-04-25T10:30:31.418070+0000","last_active":"2024-04-25T10:30:37.434282+0000","last_peered":"2024-04-25T10:30:37.434282+0000","last_clean":"2024-04-25T10:30:37.434282+0000","last_became_active":"2024-04-25T10:30:31.417737+0000","last_became_peered":"2024-04-25T10:30:31.417737+0000","last_unstale":"2024-04-25T10:30:37.434282+0000","last_undegraded":"2024-04-25T10:30:37.434282+0000","last_fullsized":"2024-04-25T10:30:37.434282+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:05.664855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415785+0000","last_change":"2024-04-25T10:30:32.435258+0000","last_active":"2024-04-25T10:30:33.415785+0000","last_peered":"2024-04-25T10:30:33.415785+0000","last_clean":"2024-04-25T10:30:33.415785+0000","last_became_active":"2024-04-25T10:30:32.434930+0000","last_became_peered":"2024-04-25T10:30:32.434930+0000","last_unstale":"2024-04-25T10:30:33.415785+0000","last_undegraded":"2024-04-25T10:30:33.415785+0000","last_fullsized":"2024-04-25T10:30:33.415785+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:22.727778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'13","reported_seq":27,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:59.589618+0000","last_change":"2024-04-25T10:30:31.415611+0000","last_active":"2024-04-25T10:30:59.589618+0000","last_peered":"2024-04-25T10:30:59.589618+0000","last_clean":"2024-04-25T10:30:59.589618+0000","last_became_active":"2024-04-25T10:30:31.415071+0000","last_became_peered":"2024-04-25T10:30:31.415071+0000","last_unstale":"2024-04-25T10:30:59.589618+0000","last_undegraded":"2024-04-25T10:30:59.589618+0000","last_fullsized":"2024-04-25T10:30:59.589618+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:30:54.077864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":8034,"num_objects":97,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":204,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":65,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":262144,"data_stored":16068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":178,"ondisk_log_size":178,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574861,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":835584,"data_stored":663402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574861,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":233472,"data_stored":73336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574862,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":253952,"data_stored":74866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574862,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":229376,"data_stored":72264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574861,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":237568,"data_stored":73370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574861,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":237568,"data_stored":72396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574861,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":233472,"data_stored":72466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574861,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":827392,"data_stored":662176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:31:11.488 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:31:11.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.630+0000 7f774bc67640 1 Processor -- start 2024-04-25T10:31:11.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.630+0000 7f774bc67640 1 -- start start 2024-04-25T10:31:11.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.631+0000 7f774bc67640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77440c7e30 0x7f77440c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:11.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.631+0000 7f774bc67640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f7744156ef0 0x7f77440b4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:11.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.631+0000 7f774bc67640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77440ce630 con 0x7f774406d000 2024-04-25T10:31:11.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.631+0000 7f774bc67640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f77440c7c60 con 0x7f7744156ef0 2024-04-25T10:31:11.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.631+0000 7f774bc67640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f77440c7980 con 0x7f77440c7e30 2024-04-25T10:31:11.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.632+0000 7f7749202640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f7744156ef0 0x7f77440b4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:11.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.632+0000 7f7749202640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f7744156ef0 0x7f77440b4ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59190/0 (socket says 172.21.15.106:59190) 2024-04-25T10:31:11.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.632+0000 7f7749202640 1 -- 172.21.15.106:0/2714962312 learned_addr learned my addr 172.21.15.106:0/2714962312 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:31:11.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.632+0000 7f7749a03640 1 --2- 172.21.15.106:0/2714962312 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77440c7e30 0x7f77440c8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:11.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.632+0000 7f7749202640 1 -- 172.21.15.106:0/2714962312 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77440c7e30 msgr2=0x7f77440c8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:11.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.632+0000 7f7749202640 1 --2- 172.21.15.106:0/2714962312 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77440c7e30 0x7f77440c8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.632+0000 7f7749202640 1 -- 172.21.15.106:0/2714962312 >> v1:172.21.15.134:6789/0 conn(0x7f774406d000 legacy=0x7f774415ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:11.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.632+0000 7f7749202640 1 -- 172.21.15.106:0/2714962312 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77440c7010 con 0x7f7744156ef0 2024-04-25T10:31:11.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.632+0000 7f7748a01640 1 -- 172.21.15.106:0/2714962312 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 512671302 0 0) 0x7f77440ce630 con 0x7f774406d000 2024-04-25T10:31:11.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.633+0000 7f7749202640 1 --2- 172.21.15.106:0/2714962312 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7744156ef0 0x7f77440b4ce0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f77440c7de0 tx=0x7f773402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab9c412810850ac8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:11.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.633+0000 7f7748a01640 1 -- 172.21.15.106:0/2714962312 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7734037070 con 0x7f7744156ef0 2024-04-25T10:31:11.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.633+0000 7f7748a01640 1 -- 172.21.15.106:0/2714962312 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7734032440 con 0x7f7744156ef0 2024-04-25T10:31:11.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.633+0000 7f7748a01640 1 -- 172.21.15.106:0/2714962312 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7734032810 con 0x7f7744156ef0 2024-04-25T10:31:11.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.634+0000 7f774bc67640 1 -- 172.21.15.106:0/2714962312 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7744156ef0 msgr2=0x7f77440b4ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:11.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.634+0000 7f774bc67640 1 --2- 172.21.15.106:0/2714962312 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7744156ef0 0x7f77440b4ce0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f77440c7de0 tx=0x7f773402a1a0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.634+0000 7f774bc67640 1 -- 172.21.15.106:0/2714962312 shutdown_connections 2024-04-25T10:31:11.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.634+0000 7f774bc67640 1 --2- 172.21.15.106:0/2714962312 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7744156ef0 0x7f77440b4ce0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.634+0000 7f774bc67640 1 --2- 172.21.15.106:0/2714962312 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77440c7e30 0x7f77440c8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.634+0000 7f774bc67640 1 -- 172.21.15.106:0/2714962312 >> 172.21.15.106:0/2714962312 conn(0x7f7744068b10 msgr2=0x7f77440d1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:11.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.634+0000 7f774bc67640 1 -- 172.21.15.106:0/2714962312 shutdown_connections 2024-04-25T10:31:11.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.634+0000 7f774bc67640 1 -- 172.21.15.106:0/2714962312 wait complete. 2024-04-25T10:31:11.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.636+0000 7f774bc67640 1 Processor -- start 2024-04-25T10:31:11.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.636+0000 7f774bc67640 1 -- start start 2024-04-25T10:31:11.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.637+0000 7f774bc67640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f774406d000 0x7f77441f6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:11.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.637+0000 7f774bc67640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77440c7e30 0x7f77440d0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:11.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.637+0000 7f774bc67640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7744156ef0 0x7f7744152ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:11.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.637+0000 7f774bc67640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f774415d6c0 con 0x7f7744156ef0 2024-04-25T10:31:11.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.637+0000 7f774bc67640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f774415d540 con 0x7f77440c7e30 2024-04-25T10:31:11.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.637+0000 7f774bc67640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f774415e6b0 con 0x7f774406d000 2024-04-25T10:31:11.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.637+0000 7f774a204640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7744156ef0 0x7f7744152ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:11.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.637+0000 7f774a204640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7744156ef0 0x7f7744152ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59202/0 (socket says 172.21.15.106:59202) 2024-04-25T10:31:11.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.637+0000 7f774a204640 1 -- 172.21.15.106:0/2661411629 learned_addr learned my addr 172.21.15.106:0/2661411629 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:31:11.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.637+0000 7f7749a03640 1 --2- 172.21.15.106:0/2661411629 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f774406d000 0x7f77441f6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:11.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.637+0000 7f7749202640 1 --2- 172.21.15.106:0/2661411629 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77440c7e30 0x7f77440d0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:11.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.637+0000 7f774a204640 1 -- 172.21.15.106:0/2661411629 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f774406d000 msgr2=0x7f77441f6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:11.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.637+0000 7f774a204640 1 --2- 172.21.15.106:0/2661411629 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f774406d000 0x7f77441f6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.638+0000 7f774a204640 1 -- 172.21.15.106:0/2661411629 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77440c7e30 msgr2=0x7f77440d0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:11.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.638+0000 7f774a204640 1 --2- 172.21.15.106:0/2661411629 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77440c7e30 0x7f77440d0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.638+0000 7f774a204640 1 -- 172.21.15.106:0/2661411629 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77441f6d70 con 0x7f7744156ef0 2024-04-25T10:31:11.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.638+0000 7f774a204640 1 --2- 172.21.15.106:0/2661411629 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7744156ef0 0x7f7744152ea0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f773c00c7b0 tx=0x7f773c00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:11.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.638+0000 7f773a7fc640 1 -- 172.21.15.106:0/2661411629 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f773c003ba0 con 0x7f7744156ef0 2024-04-25T10:31:11.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.638+0000 7f773a7fc640 1 -- 172.21.15.106:0/2661411629 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f773c003d40 con 0x7f7744156ef0 2024-04-25T10:31:11.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.638+0000 7f773a7fc640 1 -- 172.21.15.106:0/2661411629 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f773c005270 con 0x7f7744156ef0 2024-04-25T10:31:11.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.638+0000 7f774bc67640 1 -- 172.21.15.106:0/2661411629 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f774415e900 con 0x7f7744156ef0 2024-04-25T10:31:11.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.638+0000 7f774bc67640 1 -- 172.21.15.106:0/2661411629 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f774415eec0 con 0x7f7744156ef0 2024-04-25T10:31:11.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.639+0000 7f774bc67640 1 -- 172.21.15.106:0/2661411629 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f774406a3d0 con 0x7f7744156ef0 2024-04-25T10:31:11.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.639+0000 7f773a7fc640 1 -- 172.21.15.106:0/2661411629 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f773c005410 con 0x7f7744156ef0 2024-04-25T10:31:11.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.640+0000 7f773a7fc640 1 --2- 172.21.15.106:0/2661411629 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f77280744e0 0x7f77280769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:11.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.640+0000 7f773a7fc640 1 -- 172.21.15.106:0/2661411629 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f773c094f20 con 0x7f7744156ef0 2024-04-25T10:31:11.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.640+0000 7f7749a03640 1 --2- 172.21.15.106:0/2661411629 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f77280744e0 0x7f77280769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:11.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.640+0000 7f7749a03640 1 --2- 172.21.15.106:0/2661411629 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f77280744e0 0x7f77280769f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7740009aa0 tx=0x7f7740009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:11.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.644+0000 7f773a7fc640 1 -- 172.21.15.106:0/2661411629 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f773c05f0f0 con 0x7f7744156ef0 2024-04-25T10:31:11.820 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.818+0000 7f774bc67640 1 -- 172.21.15.106:0/2661411629 --> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f774415a5a0 con 0x7f77280744e0 2024-04-25T10:31:11.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.824+0000 7f773a7fc640 1 -- 172.21.15.106:0/2661411629 <== mgr.4107 v2:172.21.15.106:6832/1744210167 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251915 (secure 0 0 0) 0x7f774415a5a0 con 0x7f77280744e0 2024-04-25T10:31:11.828 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:31:11.830 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-25T10:31:11.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.827+0000 7f774bc67640 1 -- 172.21.15.106:0/2661411629 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f77280744e0 msgr2=0x7f77280769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:11.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.827+0000 7f774bc67640 1 --2- 172.21.15.106:0/2661411629 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f77280744e0 0x7f77280769f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7740009aa0 tx=0x7f7740009190 comp rx=0 tx=0).stop 2024-04-25T10:31:11.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.827+0000 7f774bc67640 1 -- 172.21.15.106:0/2661411629 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7744156ef0 msgr2=0x7f7744152ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:11.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.827+0000 7f774bc67640 1 --2- 172.21.15.106:0/2661411629 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7744156ef0 0x7f7744152ea0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f773c00c7b0 tx=0x7f773c00cbb0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.827+0000 7f774bc67640 1 -- 172.21.15.106:0/2661411629 shutdown_connections 2024-04-25T10:31:11.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.827+0000 7f774bc67640 1 --2- 172.21.15.106:0/2661411629 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f77280744e0 0x7f77280769f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.827+0000 7f774bc67640 1 --2- 172.21.15.106:0/2661411629 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7744156ef0 0x7f7744152ea0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.827+0000 7f774bc67640 1 --2- 172.21.15.106:0/2661411629 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77440c7e30 0x7f77440d0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.827+0000 7f774bc67640 1 --2- 172.21.15.106:0/2661411629 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f774406d000 0x7f77441f6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.827+0000 7f774bc67640 1 -- 172.21.15.106:0/2661411629 >> 172.21.15.106:0/2661411629 conn(0x7f7744068b10 msgr2=0x7f7744154e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:11.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.827+0000 7f774bc67640 1 -- 172.21.15.106:0/2661411629 shutdown_connections 2024-04-25T10:31:11.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.827+0000 7f774bc67640 1 -- 172.21.15.106:0/2661411629 wait complete. 2024-04-25T10:31:11.840 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2024-04-25T10:31:10.105789+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":598402,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":305,"num_write_kb":1690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":65,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":250,"ondisk_log_size":250,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219144,"kb_used_data":3016,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755576,"statfs":{"total":773094113280,"available":772869709824,"internally_reserved":0,"allocated":3088384,"data_stored":1764276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1406,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002715"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795275+0000","last_change":"2024-04-25T10:30:32.440983+0000","last_active":"2024-04-25T10:30:34.795275+0000","last_peered":"2024-04-25T10:30:34.795275+0000","last_clean":"2024-04-25T10:30:34.795275+0000","last_became_active":"2024-04-25T10:30:32.440552+0000","last_became_peered":"2024-04-25T10:30:32.440552+0000","last_unstale":"2024-04-25T10:30:34.795275+0000","last_undegraded":"2024-04-25T10:30:34.795275+0000","last_fullsized":"2024-04-25T10:30:34.795275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:47.963061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.412686+0000","last_change":"2024-04-25T10:30:32.425695+0000","last_active":"2024-04-25T10:30:33.412686+0000","last_peered":"2024-04-25T10:30:33.412686+0000","last_clean":"2024-04-25T10:30:33.412686+0000","last_became_active":"2024-04-25T10:30:32.425368+0000","last_became_peered":"2024-04-25T10:30:32.425368+0000","last_unstale":"2024-04-25T10:30:33.412686+0000","last_undegraded":"2024-04-25T10:30:33.412686+0000","last_fullsized":"2024-04-25T10:30:33.412686+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:39.120522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.196620+0000","last_change":"2024-04-25T10:30:32.430144+0000","last_active":"2024-04-25T10:30:34.196620+0000","last_peered":"2024-04-25T10:30:34.196620+0000","last_clean":"2024-04-25T10:30:34.196620+0000","last_became_active":"2024-04-25T10:30:32.429754+0000","last_became_peered":"2024-04-25T10:30:32.429754+0000","last_unstale":"2024-04-25T10:30:34.196620+0000","last_undegraded":"2024-04-25T10:30:34.196620+0000","last_fullsized":"2024-04-25T10:30:34.196620+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:11.448907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794948+0000","last_change":"2024-04-25T10:30:32.442478+0000","last_active":"2024-04-25T10:30:34.794948+0000","last_peered":"2024-04-25T10:30:34.794948+0000","last_clean":"2024-04-25T10:30:34.794948+0000","last_became_active":"2024-04-25T10:30:32.442182+0000","last_became_peered":"2024-04-25T10:30:32.442182+0000","last_unstale":"2024-04-25T10:30:34.794948+0000","last_undegraded":"2024-04-25T10:30:34.794948+0000","last_fullsized":"2024-04-25T10:30:34.794948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:22.185978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794683+0000","last_change":"2024-04-25T10:30:32.438231+0000","last_active":"2024-04-25T10:30:34.794683+0000","last_peered":"2024-04-25T10:30:34.794683+0000","last_clean":"2024-04-25T10:30:34.794683+0000","last_became_active":"2024-04-25T10:30:32.437656+0000","last_became_peered":"2024-04-25T10:30:32.437656+0000","last_unstale":"2024-04-25T10:30:34.794683+0000","last_undegraded":"2024-04-25T10:30:34.794683+0000","last_fullsized":"2024-04-25T10:30:34.794683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:56.915999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795645+0000","last_change":"2024-04-25T10:30:32.439221+0000","last_active":"2024-04-25T10:30:34.795645+0000","last_peered":"2024-04-25T10:30:34.795645+0000","last_clean":"2024-04-25T10:30:34.795645+0000","last_became_active":"2024-04-25T10:30:32.438923+0000","last_became_peered":"2024-04-25T10:30:32.438923+0000","last_unstale":"2024-04-25T10:30:34.795645+0000","last_undegraded":"2024-04-25T10:30:34.795645+0000","last_fullsized":"2024-04-25T10:30:34.795645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:40.469409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794157+0000","last_change":"2024-04-25T10:30:32.429453+0000","last_active":"2024-04-25T10:30:34.794157+0000","last_peered":"2024-04-25T10:30:34.794157+0000","last_clean":"2024-04-25T10:30:34.794157+0000","last_became_active":"2024-04-25T10:30:32.429131+0000","last_became_peered":"2024-04-25T10:30:32.429131+0000","last_unstale":"2024-04-25T10:30:34.794157+0000","last_undegraded":"2024-04-25T10:30:34.794157+0000","last_fullsized":"2024-04-25T10:30:34.794157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:26.785922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415260+0000","last_change":"2024-04-25T10:30:32.438860+0000","last_active":"2024-04-25T10:30:33.415260+0000","last_peered":"2024-04-25T10:30:33.415260+0000","last_clean":"2024-04-25T10:30:33.415260+0000","last_became_active":"2024-04-25T10:30:32.438491+0000","last_became_peered":"2024-04-25T10:30:32.438491+0000","last_unstale":"2024-04-25T10:30:33.415260+0000","last_undegraded":"2024-04-25T10:30:33.415260+0000","last_fullsized":"2024-04-25T10:30:33.415260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:08.610707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795467+0000","last_change":"2024-04-25T10:30:32.440331+0000","last_active":"2024-04-25T10:30:34.795467+0000","last_peered":"2024-04-25T10:30:34.795467+0000","last_clean":"2024-04-25T10:30:34.795467+0000","last_became_active":"2024-04-25T10:30:32.440043+0000","last_became_peered":"2024-04-25T10:30:32.440043+0000","last_unstale":"2024-04-25T10:30:34.795467+0000","last_undegraded":"2024-04-25T10:30:34.795467+0000","last_fullsized":"2024-04-25T10:30:34.795467+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:17.684685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794636+0000","last_change":"2024-04-25T10:30:32.435388+0000","last_active":"2024-04-25T10:30:34.794636+0000","last_peered":"2024-04-25T10:30:34.794636+0000","last_clean":"2024-04-25T10:30:34.794636+0000","last_became_active":"2024-04-25T10:30:32.434983+0000","last_became_peered":"2024-04-25T10:30:32.434983+0000","last_unstale":"2024-04-25T10:30:34.794636+0000","last_undegraded":"2024-04-25T10:30:34.794636+0000","last_fullsized":"2024-04-25T10:30:34.794636+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:39.256604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.793928+0000","last_change":"2024-04-25T10:30:32.435884+0000","last_active":"2024-04-25T10:30:34.793928+0000","last_peered":"2024-04-25T10:30:34.793928+0000","last_clean":"2024-04-25T10:30:34.793928+0000","last_became_active":"2024-04-25T10:30:32.434645+0000","last_became_peered":"2024-04-25T10:30:32.434645+0000","last_unstale":"2024-04-25T10:30:34.793928+0000","last_undegraded":"2024-04-25T10:30:34.793928+0000","last_fullsized":"2024-04-25T10:30:34.793928+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:07.778437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795720+0000","last_change":"2024-04-25T10:30:32.436403+0000","last_active":"2024-04-25T10:30:34.795720+0000","last_peered":"2024-04-25T10:30:34.795720+0000","last_clean":"2024-04-25T10:30:34.795720+0000","last_became_active":"2024-04-25T10:30:32.435908+0000","last_became_peered":"2024-04-25T10:30:32.435908+0000","last_unstale":"2024-04-25T10:30:34.795720+0000","last_undegraded":"2024-04-25T10:30:34.795720+0000","last_fullsized":"2024-04-25T10:30:34.795720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:03.339316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415872+0000","last_change":"2024-04-25T10:30:32.432110+0000","last_active":"2024-04-25T10:30:33.415872+0000","last_peered":"2024-04-25T10:30:33.415872+0000","last_clean":"2024-04-25T10:30:33.415872+0000","last_became_active":"2024-04-25T10:30:32.431792+0000","last_became_peered":"2024-04-25T10:30:32.431792+0000","last_unstale":"2024-04-25T10:30:33.415872+0000","last_undegraded":"2024-04-25T10:30:33.415872+0000","last_fullsized":"2024-04-25T10:30:33.415872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:34.185989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.196476+0000","last_change":"2024-04-25T10:30:32.433676+0000","last_active":"2024-04-25T10:30:34.196476+0000","last_peered":"2024-04-25T10:30:34.196476+0000","last_clean":"2024-04-25T10:30:34.196476+0000","last_became_active":"2024-04-25T10:30:32.433329+0000","last_became_peered":"2024-04-25T10:30:32.433329+0000","last_unstale":"2024-04-25T10:30:34.196476+0000","last_undegraded":"2024-04-25T10:30:34.196476+0000","last_fullsized":"2024-04-25T10:30:34.196476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:59.942368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.412754+0000","last_change":"2024-04-25T10:30:32.423582+0000","last_active":"2024-04-25T10:30:33.412754+0000","last_peered":"2024-04-25T10:30:33.412754+0000","last_clean":"2024-04-25T10:30:33.412754+0000","last_became_active":"2024-04-25T10:30:32.422991+0000","last_became_peered":"2024-04-25T10:30:32.422991+0000","last_unstale":"2024-04-25T10:30:33.412754+0000","last_undegraded":"2024-04-25T10:30:33.412754+0000","last_fullsized":"2024-04-25T10:30:33.412754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:11:47.132928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794545+0000","last_change":"2024-04-25T10:30:32.440235+0000","last_active":"2024-04-25T10:30:34.794545+0000","last_peered":"2024-04-25T10:30:34.794545+0000","last_clean":"2024-04-25T10:30:34.794545+0000","last_became_active":"2024-04-25T10:30:32.439191+0000","last_became_peered":"2024-04-25T10:30:32.439191+0000","last_unstale":"2024-04-25T10:30:34.794545+0000","last_undegraded":"2024-04-25T10:30:34.794545+0000","last_fullsized":"2024-04-25T10:30:34.794545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:51.423086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795170+0000","last_change":"2024-04-25T10:30:32.442418+0000","last_active":"2024-04-25T10:30:34.795170+0000","last_peered":"2024-04-25T10:30:34.795170+0000","last_clean":"2024-04-25T10:30:34.795170+0000","last_became_active":"2024-04-25T10:30:32.442125+0000","last_became_peered":"2024-04-25T10:30:32.442125+0000","last_unstale":"2024-04-25T10:30:34.795170+0000","last_undegraded":"2024-04-25T10:30:34.795170+0000","last_fullsized":"2024-04-25T10:30:34.795170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:14.213809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415344+0000","last_change":"2024-04-25T10:30:32.434758+0000","last_active":"2024-04-25T10:30:33.415344+0000","last_peered":"2024-04-25T10:30:33.415344+0000","last_clean":"2024-04-25T10:30:33.415344+0000","last_became_active":"2024-04-25T10:30:32.434492+0000","last_became_peered":"2024-04-25T10:30:32.434492+0000","last_unstale":"2024-04-25T10:30:33.415344+0000","last_undegraded":"2024-04-25T10:30:33.415344+0000","last_fullsized":"2024-04-25T10:30:33.415344+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:41.454407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.796066+0000","last_change":"2024-04-25T10:30:32.440399+0000","last_active":"2024-04-25T10:30:34.796066+0000","last_peered":"2024-04-25T10:30:34.796066+0000","last_clean":"2024-04-25T10:30:34.796066+0000","last_became_active":"2024-04-25T10:30:32.438227+0000","last_became_peered":"2024-04-25T10:30:32.438227+0000","last_unstale":"2024-04-25T10:30:34.796066+0000","last_undegraded":"2024-04-25T10:30:34.796066+0000","last_fullsized":"2024-04-25T10:30:34.796066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:49.571671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794149+0000","last_change":"2024-04-25T10:30:32.431609+0000","last_active":"2024-04-25T10:30:34.794149+0000","last_peered":"2024-04-25T10:30:34.794149+0000","last_clean":"2024-04-25T10:30:34.794149+0000","last_became_active":"2024-04-25T10:30:32.431314+0000","last_became_peered":"2024-04-25T10:30:32.431314+0000","last_unstale":"2024-04-25T10:30:34.794149+0000","last_undegraded":"2024-04-25T10:30:34.794149+0000","last_fullsized":"2024-04-25T10:30:34.794149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:33.421989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.196668+0000","last_change":"2024-04-25T10:30:32.432612+0000","last_active":"2024-04-25T10:30:34.196668+0000","last_peered":"2024-04-25T10:30:34.196668+0000","last_clean":"2024-04-25T10:30:34.196668+0000","last_became_active":"2024-04-25T10:30:32.432271+0000","last_became_peered":"2024-04-25T10:30:32.432271+0000","last_unstale":"2024-04-25T10:30:34.196668+0000","last_undegraded":"2024-04-25T10:30:34.196668+0000","last_fullsized":"2024-04-25T10:30:34.196668+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:14.281493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795827+0000","last_change":"2024-04-25T10:30:32.444618+0000","last_active":"2024-04-25T10:30:34.795827+0000","last_peered":"2024-04-25T10:30:34.795827+0000","last_clean":"2024-04-25T10:30:34.795827+0000","last_became_active":"2024-04-25T10:30:32.444273+0000","last_became_peered":"2024-04-25T10:30:32.444273+0000","last_unstale":"2024-04-25T10:30:34.795827+0000","last_undegraded":"2024-04-25T10:30:34.795827+0000","last_fullsized":"2024-04-25T10:30:34.795827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:09.295251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415348+0000","last_change":"2024-04-25T10:30:32.426578+0000","last_active":"2024-04-25T10:30:33.415348+0000","last_peered":"2024-04-25T10:30:33.415348+0000","last_clean":"2024-04-25T10:30:33.415348+0000","last_became_active":"2024-04-25T10:30:32.426223+0000","last_became_peered":"2024-04-25T10:30:32.426223+0000","last_unstale":"2024-04-25T10:30:33.415348+0000","last_undegraded":"2024-04-25T10:30:33.415348+0000","last_fullsized":"2024-04-25T10:30:33.415348+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:53.700585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794721+0000","last_change":"2024-04-25T10:30:32.439767+0000","last_active":"2024-04-25T10:30:34.794721+0000","last_peered":"2024-04-25T10:30:34.794721+0000","last_clean":"2024-04-25T10:30:34.794721+0000","last_became_active":"2024-04-25T10:30:32.438801+0000","last_became_peered":"2024-04-25T10:30:32.438801+0000","last_unstale":"2024-04-25T10:30:34.794721+0000","last_undegraded":"2024-04-25T10:30:34.794721+0000","last_fullsized":"2024-04-25T10:30:34.794721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:58.231512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415544+0000","last_change":"2024-04-25T10:30:32.427424+0000","last_active":"2024-04-25T10:30:33.415544+0000","last_peered":"2024-04-25T10:30:33.415544+0000","last_clean":"2024-04-25T10:30:33.415544+0000","last_became_active":"2024-04-25T10:30:32.427081+0000","last_became_peered":"2024-04-25T10:30:32.427081+0000","last_unstale":"2024-04-25T10:30:33.415544+0000","last_undegraded":"2024-04-25T10:30:33.415544+0000","last_fullsized":"2024-04-25T10:30:33.415544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:47.372451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794727+0000","last_change":"2024-04-25T10:30:32.432472+0000","last_active":"2024-04-25T10:30:34.794727+0000","last_peered":"2024-04-25T10:30:34.794727+0000","last_clean":"2024-04-25T10:30:34.794727+0000","last_became_active":"2024-04-25T10:30:32.432121+0000","last_became_peered":"2024-04-25T10:30:32.432121+0000","last_unstale":"2024-04-25T10:30:34.794727+0000","last_undegraded":"2024-04-25T10:30:34.794727+0000","last_fullsized":"2024-04-25T10:30:34.794727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:34.191339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.412571+0000","last_change":"2024-04-25T10:30:32.425108+0000","last_active":"2024-04-25T10:30:33.412571+0000","last_peered":"2024-04-25T10:30:33.412571+0000","last_clean":"2024-04-25T10:30:33.412571+0000","last_became_active":"2024-04-25T10:30:32.424794+0000","last_became_peered":"2024-04-25T10:30:32.424794+0000","last_unstale":"2024-04-25T10:30:33.412571+0000","last_undegraded":"2024-04-25T10:30:33.412571+0000","last_fullsized":"2024-04-25T10:30:33.412571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:01.282758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415734+0000","last_change":"2024-04-25T10:30:32.426672+0000","last_active":"2024-04-25T10:30:33.415734+0000","last_peered":"2024-04-25T10:30:33.415734+0000","last_clean":"2024-04-25T10:30:33.415734+0000","last_became_active":"2024-04-25T10:30:32.426348+0000","last_became_peered":"2024-04-25T10:30:32.426348+0000","last_unstale":"2024-04-25T10:30:33.415734+0000","last_undegraded":"2024-04-25T10:30:33.415734+0000","last_fullsized":"2024-04-25T10:30:33.415734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:57.645772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.196609+0000","last_change":"2024-04-25T10:30:32.427989+0000","last_active":"2024-04-25T10:30:34.196609+0000","last_peered":"2024-04-25T10:30:34.196609+0000","last_clean":"2024-04-25T10:30:34.196609+0000","last_became_active":"2024-04-25T10:30:32.427676+0000","last_became_peered":"2024-04-25T10:30:32.427676+0000","last_unstale":"2024-04-25T10:30:34.196609+0000","last_undegraded":"2024-04-25T10:30:34.196609+0000","last_fullsized":"2024-04-25T10:30:34.196609+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:15:02.332186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795935+0000","last_change":"2024-04-25T10:30:32.441112+0000","last_active":"2024-04-25T10:30:34.795935+0000","last_peered":"2024-04-25T10:30:34.795935+0000","last_clean":"2024-04-25T10:30:34.795935+0000","last_became_active":"2024-04-25T10:30:32.440815+0000","last_became_peered":"2024-04-25T10:30:32.440815+0000","last_unstale":"2024-04-25T10:30:34.795935+0000","last_undegraded":"2024-04-25T10:30:34.795935+0000","last_fullsized":"2024-04-25T10:30:34.795935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:02.956759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415178+0000","last_change":"2024-04-25T10:30:32.430902+0000","last_active":"2024-04-25T10:30:33.415178+0000","last_peered":"2024-04-25T10:30:33.415178+0000","last_clean":"2024-04-25T10:30:33.415178+0000","last_became_active":"2024-04-25T10:30:32.430615+0000","last_became_peered":"2024-04-25T10:30:32.430615+0000","last_unstale":"2024-04-25T10:30:33.415178+0000","last_undegraded":"2024-04-25T10:30:33.415178+0000","last_fullsized":"2024-04-25T10:30:33.415178+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:12.747595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795247+0000","last_change":"2024-04-25T10:30:32.435379+0000","last_active":"2024-04-25T10:30:34.795247+0000","last_peered":"2024-04-25T10:30:34.795247+0000","last_clean":"2024-04-25T10:30:34.795247+0000","last_became_active":"2024-04-25T10:30:32.435055+0000","last_became_peered":"2024-04-25T10:30:32.435055+0000","last_unstale":"2024-04-25T10:30:34.795247+0000","last_undegraded":"2024-04-25T10:30:34.795247+0000","last_fullsized":"2024-04-25T10:30:34.795247+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:13.623762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794490+0000","last_change":"2024-04-25T10:30:32.422102+0000","last_active":"2024-04-25T10:30:34.794490+0000","last_peered":"2024-04-25T10:30:34.794490+0000","last_clean":"2024-04-25T10:30:34.794490+0000","last_became_active":"2024-04-25T10:30:32.421766+0000","last_became_peered":"2024-04-25T10:30:32.421766+0000","last_unstale":"2024-04-25T10:30:34.794490+0000","last_undegraded":"2024-04-25T10:30:34.794490+0000","last_fullsized":"2024-04-25T10:30:34.794490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:30.300572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:43.470932+0000","last_change":"2024-04-25T10:30:31.418065+0000","last_active":"2024-04-25T10:30:43.470932+0000","last_peered":"2024-04-25T10:30:43.470932+0000","last_clean":"2024-04-25T10:30:43.470932+0000","last_became_active":"2024-04-25T10:30:31.417451+0000","last_became_peered":"2024-04-25T10:30:31.417451+0000","last_unstale":"2024-04-25T10:30:43.470932+0000","last_undegraded":"2024-04-25T10:30:43.470932+0000","last_fullsized":"2024-04-25T10:30:43.470932+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:17.262861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.196777+0000","last_change":"2024-04-25T10:30:32.426913+0000","last_active":"2024-04-25T10:30:34.196777+0000","last_peered":"2024-04-25T10:30:34.196777+0000","last_clean":"2024-04-25T10:30:34.196777+0000","last_became_active":"2024-04-25T10:30:32.426616+0000","last_became_peered":"2024-04-25T10:30:32.426616+0000","last_unstale":"2024-04-25T10:30:34.196777+0000","last_undegraded":"2024-04-25T10:30:34.196777+0000","last_fullsized":"2024-04-25T10:30:34.196777+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:50.614932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'7","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:53.568128+0000","last_change":"2024-04-25T10:30:31.416432+0000","last_active":"2024-04-25T10:30:53.568128+0000","last_peered":"2024-04-25T10:30:53.568128+0000","last_clean":"2024-04-25T10:30:53.568128+0000","last_became_active":"2024-04-25T10:30:31.415112+0000","last_became_peered":"2024-04-25T10:30:31.415112+0000","last_unstale":"2024-04-25T10:30:53.568128+0000","last_undegraded":"2024-04-25T10:30:53.568128+0000","last_fullsized":"2024-04-25T10:30:53.568128+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:22.674365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"20'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:48.494314+0000","last_change":"2024-04-25T10:30:31.418059+0000","last_active":"2024-04-25T10:30:48.494314+0000","last_peered":"2024-04-25T10:30:48.494314+0000","last_clean":"2024-04-25T10:30:48.494314+0000","last_became_active":"2024-04-25T10:30:31.417716+0000","last_became_peered":"2024-04-25T10:30:31.417716+0000","last_unstale":"2024-04-25T10:30:48.494314+0000","last_undegraded":"2024-04-25T10:30:48.494314+0000","last_fullsized":"2024-04-25T10:30:48.494314+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:57.752218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415428+0000","last_change":"2024-04-25T10:30:32.428078+0000","last_active":"2024-04-25T10:30:33.415428+0000","last_peered":"2024-04-25T10:30:33.415428+0000","last_clean":"2024-04-25T10:30:33.415428+0000","last_became_active":"2024-04-25T10:30:32.427700+0000","last_became_peered":"2024-04-25T10:30:32.427700+0000","last_unstale":"2024-04-25T10:30:33.415428+0000","last_undegraded":"2024-04-25T10:30:33.415428+0000","last_fullsized":"2024-04-25T10:30:33.415428+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:26.931791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"20'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:53.568903+0000","last_change":"2024-04-25T10:30:31.415030+0000","last_active":"2024-04-25T10:30:53.568903+0000","last_peered":"2024-04-25T10:30:53.568903+0000","last_clean":"2024-04-25T10:30:53.568903+0000","last_became_active":"2024-04-25T10:30:31.414542+0000","last_became_peered":"2024-04-25T10:30:31.414542+0000","last_unstale":"2024-04-25T10:30:53.568903+0000","last_undegraded":"2024-04-25T10:30:53.568903+0000","last_fullsized":"2024-04-25T10:30:53.568903+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:20.082420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.412471+0000","last_change":"2024-04-25T10:30:32.425207+0000","last_active":"2024-04-25T10:30:33.412471+0000","last_peered":"2024-04-25T10:30:33.412471+0000","last_clean":"2024-04-25T10:30:33.412471+0000","last_became_active":"2024-04-25T10:30:32.424849+0000","last_became_peered":"2024-04-25T10:30:32.424849+0000","last_unstale":"2024-04-25T10:30:33.412471+0000","last_undegraded":"2024-04-25T10:30:33.412471+0000","last_fullsized":"2024-04-25T10:30:33.412471+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:11.533686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"20'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:43.473310+0000","last_change":"2024-04-25T10:30:31.407700+0000","last_active":"2024-04-25T10:30:43.473310+0000","last_peered":"2024-04-25T10:30:43.473310+0000","last_clean":"2024-04-25T10:30:43.473310+0000","last_became_active":"2024-04-25T10:30:31.407376+0000","last_became_peered":"2024-04-25T10:30:31.407376+0000","last_unstale":"2024-04-25T10:30:43.473310+0000","last_undegraded":"2024-04-25T10:30:43.473310+0000","last_fullsized":"2024-04-25T10:30:43.473310+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:50.112742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794157+0000","last_change":"2024-04-25T10:30:32.444599+0000","last_active":"2024-04-25T10:30:34.794157+0000","last_peered":"2024-04-25T10:30:34.794157+0000","last_clean":"2024-04-25T10:30:34.794157+0000","last_became_active":"2024-04-25T10:30:32.444250+0000","last_became_peered":"2024-04-25T10:30:32.444250+0000","last_unstale":"2024-04-25T10:30:34.794157+0000","last_undegraded":"2024-04-25T10:30:34.794157+0000","last_fullsized":"2024-04-25T10:30:34.794157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:55.711525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"20'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:59.589106+0000","last_change":"2024-04-25T10:30:31.405051+0000","last_active":"2024-04-25T10:30:59.589106+0000","last_peered":"2024-04-25T10:30:59.589106+0000","last_clean":"2024-04-25T10:30:59.589106+0000","last_became_active":"2024-04-25T10:30:31.404669+0000","last_became_peered":"2024-04-25T10:30:31.404669+0000","last_unstale":"2024-04-25T10:30:59.589106+0000","last_undegraded":"2024-04-25T10:30:59.589106+0000","last_fullsized":"2024-04-25T10:30:59.589106+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:45.260847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.196763+0000","last_change":"2024-04-25T10:30:32.435804+0000","last_active":"2024-04-25T10:30:34.196763+0000","last_peered":"2024-04-25T10:30:34.196763+0000","last_clean":"2024-04-25T10:30:34.196763+0000","last_became_active":"2024-04-25T10:30:32.435471+0000","last_became_peered":"2024-04-25T10:30:32.435471+0000","last_unstale":"2024-04-25T10:30:34.196763+0000","last_undegraded":"2024-04-25T10:30:34.196763+0000","last_fullsized":"2024-04-25T10:30:34.196763+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:48.213846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:43.470877+0000","last_change":"2024-04-25T10:30:31.403560+0000","last_active":"2024-04-25T10:30:43.470877+0000","last_peered":"2024-04-25T10:30:43.470877+0000","last_clean":"2024-04-25T10:30:43.470877+0000","last_became_active":"2024-04-25T10:30:31.403167+0000","last_became_peered":"2024-04-25T10:30:31.403167+0000","last_unstale":"2024-04-25T10:30:43.470877+0000","last_undegraded":"2024-04-25T10:30:43.470877+0000","last_fullsized":"2024-04-25T10:30:43.470877+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:54.592832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795908+0000","last_change":"2024-04-25T10:30:32.424101+0000","last_active":"2024-04-25T10:30:34.795908+0000","last_peered":"2024-04-25T10:30:34.795908+0000","last_clean":"2024-04-25T10:30:34.795908+0000","last_became_active":"2024-04-25T10:30:32.423780+0000","last_became_peered":"2024-04-25T10:30:32.423780+0000","last_unstale":"2024-04-25T10:30:34.795908+0000","last_undegraded":"2024-04-25T10:30:34.795908+0000","last_fullsized":"2024-04-25T10:30:34.795908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:29.175348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'10","reported_seq":24,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:59.783452+0000","last_change":"2024-04-25T10:30:31.418201+0000","last_active":"2024-04-25T10:30:59.783452+0000","last_peered":"2024-04-25T10:30:59.783452+0000","last_clean":"2024-04-25T10:30:59.783452+0000","last_became_active":"2024-04-25T10:30:31.417824+0000","last_became_peered":"2024-04-25T10:30:31.417824+0000","last_unstale":"2024-04-25T10:30:59.783452+0000","last_undegraded":"2024-04-25T10:30:59.783452+0000","last_fullsized":"2024-04-25T10:30:59.783452+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:38.584225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795830+0000","last_change":"2024-04-25T10:30:32.431139+0000","last_active":"2024-04-25T10:30:34.795830+0000","last_peered":"2024-04-25T10:30:34.795830+0000","last_clean":"2024-04-25T10:30:34.795830+0000","last_became_active":"2024-04-25T10:30:32.429622+0000","last_became_peered":"2024-04-25T10:30:32.429622+0000","last_unstale":"2024-04-25T10:30:34.795830+0000","last_undegraded":"2024-04-25T10:30:34.795830+0000","last_fullsized":"2024-04-25T10:30:34.795830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:39.993647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'9","reported_seq":23,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:59.586443+0000","last_change":"2024-04-25T10:30:31.414775+0000","last_active":"2024-04-25T10:30:59.586443+0000","last_peered":"2024-04-25T10:30:59.586443+0000","last_clean":"2024-04-25T10:30:59.586443+0000","last_became_active":"2024-04-25T10:30:31.414422+0000","last_became_peered":"2024-04-25T10:30:31.414422+0000","last_unstale":"2024-04-25T10:30:59.586443+0000","last_undegraded":"2024-04-25T10:30:59.586443+0000","last_fullsized":"2024-04-25T10:30:59.586443+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:37.803505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.196648+0000","last_change":"2024-04-25T10:30:32.432884+0000","last_active":"2024-04-25T10:30:34.196648+0000","last_peered":"2024-04-25T10:30:34.196648+0000","last_clean":"2024-04-25T10:30:34.196648+0000","last_became_active":"2024-04-25T10:30:32.431743+0000","last_became_peered":"2024-04-25T10:30:32.431743+0000","last_unstale":"2024-04-25T10:30:34.196648+0000","last_undegraded":"2024-04-25T10:30:34.196648+0000","last_fullsized":"2024-04-25T10:30:34.196648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:15:06.737631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:53.568904+0000","last_change":"2024-04-25T10:30:31.419497+0000","last_active":"2024-04-25T10:30:53.568904+0000","last_peered":"2024-04-25T10:30:53.568904+0000","last_clean":"2024-04-25T10:30:53.568904+0000","last_became_active":"2024-04-25T10:30:31.419135+0000","last_became_peered":"2024-04-25T10:30:31.419135+0000","last_unstale":"2024-04-25T10:30:53.568904+0000","last_undegraded":"2024-04-25T10:30:53.568904+0000","last_fullsized":"2024-04-25T10:30:53.568904+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:59.403927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794867+0000","last_change":"2024-04-25T10:30:32.434197+0000","last_active":"2024-04-25T10:30:34.794867+0000","last_peered":"2024-04-25T10:30:34.794867+0000","last_clean":"2024-04-25T10:30:34.794867+0000","last_became_active":"2024-04-25T10:30:32.433623+0000","last_became_peered":"2024-04-25T10:30:32.433623+0000","last_unstale":"2024-04-25T10:30:34.794867+0000","last_undegraded":"2024-04-25T10:30:34.794867+0000","last_fullsized":"2024-04-25T10:30:34.794867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:08.027472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:53.567835+0000","last_change":"2024-04-25T10:30:31.416901+0000","last_active":"2024-04-25T10:30:53.567835+0000","last_peered":"2024-04-25T10:30:53.567835+0000","last_clean":"2024-04-25T10:30:53.567835+0000","last_became_active":"2024-04-25T10:30:31.415706+0000","last_became_peered":"2024-04-25T10:30:31.415706+0000","last_unstale":"2024-04-25T10:30:53.567835+0000","last_undegraded":"2024-04-25T10:30:53.567835+0000","last_fullsized":"2024-04-25T10:30:53.567835+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:23.243039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795344+0000","last_change":"2024-04-25T10:30:32.435089+0000","last_active":"2024-04-25T10:30:34.795344+0000","last_peered":"2024-04-25T10:30:34.795344+0000","last_clean":"2024-04-25T10:30:34.795344+0000","last_became_active":"2024-04-25T10:30:32.434737+0000","last_became_peered":"2024-04-25T10:30:32.434737+0000","last_unstale":"2024-04-25T10:30:34.795344+0000","last_undegraded":"2024-04-25T10:30:34.795344+0000","last_fullsized":"2024-04-25T10:30:34.795344+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:19.738850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'17","reported_seq":31,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:53.570024+0000","last_change":"2024-04-25T10:30:31.405163+0000","last_active":"2024-04-25T10:30:53.570024+0000","last_peered":"2024-04-25T10:30:53.570024+0000","last_clean":"2024-04-25T10:30:53.570024+0000","last_became_active":"2024-04-25T10:30:31.404775+0000","last_became_peered":"2024-04-25T10:30:31.404775+0000","last_unstale":"2024-04-25T10:30:53.570024+0000","last_undegraded":"2024-04-25T10:30:53.570024+0000","last_fullsized":"2024-04-25T10:30:53.570024+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:10:21.595653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794446+0000","last_change":"2024-04-25T10:30:32.432061+0000","last_active":"2024-04-25T10:30:34.794446+0000","last_peered":"2024-04-25T10:30:34.794446+0000","last_clean":"2024-04-25T10:30:34.794446+0000","last_became_active":"2024-04-25T10:30:32.431651+0000","last_became_peered":"2024-04-25T10:30:32.431651+0000","last_unstale":"2024-04-25T10:30:34.794446+0000","last_undegraded":"2024-04-25T10:30:34.794446+0000","last_fullsized":"2024-04-25T10:30:34.794446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:39.062490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:37.434698+0000","last_change":"2024-04-25T10:30:31.420258+0000","last_active":"2024-04-25T10:30:37.434698+0000","last_peered":"2024-04-25T10:30:37.434698+0000","last_clean":"2024-04-25T10:30:37.434698+0000","last_became_active":"2024-04-25T10:30:31.419902+0000","last_became_peered":"2024-04-25T10:30:31.419902+0000","last_unstale":"2024-04-25T10:30:37.434698+0000","last_undegraded":"2024-04-25T10:30:37.434698+0000","last_fullsized":"2024-04-25T10:30:37.434698+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:43.919867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795443+0000","last_change":"2024-04-25T10:30:32.433568+0000","last_active":"2024-04-25T10:30:34.795443+0000","last_peered":"2024-04-25T10:30:34.795443+0000","last_clean":"2024-04-25T10:30:34.795443+0000","last_became_active":"2024-04-25T10:30:32.432572+0000","last_became_peered":"2024-04-25T10:30:32.432572+0000","last_unstale":"2024-04-25T10:30:34.795443+0000","last_undegraded":"2024-04-25T10:30:34.795443+0000","last_fullsized":"2024-04-25T10:30:34.795443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:31.771919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:59.583380+0000","last_change":"2024-04-25T10:30:31.419308+0000","last_active":"2024-04-25T10:30:59.583380+0000","last_peered":"2024-04-25T10:30:59.583380+0000","last_clean":"2024-04-25T10:30:59.583380+0000","last_became_active":"2024-04-25T10:30:31.418978+0000","last_became_peered":"2024-04-25T10:30:31.418978+0000","last_unstale":"2024-04-25T10:30:59.583380+0000","last_undegraded":"2024-04-25T10:30:59.583380+0000","last_fullsized":"2024-04-25T10:30:59.583380+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:33.784804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795670+0000","last_change":"2024-04-25T10:30:32.436109+0000","last_active":"2024-04-25T10:30:34.795670+0000","last_peered":"2024-04-25T10:30:34.795670+0000","last_clean":"2024-04-25T10:30:34.795670+0000","last_became_active":"2024-04-25T10:30:32.434701+0000","last_became_peered":"2024-04-25T10:30:32.434701+0000","last_unstale":"2024-04-25T10:30:34.795670+0000","last_undegraded":"2024-04-25T10:30:34.795670+0000","last_fullsized":"2024-04-25T10:30:34.795670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:19.471867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:53.569755+0000","last_change":"2024-04-25T10:30:31.416044+0000","last_active":"2024-04-25T10:30:53.569755+0000","last_peered":"2024-04-25T10:30:53.569755+0000","last_clean":"2024-04-25T10:30:53.569755+0000","last_became_active":"2024-04-25T10:30:31.415707+0000","last_became_peered":"2024-04-25T10:30:31.415707+0000","last_unstale":"2024-04-25T10:30:53.569755+0000","last_undegraded":"2024-04-25T10:30:53.569755+0000","last_fullsized":"2024-04-25T10:30:53.569755+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:31.424135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795321+0000","last_change":"2024-04-25T10:30:32.442971+0000","last_active":"2024-04-25T10:30:34.795321+0000","last_peered":"2024-04-25T10:30:34.795321+0000","last_clean":"2024-04-25T10:30:34.795321+0000","last_became_active":"2024-04-25T10:30:32.442656+0000","last_became_peered":"2024-04-25T10:30:32.442656+0000","last_unstale":"2024-04-25T10:30:34.795321+0000","last_undegraded":"2024-04-25T10:30:34.795321+0000","last_fullsized":"2024-04-25T10:30:34.795321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:29.951142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:53.566613+0000","last_change":"2024-04-25T10:30:31.404793+0000","last_active":"2024-04-25T10:30:53.566613+0000","last_peered":"2024-04-25T10:30:53.566613+0000","last_clean":"2024-04-25T10:30:53.566613+0000","last_became_active":"2024-04-25T10:30:31.404392+0000","last_became_peered":"2024-04-25T10:30:31.404392+0000","last_unstale":"2024-04-25T10:30:53.566613+0000","last_undegraded":"2024-04-25T10:30:53.566613+0000","last_fullsized":"2024-04-25T10:30:53.566613+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:00.965450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795922+0000","last_change":"2024-04-25T10:30:32.441438+0000","last_active":"2024-04-25T10:30:34.795922+0000","last_peered":"2024-04-25T10:30:34.795922+0000","last_clean":"2024-04-25T10:30:34.795922+0000","last_became_active":"2024-04-25T10:30:32.440011+0000","last_became_peered":"2024-04-25T10:30:32.440011+0000","last_unstale":"2024-04-25T10:30:34.795922+0000","last_undegraded":"2024-04-25T10:30:34.795922+0000","last_fullsized":"2024-04-25T10:30:34.795922+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:09.127517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":110,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.793865+0000","last_change":"2024-04-25T10:30:21.429098+0000","last_active":"2024-04-25T10:30:34.793865+0000","last_peered":"2024-04-25T10:30:34.793865+0000","last_clean":"2024-04-25T10:30:34.793865+0000","last_became_active":"2024-04-25T10:30:21.427958+0000","last_became_peered":"2024-04-25T10:30:21.427958+0000","last_unstale":"2024-04-25T10:30:34.793865+0000","last_undegraded":"2024-04-25T10:30:34.793865+0000","last_fullsized":"2024-04-25T10:30:34.793865+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:20.186206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:20.186206+0000","last_clean_scrub_stamp":"2024-04-25T10:30:20.186206+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:46.046488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"20'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:53.569669+0000","last_change":"2024-04-25T10:30:31.404974+0000","last_active":"2024-04-25T10:30:53.569669+0000","last_peered":"2024-04-25T10:30:53.569669+0000","last_clean":"2024-04-25T10:30:53.569669+0000","last_became_active":"2024-04-25T10:30:31.404549+0000","last_became_peered":"2024-04-25T10:30:31.404549+0000","last_unstale":"2024-04-25T10:30:53.569669+0000","last_undegraded":"2024-04-25T10:30:53.569669+0000","last_fullsized":"2024-04-25T10:30:53.569669+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:06.191321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415412+0000","last_change":"2024-04-25T10:30:32.434409+0000","last_active":"2024-04-25T10:30:33.415412+0000","last_peered":"2024-04-25T10:30:33.415412+0000","last_clean":"2024-04-25T10:30:33.415412+0000","last_became_active":"2024-04-25T10:30:32.433830+0000","last_became_peered":"2024-04-25T10:30:32.433830+0000","last_unstale":"2024-04-25T10:30:33.415412+0000","last_undegraded":"2024-04-25T10:30:33.415412+0000","last_fullsized":"2024-04-25T10:30:33.415412+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:57.864627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"20'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:48.490759+0000","last_change":"2024-04-25T10:30:31.416025+0000","last_active":"2024-04-25T10:30:48.490759+0000","last_peered":"2024-04-25T10:30:48.490759+0000","last_clean":"2024-04-25T10:30:48.490759+0000","last_became_active":"2024-04-25T10:30:31.415675+0000","last_became_peered":"2024-04-25T10:30:31.415675+0000","last_unstale":"2024-04-25T10:30:48.490759+0000","last_undegraded":"2024-04-25T10:30:48.490759+0000","last_fullsized":"2024-04-25T10:30:48.490759+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:28:48.185715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.196505+0000","last_change":"2024-04-25T10:30:32.424630+0000","last_active":"2024-04-25T10:30:34.196505+0000","last_peered":"2024-04-25T10:30:34.196505+0000","last_clean":"2024-04-25T10:30:34.196505+0000","last_became_active":"2024-04-25T10:30:32.424303+0000","last_became_peered":"2024-04-25T10:30:32.424303+0000","last_unstale":"2024-04-25T10:30:34.196505+0000","last_undegraded":"2024-04-25T10:30:34.196505+0000","last_fullsized":"2024-04-25T10:30:34.196505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:55.535319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:48.498662+0000","last_change":"2024-04-25T10:30:31.419145+0000","last_active":"2024-04-25T10:30:48.498662+0000","last_peered":"2024-04-25T10:30:48.498662+0000","last_clean":"2024-04-25T10:30:48.498662+0000","last_became_active":"2024-04-25T10:30:31.418817+0000","last_became_peered":"2024-04-25T10:30:31.418817+0000","last_unstale":"2024-04-25T10:30:48.498662+0000","last_undegraded":"2024-04-25T10:30:48.498662+0000","last_fullsized":"2024-04-25T10:30:48.498662+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:01.608205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.412646+0000","last_change":"2024-04-25T10:30:32.419911+0000","last_active":"2024-04-25T10:30:33.412646+0000","last_peered":"2024-04-25T10:30:33.412646+0000","last_clean":"2024-04-25T10:30:33.412646+0000","last_became_active":"2024-04-25T10:30:32.419486+0000","last_became_peered":"2024-04-25T10:30:32.419486+0000","last_unstale":"2024-04-25T10:30:33.412646+0000","last_undegraded":"2024-04-25T10:30:33.412646+0000","last_fullsized":"2024-04-25T10:30:33.412646+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:54.967873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:59.589197+0000","last_change":"2024-04-25T10:30:31.405646+0000","last_active":"2024-04-25T10:30:59.589197+0000","last_peered":"2024-04-25T10:30:59.589197+0000","last_clean":"2024-04-25T10:30:59.589197+0000","last_became_active":"2024-04-25T10:30:31.405238+0000","last_became_peered":"2024-04-25T10:30:31.405238+0000","last_unstale":"2024-04-25T10:30:59.589197+0000","last_undegraded":"2024-04-25T10:30:59.589197+0000","last_fullsized":"2024-04-25T10:30:59.589197+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:13.159787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795730+0000","last_change":"2024-04-25T10:30:32.426130+0000","last_active":"2024-04-25T10:30:34.795730+0000","last_peered":"2024-04-25T10:30:34.795730+0000","last_clean":"2024-04-25T10:30:34.795730+0000","last_became_active":"2024-04-25T10:30:32.425783+0000","last_became_peered":"2024-04-25T10:30:32.425783+0000","last_unstale":"2024-04-25T10:30:34.795730+0000","last_undegraded":"2024-04-25T10:30:34.795730+0000","last_fullsized":"2024-04-25T10:30:34.795730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:30.844640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:35.119286+0000","last_change":"2024-04-25T10:30:31.412736+0000","last_active":"2024-04-25T10:30:35.119286+0000","last_peered":"2024-04-25T10:30:35.119286+0000","last_clean":"2024-04-25T10:30:35.119286+0000","last_became_active":"2024-04-25T10:30:31.412413+0000","last_became_peered":"2024-04-25T10:30:31.412413+0000","last_unstale":"2024-04-25T10:30:35.119286+0000","last_undegraded":"2024-04-25T10:30:35.119286+0000","last_fullsized":"2024-04-25T10:30:35.119286+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:19:56.236327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795688+0000","last_change":"2024-04-25T10:30:32.440858+0000","last_active":"2024-04-25T10:30:34.795688+0000","last_peered":"2024-04-25T10:30:34.795688+0000","last_clean":"2024-04-25T10:30:34.795688+0000","last_became_active":"2024-04-25T10:30:32.440519+0000","last_became_peered":"2024-04-25T10:30:32.440519+0000","last_unstale":"2024-04-25T10:30:34.795688+0000","last_undegraded":"2024-04-25T10:30:34.795688+0000","last_fullsized":"2024-04-25T10:30:34.795688+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:21.288748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"20'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:53.569891+0000","last_change":"2024-04-25T10:30:31.417958+0000","last_active":"2024-04-25T10:30:53.569891+0000","last_peered":"2024-04-25T10:30:53.569891+0000","last_clean":"2024-04-25T10:30:53.569891+0000","last_became_active":"2024-04-25T10:30:31.417541+0000","last_became_peered":"2024-04-25T10:30:31.417541+0000","last_unstale":"2024-04-25T10:30:53.569891+0000","last_undegraded":"2024-04-25T10:30:53.569891+0000","last_fullsized":"2024-04-25T10:30:53.569891+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:35.373351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.412553+0000","last_change":"2024-04-25T10:30:32.420451+0000","last_active":"2024-04-25T10:30:33.412553+0000","last_peered":"2024-04-25T10:30:33.412553+0000","last_clean":"2024-04-25T10:30:33.412553+0000","last_became_active":"2024-04-25T10:30:32.420109+0000","last_became_peered":"2024-04-25T10:30:32.420109+0000","last_unstale":"2024-04-25T10:30:33.412553+0000","last_undegraded":"2024-04-25T10:30:33.412553+0000","last_fullsized":"2024-04-25T10:30:33.412553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:22.067828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'7","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:53.569765+0000","last_change":"2024-04-25T10:30:31.406005+0000","last_active":"2024-04-25T10:30:53.569765+0000","last_peered":"2024-04-25T10:30:53.569765+0000","last_clean":"2024-04-25T10:30:53.569765+0000","last_became_active":"2024-04-25T10:30:31.404739+0000","last_became_peered":"2024-04-25T10:30:31.404739+0000","last_unstale":"2024-04-25T10:30:53.569765+0000","last_undegraded":"2024-04-25T10:30:53.569765+0000","last_fullsized":"2024-04-25T10:30:53.569765+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:31.771123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415711+0000","last_change":"2024-04-25T10:30:32.437513+0000","last_active":"2024-04-25T10:30:33.415711+0000","last_peered":"2024-04-25T10:30:33.415711+0000","last_clean":"2024-04-25T10:30:33.415711+0000","last_became_active":"2024-04-25T10:30:32.437171+0000","last_became_peered":"2024-04-25T10:30:32.437171+0000","last_unstale":"2024-04-25T10:30:33.415711+0000","last_undegraded":"2024-04-25T10:30:33.415711+0000","last_fullsized":"2024-04-25T10:30:33.415711+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:50.924843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'7","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:59.786001+0000","last_change":"2024-04-25T10:30:31.413140+0000","last_active":"2024-04-25T10:30:59.786001+0000","last_peered":"2024-04-25T10:30:59.786001+0000","last_clean":"2024-04-25T10:30:59.786001+0000","last_became_active":"2024-04-25T10:30:31.412821+0000","last_became_peered":"2024-04-25T10:30:31.412821+0000","last_unstale":"2024-04-25T10:30:59.786001+0000","last_undegraded":"2024-04-25T10:30:59.786001+0000","last_fullsized":"2024-04-25T10:30:59.786001+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:37.177254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.196799+0000","last_change":"2024-04-25T10:30:32.434333+0000","last_active":"2024-04-25T10:30:34.196799+0000","last_peered":"2024-04-25T10:30:34.196799+0000","last_clean":"2024-04-25T10:30:34.196799+0000","last_became_active":"2024-04-25T10:30:32.434000+0000","last_became_peered":"2024-04-25T10:30:32.434000+0000","last_unstale":"2024-04-25T10:30:34.196799+0000","last_undegraded":"2024-04-25T10:30:34.196799+0000","last_fullsized":"2024-04-25T10:30:34.196799+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:54.057457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:43.468255+0000","last_change":"2024-04-25T10:30:31.419475+0000","last_active":"2024-04-25T10:30:43.468255+0000","last_peered":"2024-04-25T10:30:43.468255+0000","last_clean":"2024-04-25T10:30:43.468255+0000","last_became_active":"2024-04-25T10:30:31.417605+0000","last_became_peered":"2024-04-25T10:30:31.417605+0000","last_unstale":"2024-04-25T10:30:43.468255+0000","last_undegraded":"2024-04-25T10:30:43.468255+0000","last_fullsized":"2024-04-25T10:30:43.468255+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:00.948269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.794414+0000","last_change":"2024-04-25T10:30:32.419693+0000","last_active":"2024-04-25T10:30:34.794414+0000","last_peered":"2024-04-25T10:30:34.794414+0000","last_clean":"2024-04-25T10:30:34.794414+0000","last_became_active":"2024-04-25T10:30:32.419370+0000","last_became_peered":"2024-04-25T10:30:32.419370+0000","last_unstale":"2024-04-25T10:30:34.794414+0000","last_undegraded":"2024-04-25T10:30:34.794414+0000","last_fullsized":"2024-04-25T10:30:34.794414+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:31.199065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"20'7","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:59.589760+0000","last_change":"2024-04-25T10:30:31.410767+0000","last_active":"2024-04-25T10:30:59.589760+0000","last_peered":"2024-04-25T10:30:59.589760+0000","last_clean":"2024-04-25T10:30:59.589760+0000","last_became_active":"2024-04-25T10:30:31.410404+0000","last_became_peered":"2024-04-25T10:30:31.410404+0000","last_unstale":"2024-04-25T10:30:59.589760+0000","last_undegraded":"2024-04-25T10:30:59.589760+0000","last_fullsized":"2024-04-25T10:30:59.589760+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:37.431532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415585+0000","last_change":"2024-04-25T10:30:32.433687+0000","last_active":"2024-04-25T10:30:33.415585+0000","last_peered":"2024-04-25T10:30:33.415585+0000","last_clean":"2024-04-25T10:30:33.415585+0000","last_became_active":"2024-04-25T10:30:32.433389+0000","last_became_peered":"2024-04-25T10:30:32.433389+0000","last_unstale":"2024-04-25T10:30:33.415585+0000","last_undegraded":"2024-04-25T10:30:33.415585+0000","last_fullsized":"2024-04-25T10:30:33.415585+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:39.155317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"20'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:53.569498+0000","last_change":"2024-04-25T10:30:31.419118+0000","last_active":"2024-04-25T10:30:53.569498+0000","last_peered":"2024-04-25T10:30:53.569498+0000","last_clean":"2024-04-25T10:30:53.569498+0000","last_became_active":"2024-04-25T10:30:31.418822+0000","last_became_peered":"2024-04-25T10:30:31.418822+0000","last_unstale":"2024-04-25T10:30:53.569498+0000","last_undegraded":"2024-04-25T10:30:53.569498+0000","last_fullsized":"2024-04-25T10:30:53.569498+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:22.755042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415890+0000","last_change":"2024-04-25T10:30:32.427894+0000","last_active":"2024-04-25T10:30:33.415890+0000","last_peered":"2024-04-25T10:30:33.415890+0000","last_clean":"2024-04-25T10:30:33.415890+0000","last_became_active":"2024-04-25T10:30:32.427504+0000","last_became_peered":"2024-04-25T10:30:32.427504+0000","last_unstale":"2024-04-25T10:30:33.415890+0000","last_undegraded":"2024-04-25T10:30:33.415890+0000","last_fullsized":"2024-04-25T10:30:33.415890+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:53.877484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:43.469382+0000","last_change":"2024-04-25T10:30:31.417115+0000","last_active":"2024-04-25T10:30:43.469382+0000","last_peered":"2024-04-25T10:30:43.469382+0000","last_clean":"2024-04-25T10:30:43.469382+0000","last_became_active":"2024-04-25T10:30:31.416766+0000","last_became_peered":"2024-04-25T10:30:31.416766+0000","last_unstale":"2024-04-25T10:30:43.469382+0000","last_undegraded":"2024-04-25T10:30:43.469382+0000","last_fullsized":"2024-04-25T10:30:43.469382+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:06.021270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795407+0000","last_change":"2024-04-25T10:30:32.431674+0000","last_active":"2024-04-25T10:30:34.795407+0000","last_peered":"2024-04-25T10:30:34.795407+0000","last_clean":"2024-04-25T10:30:34.795407+0000","last_became_active":"2024-04-25T10:30:32.431395+0000","last_became_peered":"2024-04-25T10:30:32.431395+0000","last_unstale":"2024-04-25T10:30:34.795407+0000","last_undegraded":"2024-04-25T10:30:34.795407+0000","last_fullsized":"2024-04-25T10:30:34.795407+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:49.726323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795508+0000","last_change":"2024-04-25T10:30:32.437700+0000","last_active":"2024-04-25T10:30:34.795508+0000","last_peered":"2024-04-25T10:30:34.795508+0000","last_clean":"2024-04-25T10:30:34.795508+0000","last_became_active":"2024-04-25T10:30:32.437242+0000","last_became_peered":"2024-04-25T10:30:32.437242+0000","last_unstale":"2024-04-25T10:30:34.795508+0000","last_undegraded":"2024-04-25T10:30:34.795508+0000","last_fullsized":"2024-04-25T10:30:34.795508+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:06.499414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:59.589520+0000","last_change":"2024-04-25T10:30:31.419678+0000","last_active":"2024-04-25T10:30:59.589520+0000","last_peered":"2024-04-25T10:30:59.589520+0000","last_clean":"2024-04-25T10:30:59.589520+0000","last_became_active":"2024-04-25T10:30:31.419269+0000","last_became_peered":"2024-04-25T10:30:31.419269+0000","last_unstale":"2024-04-25T10:30:59.589520+0000","last_undegraded":"2024-04-25T10:30:59.589520+0000","last_fullsized":"2024-04-25T10:30:59.589520+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:33.034260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.795138+0000","last_change":"2024-04-25T10:30:32.434410+0000","last_active":"2024-04-25T10:30:34.795138+0000","last_peered":"2024-04-25T10:30:34.795138+0000","last_clean":"2024-04-25T10:30:34.795138+0000","last_became_active":"2024-04-25T10:30:32.434093+0000","last_became_peered":"2024-04-25T10:30:32.434093+0000","last_unstale":"2024-04-25T10:30:34.795138+0000","last_undegraded":"2024-04-25T10:30:34.795138+0000","last_fullsized":"2024-04-25T10:30:34.795138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:46.964704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'11","reported_seq":25,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:59.589428+0000","last_change":"2024-04-25T10:30:31.418360+0000","last_active":"2024-04-25T10:30:59.589428+0000","last_peered":"2024-04-25T10:30:59.589428+0000","last_clean":"2024-04-25T10:30:59.589428+0000","last_became_active":"2024-04-25T10:30:31.418005+0000","last_became_peered":"2024-04-25T10:30:31.418005+0000","last_unstale":"2024-04-25T10:30:59.589428+0000","last_undegraded":"2024-04-25T10:30:59.589428+0000","last_fullsized":"2024-04-25T10:30:59.589428+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:15.556859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:34.196705+0000","last_change":"2024-04-25T10:30:32.439515+0000","last_active":"2024-04-25T10:30:34.196705+0000","last_peered":"2024-04-25T10:30:34.196705+0000","last_clean":"2024-04-25T10:30:34.196705+0000","last_became_active":"2024-04-25T10:30:32.439101+0000","last_became_peered":"2024-04-25T10:30:32.439101+0000","last_unstale":"2024-04-25T10:30:34.196705+0000","last_undegraded":"2024-04-25T10:30:34.196705+0000","last_fullsized":"2024-04-25T10:30:34.196705+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:33.797903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:37.434282+0000","last_change":"2024-04-25T10:30:31.418070+0000","last_active":"2024-04-25T10:30:37.434282+0000","last_peered":"2024-04-25T10:30:37.434282+0000","last_clean":"2024-04-25T10:30:37.434282+0000","last_became_active":"2024-04-25T10:30:31.417737+0000","last_became_peered":"2024-04-25T10:30:31.417737+0000","last_unstale":"2024-04-25T10:30:37.434282+0000","last_undegraded":"2024-04-25T10:30:37.434282+0000","last_fullsized":"2024-04-25T10:30:37.434282+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:05.664855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:33.415785+0000","last_change":"2024-04-25T10:30:32.435258+0000","last_active":"2024-04-25T10:30:33.415785+0000","last_peered":"2024-04-25T10:30:33.415785+0000","last_clean":"2024-04-25T10:30:33.415785+0000","last_became_active":"2024-04-25T10:30:32.434930+0000","last_became_peered":"2024-04-25T10:30:32.434930+0000","last_unstale":"2024-04-25T10:30:33.415785+0000","last_undegraded":"2024-04-25T10:30:33.415785+0000","last_fullsized":"2024-04-25T10:30:33.415785+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:22.727778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'13","reported_seq":27,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:30:59.589618+0000","last_change":"2024-04-25T10:30:31.415611+0000","last_active":"2024-04-25T10:30:59.589618+0000","last_peered":"2024-04-25T10:30:59.589618+0000","last_clean":"2024-04-25T10:30:59.589618+0000","last_became_active":"2024-04-25T10:30:31.415071+0000","last_became_peered":"2024-04-25T10:30:31.415071+0000","last_unstale":"2024-04-25T10:30:59.589618+0000","last_undegraded":"2024-04-25T10:30:59.589618+0000","last_fullsized":"2024-04-25T10:30:59.589618+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:30:54.077864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":8034,"num_objects":97,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":204,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":65,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":262144,"data_stored":16068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":178,"ondisk_log_size":178,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574861,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":835584,"data_stored":663402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574861,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":233472,"data_stored":73336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574862,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":253952,"data_stored":74866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574862,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":229376,"data_stored":72264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574861,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":237568,"data_stored":73370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574861,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":237568,"data_stored":72396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574861,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":233472,"data_stored":72466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574861,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":827392,"data_stored":662176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:31:11.844 INFO:tasks.mon_thrash.ceph_manager:clean! 2024-04-25T10:31:11.845 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:31:11.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.988+0000 7fe527d1f640 1 Processor -- start 2024-04-25T10:31:11.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.988+0000 7fe527d1f640 1 -- start start 2024-04-25T10:31:11.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.990+0000 7fe527d1f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe5200c7e30 0x7fe5200c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:11.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.990+0000 7fe527d1f640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fe520156ef0 0x7fe5200b4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:11.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.990+0000 7fe527d1f640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5200ce630 con 0x7fe52006d000 2024-04-25T10:31:11.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.990+0000 7fe527d1f640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fe5200c7c60 con 0x7fe520156ef0 2024-04-25T10:31:11.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.990+0000 7fe527d1f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe5200c7980 con 0x7fe5200c7e30 2024-04-25T10:31:11.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.990+0000 7fe525abb640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe5200c7e30 0x7fe5200c8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:11.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.990+0000 7fe5252ba640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fe520156ef0 0x7fe5200b4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:11.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.990+0000 7fe525abb640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe5200c7e30 0x7fe5200c8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.106:47382/0 (socket says 172.21.15.106:47382) 2024-04-25T10:31:11.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.990+0000 7fe525abb640 1 -- 172.21.15.106:0/126433858 learned_addr learned my addr 172.21.15.106:0/126433858 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:31:11.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.990+0000 7fe525abb640 1 -- 172.21.15.106:0/126433858 >> v2:172.21.15.134:3300/0 conn(0x7fe520156ef0 msgr2=0x7fe5200b4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:11.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.990+0000 7fe525abb640 1 --2- 172.21.15.106:0/126433858 >> v2:172.21.15.134:3300/0 conn(0x7fe520156ef0 0x7fe5200b4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.990+0000 7fe525abb640 1 -- 172.21.15.106:0/126433858 >> v1:172.21.15.134:6789/0 conn(0x7fe52006d000 legacy=0x7fe52015ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:11.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.990+0000 7fe525abb640 1 -- 172.21.15.106:0/126433858 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5200c7010 con 0x7fe5200c7e30 2024-04-25T10:31:11.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.991+0000 7fe525abb640 1 --2- 172.21.15.106:0/126433858 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe5200c7e30 0x7fe5200c8d90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe5200c7de0 tx=0x7fe51c02a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=631543a705c5b127 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:11.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.991+0000 7fe524ab9640 1 -- 172.21.15.106:0/126433858 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51c036070 con 0x7fe5200c7e30 2024-04-25T10:31:11.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.991+0000 7fe524ab9640 1 -- 172.21.15.106:0/126433858 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe51c031600 con 0x7fe5200c7e30 2024-04-25T10:31:11.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.991+0000 7fe524ab9640 1 -- 172.21.15.106:0/126433858 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51c0319d0 con 0x7fe5200c7e30 2024-04-25T10:31:11.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.992+0000 7fe527d1f640 1 -- 172.21.15.106:0/126433858 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe5200c7e30 msgr2=0x7fe5200c8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:11.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.992+0000 7fe527d1f640 1 --2- 172.21.15.106:0/126433858 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe5200c7e30 0x7fe5200c8d90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe5200c7de0 tx=0x7fe51c02a680 comp rx=0 tx=0).stop 2024-04-25T10:31:11.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.992+0000 7fe527d1f640 1 -- 172.21.15.106:0/126433858 shutdown_connections 2024-04-25T10:31:11.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.992+0000 7fe527d1f640 1 --2- 172.21.15.106:0/126433858 >> v2:172.21.15.134:3300/0 conn(0x7fe520156ef0 0x7fe5200b4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.992+0000 7fe527d1f640 1 --2- 172.21.15.106:0/126433858 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe5200c7e30 0x7fe5200c8d90 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.992+0000 7fe527d1f640 1 -- 172.21.15.106:0/126433858 >> 172.21.15.106:0/126433858 conn(0x7fe520068b10 msgr2=0x7fe5200d1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:11.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.992+0000 7fe527d1f640 1 -- 172.21.15.106:0/126433858 shutdown_connections 2024-04-25T10:31:11.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.992+0000 7fe527d1f640 1 -- 172.21.15.106:0/126433858 wait complete. 2024-04-25T10:31:11.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.994+0000 7fe527d1f640 1 Processor -- start 2024-04-25T10:31:11.996 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.994+0000 7fe527d1f640 1 -- start start 2024-04-25T10:31:11.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.995+0000 7fe527d1f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe52006d000 0x7fe5201f3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:11.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.995+0000 7fe527d1f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe5200c7e30 0x7fe5200d10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:11.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.995+0000 7fe527d1f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe520156ef0 0x7fe520149760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:11.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.995+0000 7fe527d1f640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe52015d6c0 con 0x7fe52006d000 2024-04-25T10:31:11.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.995+0000 7fe527d1f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe52015d540 con 0x7fe520156ef0 2024-04-25T10:31:11.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.995+0000 7fe527d1f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe520148d70 con 0x7fe5200c7e30 2024-04-25T10:31:11.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.995+0000 7fe525abb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe52006d000 0x7fe5201f3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:11.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.995+0000 7fe525abb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe52006d000 0x7fe5201f3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59214/0 (socket says 172.21.15.106:59214) 2024-04-25T10:31:11.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.995+0000 7fe525abb640 1 -- 172.21.15.106:0/927662626 learned_addr learned my addr 172.21.15.106:0/927662626 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:31:11.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.995+0000 7fe5252ba640 1 --2- 172.21.15.106:0/927662626 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe5200c7e30 0x7fe5200d10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:11.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.995+0000 7fe5262bc640 1 --2- 172.21.15.106:0/927662626 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe520156ef0 0x7fe520149760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:11.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.995+0000 7fe525abb640 1 -- 172.21.15.106:0/927662626 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe5200c7e30 msgr2=0x7fe5200d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:11.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.995+0000 7fe525abb640 1 --2- 172.21.15.106:0/927662626 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe5200c7e30 0x7fe5200d10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.995+0000 7fe525abb640 1 -- 172.21.15.106:0/927662626 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe520156ef0 msgr2=0x7fe520149760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:12.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.995+0000 7fe525abb640 1 --2- 172.21.15.106:0/927662626 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe520156ef0 0x7fe520149760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:12.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.995+0000 7fe525abb640 1 -- 172.21.15.106:0/927662626 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe520153590 con 0x7fe52006d000 2024-04-25T10:31:12.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.996+0000 7fe525abb640 1 --2- 172.21.15.106:0/927662626 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe52006d000 0x7fe5201f3630 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe51c009f40 tx=0x7fe51c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:12.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.996+0000 7fe5167fc640 1 -- 172.21.15.106:0/927662626 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51c03f020 con 0x7fe52006d000 2024-04-25T10:31:12.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.996+0000 7fe5167fc640 1 -- 172.21.15.106:0/927662626 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe51c037040 con 0x7fe52006d000 2024-04-25T10:31:12.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.996+0000 7fe527d1f640 1 -- 172.21.15.106:0/927662626 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe520148b60 con 0x7fe52006d000 2024-04-25T10:31:12.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.996+0000 7fe5167fc640 1 -- 172.21.15.106:0/927662626 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51c036070 con 0x7fe52006d000 2024-04-25T10:31:12.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.996+0000 7fe527d1f640 1 -- 172.21.15.106:0/927662626 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5200d0010 con 0x7fe52006d000 2024-04-25T10:31:12.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.997+0000 7fe527d1f640 1 -- 172.21.15.106:0/927662626 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe52006f650 con 0x7fe52006d000 2024-04-25T10:31:12.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.998+0000 7fe5167fc640 1 -- 172.21.15.106:0/927662626 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe51c042020 con 0x7fe52006d000 2024-04-25T10:31:12.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.998+0000 7fe5167fc640 1 --2- 172.21.15.106:0/927662626 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fe5040744e0 0x7fe5040769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:12.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.998+0000 7fe5167fc640 1 -- 172.21.15.106:0/927662626 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe51c0b5560 con 0x7fe52006d000 2024-04-25T10:31:12.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.998+0000 7fe5252ba640 1 --2- 172.21.15.106:0/927662626 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fe5040744e0 0x7fe5040769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:12.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:11.998+0000 7fe5252ba640 1 --2- 172.21.15.106:0/927662626 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fe5040744e0 0x7fe5040769f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe52014d590 tx=0x7fe510009110 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:12.004 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.003+0000 7fe5167fc640 1 -- 172.21.15.106:0/927662626 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe52006f650 con 0x7fe52006d000 2024-04-25T10:31:12.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.181+0000 7fe527d1f640 1 -- 172.21.15.106:0/927662626 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe52006a3d0 con 0x7fe52006d000 2024-04-25T10:31:12.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.182+0000 7fe5167fc640 1 -- 172.21.15.106:0/927662626 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16641 (secure 0 0 0) 0x7fe51c049030 con 0x7fe52006d000 2024-04-25T10:31:12.185 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:31:12.185 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":20,"fsid":"3ee77985-e8e4-41cf-81e9-7c774cf7891f","created":"2024-04-25T10:30:07.981054+0000","modified":"2024-04-25T10:30:33.409267+0000","last_up_change":"2024-04-25T10:30:19.184325+0000","last_in_change":"2024-04-25T10:30:10.217553+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:30:20.111046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:30:30.314914+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:30:30.715500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a5962cb1-dbbd-47b1-8d15-9bae114bfdf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1240717079},{"type":"v1","addr":"172.21.15.106:6801","nonce":1240717079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1240717079},{"type":"v1","addr":"172.21.15.106:6803","nonce":1240717079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1240717079},{"type":"v1","addr":"172.21.15.106:6807","nonce":1240717079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1240717079},{"type":"v1","addr":"172.21.15.106:6805","nonce":1240717079}]},"public_addr":"172.21.15.106:6801/1240717079","cluster_addr":"172.21.15.106:6803/1240717079","heartbeat_back_addr":"172.21.15.106:6807/1240717079","heartbeat_front_addr":"172.21.15.106:6805/1240717079","state":["exists","up"]},{"osd":1,"uuid":"94436e13-4769-4752-8397-b5e396a6f7ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4089653051},{"type":"v1","addr":"172.21.15.106:6825","nonce":4089653051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4089653051},{"type":"v1","addr":"172.21.15.106:6827","nonce":4089653051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4089653051},{"type":"v1","addr":"172.21.15.106:6831","nonce":4089653051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4089653051},{"type":"v1","addr":"172.21.15.106:6829","nonce":4089653051}]},"public_addr":"172.21.15.106:6825/4089653051","cluster_addr":"172.21.15.106:6827/4089653051","heartbeat_back_addr":"172.21.15.106:6831/4089653051","heartbeat_front_addr":"172.21.15.106:6829/4089653051","state":["exists","up"]},{"osd":2,"uuid":"429aae0b-70c6-4e7b-9043-6a11334c7efc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":1441018239},{"type":"v1","addr":"172.21.15.106:6809","nonce":1441018239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1441018239},{"type":"v1","addr":"172.21.15.106:6811","nonce":1441018239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1441018239},{"type":"v1","addr":"172.21.15.106:6815","nonce":1441018239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":1441018239},{"type":"v1","addr":"172.21.15.106:6813","nonce":1441018239}]},"public_addr":"172.21.15.106:6809/1441018239","cluster_addr":"172.21.15.106:6811/1441018239","heartbeat_back_addr":"172.21.15.106:6815/1441018239","heartbeat_front_addr":"172.21.15.106:6813/1441018239","state":["exists","up"]},{"osd":3,"uuid":"9593a4d1-772f-4667-ab0d-2982f841e342","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":1846968711},{"type":"v1","addr":"172.21.15.106:6817","nonce":1846968711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1846968711},{"type":"v1","addr":"172.21.15.106:6819","nonce":1846968711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":1846968711},{"type":"v1","addr":"172.21.15.106:6823","nonce":1846968711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":1846968711},{"type":"v1","addr":"172.21.15.106:6821","nonce":1846968711}]},"public_addr":"172.21.15.106:6817/1846968711","cluster_addr":"172.21.15.106:6819/1846968711","heartbeat_back_addr":"172.21.15.106:6823/1846968711","heartbeat_front_addr":"172.21.15.106:6821/1846968711","state":["exists","up"]},{"osd":4,"uuid":"1b61f16d-5c60-480a-b449-c745aa3f6b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":729224739},{"type":"v1","addr":"172.21.15.134:6825","nonce":729224739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":729224739},{"type":"v1","addr":"172.21.15.134:6827","nonce":729224739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":729224739},{"type":"v1","addr":"172.21.15.134:6831","nonce":729224739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":729224739},{"type":"v1","addr":"172.21.15.134:6829","nonce":729224739}]},"public_addr":"172.21.15.134:6825/729224739","cluster_addr":"172.21.15.134:6827/729224739","heartbeat_back_addr":"172.21.15.134:6831/729224739","heartbeat_front_addr":"172.21.15.134:6829/729224739","state":["exists","up"]},{"osd":5,"uuid":"944cdd7f-47c0-4b71-b7e0-d093542f4528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3132275652},{"type":"v1","addr":"172.21.15.134:6809","nonce":3132275652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3132275652},{"type":"v1","addr":"172.21.15.134:6811","nonce":3132275652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3132275652},{"type":"v1","addr":"172.21.15.134:6815","nonce":3132275652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3132275652},{"type":"v1","addr":"172.21.15.134:6813","nonce":3132275652}]},"public_addr":"172.21.15.134:6809/3132275652","cluster_addr":"172.21.15.134:6811/3132275652","heartbeat_back_addr":"172.21.15.134:6815/3132275652","heartbeat_front_addr":"172.21.15.134:6813/3132275652","state":["exists","up"]},{"osd":6,"uuid":"cc2446e2-1d21-4135-ac04-6af7c990058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":992702668},{"type":"v1","addr":"172.21.15.134:6801","nonce":992702668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":992702668},{"type":"v1","addr":"172.21.15.134:6803","nonce":992702668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":992702668},{"type":"v1","addr":"172.21.15.134:6807","nonce":992702668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":992702668},{"type":"v1","addr":"172.21.15.134:6805","nonce":992702668}]},"public_addr":"172.21.15.134:6801/992702668","cluster_addr":"172.21.15.134:6803/992702668","heartbeat_back_addr":"172.21.15.134:6807/992702668","heartbeat_front_addr":"172.21.15.134:6805/992702668","state":["exists","up"]},{"osd":7,"uuid":"3df1fc7a-82fc-4257-8629-98027b6e2aed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":3291062284},{"type":"v1","addr":"172.21.15.134:6817","nonce":3291062284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":3291062284},{"type":"v1","addr":"172.21.15.134:6819","nonce":3291062284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":3291062284},{"type":"v1","addr":"172.21.15.134:6823","nonce":3291062284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":3291062284},{"type":"v1","addr":"172.21.15.134:6821","nonce":3291062284}]},"public_addr":"172.21.15.134:6817/3291062284","cluster_addr":"172.21.15.134:6819/3291062284","heartbeat_back_addr":"172.21.15.134:6823/3291062284","heartbeat_front_addr":"172.21.15.134:6821/3291062284","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.403941+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.382563+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.411544+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.330444+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.405430+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.387737+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.408922+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.454450+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:31:12.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.184+0000 7fe527d1f640 1 -- 172.21.15.106:0/927662626 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fe5040744e0 msgr2=0x7fe5040769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:12.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.184+0000 7fe527d1f640 1 --2- 172.21.15.106:0/927662626 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fe5040744e0 0x7fe5040769f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe52014d590 tx=0x7fe510009110 comp rx=0 tx=0).stop 2024-04-25T10:31:12.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.184+0000 7fe527d1f640 1 -- 172.21.15.106:0/927662626 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe52006d000 msgr2=0x7fe5201f3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:12.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.184+0000 7fe527d1f640 1 --2- 172.21.15.106:0/927662626 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe52006d000 0x7fe5201f3630 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe51c009f40 tx=0x7fe51c004a40 comp rx=0 tx=0).stop 2024-04-25T10:31:12.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.184+0000 7fe527d1f640 1 -- 172.21.15.106:0/927662626 shutdown_connections 2024-04-25T10:31:12.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.184+0000 7fe527d1f640 1 --2- 172.21.15.106:0/927662626 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fe5040744e0 0x7fe5040769f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:12.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.184+0000 7fe527d1f640 1 --2- 172.21.15.106:0/927662626 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe520156ef0 0x7fe520149760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:12.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.184+0000 7fe527d1f640 1 --2- 172.21.15.106:0/927662626 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe5200c7e30 0x7fe5200d10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:12.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.184+0000 7fe527d1f640 1 --2- 172.21.15.106:0/927662626 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe52006d000 0x7fe5201f3630 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:12.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.184+0000 7fe527d1f640 1 -- 172.21.15.106:0/927662626 >> 172.21.15.106:0/927662626 conn(0x7fe520068b10 msgr2=0x7fe5201546b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:12.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.184+0000 7fe527d1f640 1 -- 172.21.15.106:0/927662626 shutdown_connections 2024-04-25T10:31:12.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.184+0000 7fe527d1f640 1 -- 172.21.15.106:0/927662626 wait complete. 2024-04-25T10:31:12.198 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:30:20.111046+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:30:30.314914+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:30:30.715500+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:31:12.198 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:31:12.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.305+0000 7f5cd0b86640 1 Processor -- start 2024-04-25T10:31:12.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.305+0000 7f5cd0b86640 1 -- start start 2024-04-25T10:31:12.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.307+0000 7f5cd0b86640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f5ccc074910 0x7f5ccc0c9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:12.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.307+0000 7f5cd0b86640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ccc157080 0x7f5ccc15cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:12.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.307+0000 7f5cd0b86640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ccc070af0 con 0x7f5ccc0684e0 2024-04-25T10:31:12.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.307+0000 7f5cd0b86640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f5ccc0733a0 con 0x7f5ccc074910 2024-04-25T10:31:12.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.307+0000 7f5cd0b86640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f5ccc0708a0 con 0x7f5ccc157080 2024-04-25T10:31:12.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.307+0000 7f5cca59c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f5ccc074910 0x7f5ccc0c9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:12.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.307+0000 7f5cca59c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f5ccc074910 0x7f5ccc0c9100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59220/0 (socket says 172.21.15.106:59220) 2024-04-25T10:31:12.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.307+0000 7f5cca59c640 1 -- 172.21.15.106:0/1406662030 learned_addr learned my addr 172.21.15.106:0/1406662030 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:31:12.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.307+0000 7f5cc9d9b640 1 --2- 172.21.15.106:0/1406662030 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ccc157080 0x7f5ccc15cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:12.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.307+0000 7f5cca59c640 1 -- 172.21.15.106:0/1406662030 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ccc157080 msgr2=0x7f5ccc15cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:12.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.307+0000 7f5cca59c640 1 --2- 172.21.15.106:0/1406662030 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ccc157080 0x7f5ccc15cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:12.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.307+0000 7f5cca59c640 1 -- 172.21.15.106:0/1406662030 >> v1:172.21.15.134:6789/0 conn(0x7f5ccc0684e0 legacy=0x7f5ccc0c9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:12.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.307+0000 7f5cca59c640 1 -- 172.21.15.106:0/1406662030 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ccc070dd0 con 0x7f5ccc074910 2024-04-25T10:31:12.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.307+0000 7f5cc9d9b640 1 --2- 172.21.15.106:0/1406662030 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ccc157080 0x7f5ccc15cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:31:12.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.308+0000 7f5cca59c640 1 --2- 172.21.15.106:0/1406662030 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ccc074910 0x7f5ccc0c9100 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc009e50 tx=0x7f5cbc02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6438bd4881447f81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:12.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.308+0000 7f5cc959a640 1 -- 172.21.15.106:0/1406662030 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc036070 con 0x7f5ccc074910 2024-04-25T10:31:12.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.308+0000 7f5cc959a640 1 -- 172.21.15.106:0/1406662030 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cbc031600 con 0x7f5ccc074910 2024-04-25T10:31:12.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.308+0000 7f5cc959a640 1 -- 172.21.15.106:0/1406662030 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc0319d0 con 0x7f5ccc074910 2024-04-25T10:31:12.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.309+0000 7f5cd0b86640 1 -- 172.21.15.106:0/1406662030 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ccc074910 msgr2=0x7f5ccc0c9100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:12.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.309+0000 7f5cd0b86640 1 --2- 172.21.15.106:0/1406662030 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ccc074910 0x7f5ccc0c9100 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc009e50 tx=0x7f5cbc02a710 comp rx=0 tx=0).stop 2024-04-25T10:31:12.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.309+0000 7f5cd0b86640 1 -- 172.21.15.106:0/1406662030 shutdown_connections 2024-04-25T10:31:12.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.309+0000 7f5cd0b86640 1 --2- 172.21.15.106:0/1406662030 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ccc157080 0x7f5ccc15cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:12.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.309+0000 7f5cd0b86640 1 --2- 172.21.15.106:0/1406662030 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ccc074910 0x7f5ccc0c9100 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:12.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.309+0000 7f5cd0b86640 1 -- 172.21.15.106:0/1406662030 >> 172.21.15.106:0/1406662030 conn(0x7f5ccc0ce570 msgr2=0x7f5ccc0ce980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:12.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.309+0000 7f5cd0b86640 1 -- 172.21.15.106:0/1406662030 shutdown_connections 2024-04-25T10:31:12.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.309+0000 7f5cd0b86640 1 -- 172.21.15.106:0/1406662030 wait complete. 2024-04-25T10:31:12.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.311+0000 7f5cd0b86640 1 Processor -- start 2024-04-25T10:31:12.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.311+0000 7f5cd0b86640 1 -- start start 2024-04-25T10:31:12.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.312+0000 7f5cd0b86640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ccc0684e0 0x7f5ccc0ccd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:12.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.312+0000 7f5cd0b86640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ccc074910 0x7f5ccc14e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:12.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.312+0000 7f5cd0b86640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ccc157080 0x7f5ccc06f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:12.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.312+0000 7f5cd0b86640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5ccc074110 con 0x7f5ccc074910 2024-04-25T10:31:12.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.312+0000 7f5cd0b86640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5ccc073ec0 con 0x7f5ccc0684e0 2024-04-25T10:31:12.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.312+0000 7f5cd0b86640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f5ccc0743f0 con 0x7f5ccc157080 2024-04-25T10:31:12.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.312+0000 7f5cc9d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ccc074910 0x7f5ccc14e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:12.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.312+0000 7f5cc9d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ccc074910 0x7f5ccc14e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59224/0 (socket says 172.21.15.106:59224) 2024-04-25T10:31:12.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.312+0000 7f5cc9d9b640 1 -- 172.21.15.106:0/1903655888 learned_addr learned my addr 172.21.15.106:0/1903655888 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:31:12.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.312+0000 7f5cca59c640 1 --2- 172.21.15.106:0/1903655888 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ccc0684e0 0x7f5ccc0ccd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:12.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.312+0000 7f5ccad9d640 1 --2- 172.21.15.106:0/1903655888 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ccc157080 0x7f5ccc06f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:12.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.312+0000 7f5cc9d9b640 1 -- 172.21.15.106:0/1903655888 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ccc157080 msgr2=0x7f5ccc06f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:12.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.312+0000 7f5cc9d9b640 1 --2- 172.21.15.106:0/1903655888 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ccc157080 0x7f5ccc06f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:12.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.312+0000 7f5cc9d9b640 1 -- 172.21.15.106:0/1903655888 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ccc0684e0 msgr2=0x7f5ccc0ccd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:12.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.312+0000 7f5cc9d9b640 1 --2- 172.21.15.106:0/1903655888 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ccc0684e0 0x7f5ccc0ccd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:12.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.312+0000 7f5cc9d9b640 1 -- 172.21.15.106:0/1903655888 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ccc14eba0 con 0x7f5ccc074910 2024-04-25T10:31:12.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.313+0000 7f5cc9d9b640 1 --2- 172.21.15.106:0/1903655888 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ccc074910 0x7f5ccc14e660 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5cc000d800 tx=0x7f5cc000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:12.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.313+0000 7f5cbaffd640 1 -- 172.21.15.106:0/1903655888 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc0004030 con 0x7f5ccc074910 2024-04-25T10:31:12.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.313+0000 7f5cbaffd640 1 -- 172.21.15.106:0/1903655888 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cc00041d0 con 0x7f5ccc074910 2024-04-25T10:31:12.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.313+0000 7f5cd0b86640 1 -- 172.21.15.106:0/1903655888 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ccc15b2b0 con 0x7f5ccc074910 2024-04-25T10:31:12.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.313+0000 7f5cbaffd640 1 -- 172.21.15.106:0/1903655888 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc00096a0 con 0x7f5ccc074910 2024-04-25T10:31:12.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.313+0000 7f5cd0b86640 1 -- 172.21.15.106:0/1903655888 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ccc15b870 con 0x7f5ccc074910 2024-04-25T10:31:12.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.314+0000 7f5cd0b86640 1 -- 172.21.15.106:0/1903655888 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ccc06a3d0 con 0x7f5ccc074910 2024-04-25T10:31:12.318 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.314+0000 7f5cbaffd640 1 -- 172.21.15.106:0/1903655888 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5cc0009840 con 0x7f5ccc074910 2024-04-25T10:31:12.318 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.315+0000 7f5cbaffd640 1 --2- 172.21.15.106:0/1903655888 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f5cac0744b0 0x7f5cac0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:12.318 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.315+0000 7f5cbaffd640 1 -- 172.21.15.106:0/1903655888 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5cc0095380 con 0x7f5ccc074910 2024-04-25T10:31:12.318 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.315+0000 7f5cca59c640 1 --2- 172.21.15.106:0/1903655888 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f5cac0744b0 0x7f5cac0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:12.318 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.315+0000 7f5cca59c640 1 --2- 172.21.15.106:0/1903655888 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f5cac0744b0 0x7f5cac0769c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc033000 tx=0x7f5cbc009000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:12.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.320+0000 7f5cbaffd640 1 -- 172.21.15.106:0/1903655888 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5cc005f4c0 con 0x7f5ccc074910 2024-04-25T10:31:12.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.496+0000 7f5cd0b86640 1 -- 172.21.15.106:0/1903655888 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5ccc0ced00 con 0x7f5ccc074910 2024-04-25T10:31:12.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.496+0000 7f5cbaffd640 1 -- 172.21.15.106:0/1903655888 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f5cc00637c0 con 0x7f5ccc074910 2024-04-25T10:31:12.499 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-25T10:31:12.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.498+0000 7f5cd0b86640 1 -- 172.21.15.106:0/1903655888 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f5cac0744b0 msgr2=0x7f5cac0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:12.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.498+0000 7f5cd0b86640 1 --2- 172.21.15.106:0/1903655888 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f5cac0744b0 0x7f5cac0769c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc033000 tx=0x7f5cbc009000 comp rx=0 tx=0).stop 2024-04-25T10:31:12.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.498+0000 7f5cd0b86640 1 -- 172.21.15.106:0/1903655888 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ccc074910 msgr2=0x7f5ccc14e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:12.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.498+0000 7f5cd0b86640 1 --2- 172.21.15.106:0/1903655888 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ccc074910 0x7f5ccc14e660 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5cc000d800 tx=0x7f5cc000dcc0 comp rx=0 tx=0).stop 2024-04-25T10:31:12.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.498+0000 7f5cd0b86640 1 -- 172.21.15.106:0/1903655888 shutdown_connections 2024-04-25T10:31:12.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.498+0000 7f5cd0b86640 1 --2- 172.21.15.106:0/1903655888 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f5cac0744b0 0x7f5cac0769c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:12.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.498+0000 7f5cd0b86640 1 --2- 172.21.15.106:0/1903655888 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ccc157080 0x7f5ccc06f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:12.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.498+0000 7f5cd0b86640 1 --2- 172.21.15.106:0/1903655888 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ccc074910 0x7f5ccc14e660 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:12.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.498+0000 7f5cd0b86640 1 --2- 172.21.15.106:0/1903655888 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ccc0684e0 0x7f5ccc0ccd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:12.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.498+0000 7f5cd0b86640 1 -- 172.21.15.106:0/1903655888 >> 172.21.15.106:0/1903655888 conn(0x7f5ccc0ce570 msgr2=0x7f5ccc155db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:12.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.498+0000 7f5cd0b86640 1 -- 172.21.15.106:0/1903655888 shutdown_connections 2024-04-25T10:31:12.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.498+0000 7f5cd0b86640 1 -- 172.21.15.106:0/1903655888 wait complete. 2024-04-25T10:31:12.512 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:31:12.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.621+0000 7f9ef571f640 1 Processor -- start 2024-04-25T10:31:12.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.621+0000 7f9ef571f640 1 -- start start 2024-04-25T10:31:12.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.623+0000 7f9ef571f640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9ef00c7ad0 0x7f9ef006ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:12.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.623+0000 7f9ef571f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ef0156f70 0x7f9ef006d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:12.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.623+0000 7f9ef571f640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f9ef006daa0 0x7f9ef00c8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:12.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.623+0000 7f9ef571f640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f9ef015d6d0 con 0x7f9ef00c7ad0 2024-04-25T10:31:12.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.623+0000 7f9ef571f640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f9ef015d9b0 con 0x7f9ef006daa0 2024-04-25T10:31:12.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.623+0000 7f9ef571f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f9ef00c7150 con 0x7f9ef0156f70 2024-04-25T10:31:12.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.623+0000 7f9eeeffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9ef00c7ad0 0x7f9ef006ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:12.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.623+0000 7f9eeeffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9ef00c7ad0 0x7f9ef006ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59232/0 (socket says 172.21.15.106:59232) 2024-04-25T10:31:12.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.623+0000 7f9eeeffd640 1 -- 172.21.15.106:0/740164515 learned_addr learned my addr 172.21.15.106:0/740164515 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:31:12.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.623+0000 7f9eeeffd640 1 -- 172.21.15.106:0/740164515 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ef0156f70 msgr2=0x7f9ef006d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:12.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.623+0000 7f9eeeffd640 1 --2- 172.21.15.106:0/740164515 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ef0156f70 0x7f9ef006d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:12.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.623+0000 7f9eeeffd640 1 -- 172.21.15.106:0/740164515 >> v2:172.21.15.134:3300/0 conn(0x7f9ef006daa0 msgr2=0x7f9ef00c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:12.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.623+0000 7f9eeeffd640 1 --2- 172.21.15.106:0/740164515 >> v2:172.21.15.134:3300/0 conn(0x7f9ef006daa0 0x7f9ef00c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:12.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.623+0000 7f9eeeffd640 1 -- 172.21.15.106:0/740164515 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ef00c7430 con 0x7f9ef00c7ad0 2024-04-25T10:31:12.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.624+0000 7f9eeeffd640 1 --2- 172.21.15.106:0/740164515 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ef00c7ad0 0x7f9ef006ea60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9ef00b51d0 tx=0x7f9ee002a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5f670111498d5ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:12.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.626+0000 7f9eedffb640 1 -- 172.21.15.106:0/740164515 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee0037070 con 0x7f9ef00c7ad0 2024-04-25T10:31:12.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.626+0000 7f9eedffb640 1 -- 172.21.15.106:0/740164515 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ee0032440 con 0x7f9ef00c7ad0 2024-04-25T10:31:12.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.626+0000 7f9eedffb640 1 -- 172.21.15.106:0/740164515 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee0032810 con 0x7f9ef00c7ad0 2024-04-25T10:31:12.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.627+0000 7f9ef571f640 1 -- 172.21.15.106:0/740164515 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ef00c7ad0 msgr2=0x7f9ef006ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:12.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.627+0000 7f9ef571f640 1 --2- 172.21.15.106:0/740164515 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ef00c7ad0 0x7f9ef006ea60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9ef00b51d0 tx=0x7f9ee002a200 comp rx=0 tx=0).stop 2024-04-25T10:31:12.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.628+0000 7f9ef571f640 1 -- 172.21.15.106:0/740164515 shutdown_connections 2024-04-25T10:31:12.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.628+0000 7f9ef571f640 1 --2- 172.21.15.106:0/740164515 >> v2:172.21.15.134:3300/0 conn(0x7f9ef006daa0 0x7f9ef00c8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:12.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.628+0000 7f9ef571f640 1 --2- 172.21.15.106:0/740164515 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ef0156f70 0x7f9ef006d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:12.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.628+0000 7f9ef571f640 1 --2- 172.21.15.106:0/740164515 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ef00c7ad0 0x7f9ef006ea60 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:12.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.628+0000 7f9ef571f640 1 -- 172.21.15.106:0/740164515 >> 172.21.15.106:0/740164515 conn(0x7f9ef0068ed0 msgr2=0x7f9ef00d20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:12.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.628+0000 7f9ef571f640 1 -- 172.21.15.106:0/740164515 shutdown_connections 2024-04-25T10:31:12.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.628+0000 7f9ef571f640 1 -- 172.21.15.106:0/740164515 wait complete. 2024-04-25T10:31:12.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.630+0000 7f9ef571f640 1 Processor -- start 2024-04-25T10:31:12.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.630+0000 7f9ef571f640 1 -- start start 2024-04-25T10:31:12.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.632+0000 7f9ef571f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ef006daa0 0x7f9ef0159dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:12.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.632+0000 7f9ef571f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ef00c7ad0 0x7f9ef00cb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:12.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.632+0000 7f9ef571f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ef0156f70 0x7f9ef0152a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:12.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.632+0000 7f9ef571f640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef00ce6c0 con 0x7f9ef00c7ad0 2024-04-25T10:31:12.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.632+0000 7f9ef571f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef00ce360 con 0x7f9ef006daa0 2024-04-25T10:31:12.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.632+0000 7f9ef571f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f9ef00c8660 con 0x7f9ef0156f70 2024-04-25T10:31:12.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.632+0000 7f9eee7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ef00c7ad0 0x7f9ef00cb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:12.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.632+0000 7f9eee7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ef00c7ad0 0x7f9ef00cb7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59246/0 (socket says 172.21.15.106:59246) 2024-04-25T10:31:12.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.632+0000 7f9eee7fc640 1 -- 172.21.15.106:0/2792316240 learned_addr learned my addr 172.21.15.106:0/2792316240 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:31:12.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.632+0000 7f9eeeffd640 1 --2- 172.21.15.106:0/2792316240 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ef006daa0 0x7f9ef0159dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:12.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.632+0000 7f9eef7fe640 1 --2- 172.21.15.106:0/2792316240 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ef0156f70 0x7f9ef0152a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:12.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.632+0000 7f9eee7fc640 1 -- 172.21.15.106:0/2792316240 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ef0156f70 msgr2=0x7f9ef0152a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:12.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.632+0000 7f9eee7fc640 1 --2- 172.21.15.106:0/2792316240 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ef0156f70 0x7f9ef0152a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:12.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.632+0000 7f9eee7fc640 1 -- 172.21.15.106:0/2792316240 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ef006daa0 msgr2=0x7f9ef0159dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:12.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.632+0000 7f9eee7fc640 1 --2- 172.21.15.106:0/2792316240 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ef006daa0 0x7f9ef0159dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:12.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.632+0000 7f9eee7fc640 1 -- 172.21.15.106:0/2792316240 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ef015a300 con 0x7f9ef00c7ad0 2024-04-25T10:31:12.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.632+0000 7f9eeeffd640 1 --2- 172.21.15.106:0/2792316240 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ef006daa0 0x7f9ef0159dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:31:12.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.633+0000 7f9eee7fc640 1 --2- 172.21.15.106:0/2792316240 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ef00c7ad0 0x7f9ef00cb7a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9ee400b440 tx=0x7f9ee400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:12.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.633+0000 7f9ed77fe640 1 -- 172.21.15.106:0/2792316240 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee4015020 con 0x7f9ef00c7ad0 2024-04-25T10:31:12.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.633+0000 7f9ed77fe640 1 -- 172.21.15.106:0/2792316240 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ee4011830 con 0x7f9ef00c7ad0 2024-04-25T10:31:12.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.633+0000 7f9ed77fe640 1 -- 172.21.15.106:0/2792316240 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee4011c20 con 0x7f9ef00c7ad0 2024-04-25T10:31:12.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.633+0000 7f9ef571f640 1 -- 172.21.15.106:0/2792316240 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ef01f6dc0 con 0x7f9ef00c7ad0 2024-04-25T10:31:12.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.633+0000 7f9ef571f640 1 -- 172.21.15.106:0/2792316240 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ef01f7380 con 0x7f9ef00c7ad0 2024-04-25T10:31:12.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.634+0000 7f9ef571f640 1 -- 172.21.15.106:0/2792316240 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ef0156bb0 con 0x7f9ef00c7ad0 2024-04-25T10:31:12.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.635+0000 7f9ed77fe640 1 -- 172.21.15.106:0/2792316240 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9ee4018020 con 0x7f9ef00c7ad0 2024-04-25T10:31:12.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.635+0000 7f9ed77fe640 1 --2- 172.21.15.106:0/2792316240 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f9edc0744e0 0x7f9edc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:12.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.635+0000 7f9ed77fe640 1 -- 172.21.15.106:0/2792316240 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5825+0+0 (secure 0 0 0) 0x7f9ee4095060 con 0x7f9ef00c7ad0 2024-04-25T10:31:12.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.635+0000 7f9eeeffd640 1 --2- 172.21.15.106:0/2792316240 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f9edc0744e0 0x7f9edc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:12.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.636+0000 7f9eeeffd640 1 --2- 172.21.15.106:0/2792316240 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f9edc0744e0 0x7f9edc0769f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9ee0034000 tx=0x7f9ee002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:12.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.640+0000 7f9ed77fe640 1 -- 172.21.15.106:0/2792316240 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9ef0156bb0 con 0x7f9ef00c7ad0 2024-04-25T10:31:12.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.813+0000 7f9ef571f640 1 -- 172.21.15.106:0/2792316240 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9ef006a3d0 con 0x7f9ef00c7ad0 2024-04-25T10:31:12.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.814+0000 7f9ed77fe640 1 -- 172.21.15.106:0/2792316240 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f9ee401f020 con 0x7f9ef00c7ad0 2024-04-25T10:31:12.816 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-25T10:31:12.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.815+0000 7f9ef571f640 1 -- 172.21.15.106:0/2792316240 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f9edc0744e0 msgr2=0x7f9edc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:12.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.816+0000 7f9ef571f640 1 --2- 172.21.15.106:0/2792316240 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f9edc0744e0 0x7f9edc0769f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9ee0034000 tx=0x7f9ee002e000 comp rx=0 tx=0).stop 2024-04-25T10:31:12.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.816+0000 7f9ef571f640 1 -- 172.21.15.106:0/2792316240 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ef00c7ad0 msgr2=0x7f9ef00cb7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:12.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.816+0000 7f9ef571f640 1 --2- 172.21.15.106:0/2792316240 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ef00c7ad0 0x7f9ef00cb7a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9ee400b440 tx=0x7f9ee400b900 comp rx=0 tx=0).stop 2024-04-25T10:31:12.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.816+0000 7f9ef571f640 1 -- 172.21.15.106:0/2792316240 shutdown_connections 2024-04-25T10:31:12.819 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.816+0000 7f9ef571f640 1 --2- 172.21.15.106:0/2792316240 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f9edc0744e0 0x7f9edc0769f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:12.819 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.816+0000 7f9ef571f640 1 --2- 172.21.15.106:0/2792316240 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ef0156f70 0x7f9ef0152a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:12.819 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.816+0000 7f9ef571f640 1 --2- 172.21.15.106:0/2792316240 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ef00c7ad0 0x7f9ef00cb7a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:12.819 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.816+0000 7f9ef571f640 1 --2- 172.21.15.106:0/2792316240 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ef006daa0 0x7f9ef0159dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:12.819 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.816+0000 7f9ef571f640 1 -- 172.21.15.106:0/2792316240 >> 172.21.15.106:0/2792316240 conn(0x7f9ef0068ed0 msgr2=0x7f9ef00c8980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:12.819 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.816+0000 7f9ef571f640 1 -- 172.21.15.106:0/2792316240 shutdown_connections 2024-04-25T10:31:12.819 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.816+0000 7f9ef571f640 1 -- 172.21.15.106:0/2792316240 wait complete. 2024-04-25T10:31:12.829 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:31:12.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.936+0000 7fe02299d640 1 Processor -- start 2024-04-25T10:31:12.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.936+0000 7fe02299d640 1 -- start start 2024-04-25T10:31:12.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.938+0000 7fe02299d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe01c0c7940 0x7fe01c0c7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:12.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.938+0000 7fe02299d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe01c06ed10 0x7fe01c06f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:12.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.938+0000 7fe02299d640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe01c15a940 con 0x7fe01c06ccd0 2024-04-25T10:31:12.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.938+0000 7fe02299d640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fe01c154e50 con 0x7fe01c06ed10 2024-04-25T10:31:12.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.938+0000 7fe02299d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe01c15ac20 con 0x7fe01c0c7940 2024-04-25T10:31:12.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.938+0000 7fe02219c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe01c06ed10 0x7fe01c06f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:12.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.938+0000 7fe02219c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe01c06ed10 0x7fe01c06f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59258/0 (socket says 172.21.15.106:59258) 2024-04-25T10:31:12.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.938+0000 7fe02219c640 1 -- 172.21.15.106:0/4163833840 learned_addr learned my addr 172.21.15.106:0/4163833840 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:31:12.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.938+0000 7fe02119a640 1 --2- 172.21.15.106:0/4163833840 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe01c0c7940 0x7fe01c0c7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:12.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.938+0000 7fe02219c640 1 -- 172.21.15.106:0/4163833840 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe01c0c7940 msgr2=0x7fe01c0c7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:12.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.938+0000 7fe02219c640 1 --2- 172.21.15.106:0/4163833840 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe01c0c7940 0x7fe01c0c7d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:12.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.938+0000 7fe02219c640 1 -- 172.21.15.106:0/4163833840 >> v1:172.21.15.134:6789/0 conn(0x7fe01c06ccd0 legacy=0x7fe01c154740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:12.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.938+0000 7fe02219c640 1 -- 172.21.15.106:0/4163833840 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe01c15af00 con 0x7fe01c06ed10 2024-04-25T10:31:12.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.939+0000 7fe02219c640 1 --2- 172.21.15.106:0/4163833840 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe01c06ed10 0x7fe01c06f120 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe014009870 tx=0x7fe01402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=790419eb1474834e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:12.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.939+0000 7fe020999640 1 -- 172.21.15.106:0/4163833840 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe014037070 con 0x7fe01c06ed10 2024-04-25T10:31:12.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.939+0000 7fe020999640 1 -- 172.21.15.106:0/4163833840 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe014032440 con 0x7fe01c06ed10 2024-04-25T10:31:12.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.939+0000 7fe020999640 1 -- 172.21.15.106:0/4163833840 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe014032810 con 0x7fe01c06ed10 2024-04-25T10:31:12.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.939+0000 7fe02299d640 1 -- 172.21.15.106:0/4163833840 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe01c06ed10 msgr2=0x7fe01c06f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:12.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.939+0000 7fe02299d640 1 --2- 172.21.15.106:0/4163833840 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe01c06ed10 0x7fe01c06f120 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe014009870 tx=0x7fe01402a2a0 comp rx=0 tx=0).stop 2024-04-25T10:31:12.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.939+0000 7fe02299d640 1 -- 172.21.15.106:0/4163833840 shutdown_connections 2024-04-25T10:31:12.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.939+0000 7fe02299d640 1 --2- 172.21.15.106:0/4163833840 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe01c06ed10 0x7fe01c06f120 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:12.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.939+0000 7fe02299d640 1 --2- 172.21.15.106:0/4163833840 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe01c0c7940 0x7fe01c0c7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:12.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.940+0000 7fe02299d640 1 -- 172.21.15.106:0/4163833840 >> 172.21.15.106:0/4163833840 conn(0x7fe01c0ce830 msgr2=0x7fe01c06c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:12.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.940+0000 7fe02299d640 1 -- 172.21.15.106:0/4163833840 shutdown_connections 2024-04-25T10:31:12.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.940+0000 7fe02299d640 1 -- 172.21.15.106:0/4163833840 wait complete. 2024-04-25T10:31:12.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.941+0000 7fe02299d640 1 Processor -- start 2024-04-25T10:31:12.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.941+0000 7fe02299d640 1 -- start start 2024-04-25T10:31:12.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.942+0000 7fe02299d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe01c06ccd0 0x7fe01c1f0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:12.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.942+0000 7fe02299d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe01c06ed10 0x7fe01c14d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:12.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.942+0000 7fe02299d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe01c0c7940 0x7fe01c1482c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:12.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.942+0000 7fe02299d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe01c0c9610 con 0x7fe01c06ccd0 2024-04-25T10:31:12.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.942+0000 7fe02299d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe01c06f660 con 0x7fe01c06ed10 2024-04-25T10:31:12.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.942+0000 7fe02299d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe01c0b45f0 con 0x7fe01c0c7940 2024-04-25T10:31:12.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.943+0000 7fe02199b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe01c06ccd0 0x7fe01c1f0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:12.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.943+0000 7fe02199b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe01c06ccd0 0x7fe01c1f0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59262/0 (socket says 172.21.15.106:59262) 2024-04-25T10:31:12.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.943+0000 7fe02199b640 1 -- 172.21.15.106:0/3552749099 learned_addr learned my addr 172.21.15.106:0/3552749099 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:31:12.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.943+0000 7fe02219c640 1 --2- 172.21.15.106:0/3552749099 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe01c0c7940 0x7fe01c1482c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:12.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.943+0000 7fe02119a640 1 --2- 172.21.15.106:0/3552749099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe01c06ed10 0x7fe01c14d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:12.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.943+0000 7fe02199b640 1 -- 172.21.15.106:0/3552749099 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe01c0c7940 msgr2=0x7fe01c1482c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:12.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.943+0000 7fe02199b640 1 --2- 172.21.15.106:0/3552749099 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe01c0c7940 0x7fe01c1482c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:12.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.943+0000 7fe02199b640 1 -- 172.21.15.106:0/3552749099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe01c06ed10 msgr2=0x7fe01c14d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:12.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.943+0000 7fe02199b640 1 --2- 172.21.15.106:0/3552749099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe01c06ed10 0x7fe01c14d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:12.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.943+0000 7fe02199b640 1 -- 172.21.15.106:0/3552749099 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe01c1f1230 con 0x7fe01c06ccd0 2024-04-25T10:31:12.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.943+0000 7fe02199b640 1 --2- 172.21.15.106:0/3552749099 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe01c06ccd0 0x7fe01c1f0cf0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe01800ba50 tx=0x7fe01800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:12.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.943+0000 7fe0127fc640 1 -- 172.21.15.106:0/3552749099 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe018019070 con 0x7fe01c06ccd0 2024-04-25T10:31:12.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.943+0000 7fe0127fc640 1 -- 172.21.15.106:0/3552749099 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe01800cbc0 con 0x7fe01c06ccd0 2024-04-25T10:31:12.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.943+0000 7fe0127fc640 1 -- 172.21.15.106:0/3552749099 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe018014440 con 0x7fe01c06ccd0 2024-04-25T10:31:12.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.943+0000 7fe02299d640 1 -- 172.21.15.106:0/3552749099 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe01c0cd940 con 0x7fe01c06ccd0 2024-04-25T10:31:12.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.943+0000 7fe02299d640 1 -- 172.21.15.106:0/3552749099 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe01c0cdf00 con 0x7fe01c06ccd0 2024-04-25T10:31:12.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.944+0000 7fe02299d640 1 -- 172.21.15.106:0/3552749099 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe01c1f1400 con 0x7fe01c06ccd0 2024-04-25T10:31:12.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.945+0000 7fe0127fc640 1 -- 172.21.15.106:0/3552749099 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe01801c070 con 0x7fe01c06ccd0 2024-04-25T10:31:12.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.945+0000 7fe0127fc640 1 --2- 172.21.15.106:0/3552749099 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fe0000744b0 0x7fe0000769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:12.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.946+0000 7fe02119a640 1 --2- 172.21.15.106:0/3552749099 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fe0000744b0 0x7fe0000769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:12.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.946+0000 7fe0127fc640 1 -- 172.21.15.106:0/3552749099 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5825+0+0 (secure 0 0 0) 0x7fe018094f40 con 0x7fe01c06ccd0 2024-04-25T10:31:12.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.946+0000 7fe02119a640 1 --2- 172.21.15.106:0/3552749099 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fe0000744b0 0x7fe0000769c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe01c068910 tx=0x7fe00c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:12.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:12.950+0000 7fe0127fc640 1 -- 172.21.15.106:0/3552749099 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe01c1f1400 con 0x7fe01c06ccd0 2024-04-25T10:31:13.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.126+0000 7fe02299d640 1 -- 172.21.15.106:0/3552749099 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe01c050740 con 0x7fe01c06ccd0 2024-04-25T10:31:13.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.127+0000 7fe0127fc640 1 -- 172.21.15.106:0/3552749099 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7fe01805f000 con 0x7fe01c06ccd0 2024-04-25T10:31:13.129 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-25T10:31:13.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.128+0000 7fe02299d640 1 -- 172.21.15.106:0/3552749099 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fe0000744b0 msgr2=0x7fe0000769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:13.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.128+0000 7fe02299d640 1 --2- 172.21.15.106:0/3552749099 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fe0000744b0 0x7fe0000769c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe01c068910 tx=0x7fe00c008040 comp rx=0 tx=0).stop 2024-04-25T10:31:13.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.128+0000 7fe02299d640 1 -- 172.21.15.106:0/3552749099 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe01c06ccd0 msgr2=0x7fe01c1f0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:13.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.128+0000 7fe02299d640 1 --2- 172.21.15.106:0/3552749099 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe01c06ccd0 0x7fe01c1f0cf0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe01800ba50 tx=0x7fe01800bf10 comp rx=0 tx=0).stop 2024-04-25T10:31:13.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.128+0000 7fe02299d640 1 -- 172.21.15.106:0/3552749099 shutdown_connections 2024-04-25T10:31:13.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.128+0000 7fe02299d640 1 --2- 172.21.15.106:0/3552749099 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fe0000744b0 0x7fe0000769c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:13.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.128+0000 7fe02299d640 1 --2- 172.21.15.106:0/3552749099 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe01c0c7940 0x7fe01c1482c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:13.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.129+0000 7fe02299d640 1 --2- 172.21.15.106:0/3552749099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe01c06ed10 0x7fe01c14d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:13.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.129+0000 7fe02299d640 1 --2- 172.21.15.106:0/3552749099 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe01c06ccd0 0x7fe01c1f0cf0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:13.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.129+0000 7fe02299d640 1 -- 172.21.15.106:0/3552749099 >> 172.21.15.106:0/3552749099 conn(0x7fe01c0ce830 msgr2=0x7fe01c073090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:13.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.129+0000 7fe02299d640 1 -- 172.21.15.106:0/3552749099 shutdown_connections 2024-04-25T10:31:13.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.129+0000 7fe02299d640 1 -- 172.21.15.106:0/3552749099 wait complete. 2024-04-25T10:31:13.142 DEBUG:tasks.mon_thrash:Yielding 2024-04-25T10:31:13.142 INFO:teuthology.run_tasks:Running task workunit... 2024-04-25T10:31:13.154 INFO:tasks.workunit:Pulling workunits from ref 994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T10:31:13.155 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-25T10:31:13.155 INFO:tasks.workunit:timeout=3h 2024-04-25T10:31:13.155 INFO:tasks.workunit:cleanup=True 2024-04-25T10:31:13.155 DEBUG:teuthology.orchestra.run.smithi106:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-25T10:31:13.156 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:31:13.171 INFO:teuthology.orchestra.run.smithi106.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-25T10:31:13.172 INFO:teuthology.orchestra.run.smithi106.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-04-25T10:31:13.172 INFO:teuthology.orchestra.run.smithi106.stdout:Device: 2bh/43d Inode: 1 Links: 2 2024-04-25T10:31:13.172 INFO:teuthology.orchestra.run.smithi106.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:31:13.172 INFO:teuthology.orchestra.run.smithi106.stdout:Context: system_u:object_r:fusefs_t:s0 2024-04-25T10:31:13.172 INFO:teuthology.orchestra.run.smithi106.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-04-25T10:31:13.172 INFO:teuthology.orchestra.run.smithi106.stdout:Modify: 2024-04-25 10:30:37.426318942 +0000 2024-04-25T10:31:13.172 INFO:teuthology.orchestra.run.smithi106.stdout:Change: 2024-04-25 10:31:09.791443201 +0000 2024-04-25T10:31:13.172 INFO:teuthology.orchestra.run.smithi106.stdout: Birth: - 2024-04-25T10:31:13.172 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-25T10:31:13.172 DEBUG:teuthology.orchestra.run.smithi106:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-25T10:31:13.246 DEBUG:teuthology.orchestra.run.smithi106:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T10:31:13.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.264+0000 7fd920746640 1 Processor -- start 2024-04-25T10:31:13.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.264+0000 7fd920746640 1 -- start start 2024-04-25T10:31:13.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.265+0000 7fd920746640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fd9180cc250 0x7fd918159a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:13.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.265+0000 7fd920746640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd9180c6a80 0x7fd9180c8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:13.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.265+0000 7fd920746640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9180c9d40 con 0x7fd9180cc6b0 2024-04-25T10:31:13.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.265+0000 7fd920746640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fd918159fa0 con 0x7fd9180cc250 2024-04-25T10:31:13.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.265+0000 7fd920746640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fd9180c64c0 con 0x7fd9180c6a80 2024-04-25T10:31:13.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.265+0000 7fd91e4e2640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fd9180cc250 0x7fd918159a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:13.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.265+0000 7fd91dce1640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fd9180cc6b0 0x7fd918153c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:46706/0 (socket says 172.21.15.106:46706) 2024-04-25T10:31:13.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.265+0000 7fd91dce1640 1 -- 172.21.15.106:0/3351372586 learned_addr learned my addr 172.21.15.106:0/3351372586 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:31:13.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.265+0000 7fd91ece3640 1 --2- 172.21.15.106:0/3351372586 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd9180c6a80 0x7fd9180c8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:13.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.266+0000 7fd91e4e2640 1 -- 172.21.15.106:0/3351372586 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd9180c6a80 msgr2=0x7fd9180c8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:13.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.266+0000 7fd91e4e2640 1 --2- 172.21.15.106:0/3351372586 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd9180c6a80 0x7fd9180c8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:13.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.266+0000 7fd91e4e2640 1 -- 172.21.15.106:0/3351372586 >> v1:172.21.15.106:6789/0 conn(0x7fd9180cc6b0 legacy=0x7fd918153c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:13.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.266+0000 7fd91d4e0640 1 -- 172.21.15.106:0/3351372586 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 830248317 0 0) 0x7fd9180c9d40 con 0x7fd9180cc6b0 2024-04-25T10:31:13.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.266+0000 7fd91e4e2640 1 -- 172.21.15.106:0/3351372586 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9180c67a0 con 0x7fd9180cc250 2024-04-25T10:31:13.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.266+0000 7fd91ece3640 1 --2- 172.21.15.106:0/3351372586 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd9180c6a80 0x7fd9180c8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:31:13.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.267+0000 7fd91e4e2640 1 --2- 172.21.15.106:0/3351372586 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd9180cc250 0x7fd918159a60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd908009d90 tx=0x7fd90802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=882a2170eb4cd4e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:13.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.268+0000 7fd91d4e0640 1 -- 172.21.15.106:0/3351372586 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd908007740 con 0x7fd9180cc250 2024-04-25T10:31:13.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.268+0000 7fd91d4e0640 1 -- 172.21.15.106:0/3351372586 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9080078e0 con 0x7fd9180cc250 2024-04-25T10:31:13.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.268+0000 7fd91d4e0640 1 -- 172.21.15.106:0/3351372586 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd908007cb0 con 0x7fd9180cc250 2024-04-25T10:31:13.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.268+0000 7fd920746640 1 -- 172.21.15.106:0/3351372586 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd9180cc250 msgr2=0x7fd918159a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:13.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.268+0000 7fd920746640 1 --2- 172.21.15.106:0/3351372586 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd9180cc250 0x7fd918159a60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd908009d90 tx=0x7fd90802a660 comp rx=0 tx=0).stop 2024-04-25T10:31:13.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.268+0000 7fd920746640 1 -- 172.21.15.106:0/3351372586 shutdown_connections 2024-04-25T10:31:13.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.268+0000 7fd920746640 1 --2- 172.21.15.106:0/3351372586 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd9180c6a80 0x7fd9180c8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:13.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.268+0000 7fd920746640 1 --2- 172.21.15.106:0/3351372586 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd9180cc250 0x7fd918159a60 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:13.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.268+0000 7fd920746640 1 -- 172.21.15.106:0/3351372586 >> 172.21.15.106:0/3351372586 conn(0x7fd91806d6d0 msgr2=0x7fd91806dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:13.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.268+0000 7fd920746640 1 -- 172.21.15.106:0/3351372586 shutdown_connections 2024-04-25T10:31:13.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.268+0000 7fd920746640 1 -- 172.21.15.106:0/3351372586 wait complete. 2024-04-25T10:31:13.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.271+0000 7fd920746640 1 Processor -- start 2024-04-25T10:31:13.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.271+0000 7fd920746640 1 -- start start 2024-04-25T10:31:13.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.272+0000 7fd920746640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9180c6a80 0x7fd9180ce180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:13.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.272+0000 7fd920746640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd9180cc6b0 0x7fd91814e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:13.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.272+0000 7fd920746640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd918154c80 0x7fd91814ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:13.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.272+0000 7fd920746640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd9180c9920 con 0x7fd918154c80 2024-04-25T10:31:13.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.272+0000 7fd920746640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd9180c7450 con 0x7fd9180c6a80 2024-04-25T10:31:13.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.272+0000 7fd920746640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fd9180c9aa0 con 0x7fd9180cc6b0 2024-04-25T10:31:13.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.272+0000 7fd91ece3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd918154c80 0x7fd91814ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:13.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.272+0000 7fd91ece3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd918154c80 0x7fd91814ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59278/0 (socket says 172.21.15.106:59278) 2024-04-25T10:31:13.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.272+0000 7fd91ece3640 1 -- 172.21.15.106:0/972223510 learned_addr learned my addr 172.21.15.106:0/972223510 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:31:13.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.272+0000 7fd91dce1640 1 --2- 172.21.15.106:0/972223510 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd9180cc6b0 0x7fd91814e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:13.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.272+0000 7fd91e4e2640 1 --2- 172.21.15.106:0/972223510 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9180c6a80 0x7fd9180ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:13.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.272+0000 7fd91ece3640 1 -- 172.21.15.106:0/972223510 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd9180cc6b0 msgr2=0x7fd91814e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:13.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.272+0000 7fd91ece3640 1 --2- 172.21.15.106:0/972223510 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd9180cc6b0 0x7fd91814e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:13.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.272+0000 7fd91ece3640 1 -- 172.21.15.106:0/972223510 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9180c6a80 msgr2=0x7fd9180ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:13.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.272+0000 7fd91ece3640 1 --2- 172.21.15.106:0/972223510 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9180c6a80 0x7fd9180ce180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:13.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.272+0000 7fd91ece3640 1 -- 172.21.15.106:0/972223510 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9181549f0 con 0x7fd918154c80 2024-04-25T10:31:13.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.273+0000 7fd91ece3640 1 --2- 172.21.15.106:0/972223510 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd918154c80 0x7fd91814ea70 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd91000d800 tx=0x7fd91000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:13.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.273+0000 7fd916ffd640 1 -- 172.21.15.106:0/972223510 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd910004030 con 0x7fd918154c80 2024-04-25T10:31:13.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.273+0000 7fd916ffd640 1 -- 172.21.15.106:0/972223510 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9100041d0 con 0x7fd918154c80 2024-04-25T10:31:13.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.273+0000 7fd920746640 1 -- 172.21.15.106:0/972223510 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9181558f0 con 0x7fd918154c80 2024-04-25T10:31:13.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.273+0000 7fd916ffd640 1 -- 172.21.15.106:0/972223510 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9100096a0 con 0x7fd918154c80 2024-04-25T10:31:13.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.273+0000 7fd920746640 1 -- 172.21.15.106:0/972223510 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd918155eb0 con 0x7fd918154c80 2024-04-25T10:31:13.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.274+0000 7fd920746640 1 -- 172.21.15.106:0/972223510 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd91806a3d0 con 0x7fd918154c80 2024-04-25T10:31:13.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.275+0000 7fd916ffd640 1 -- 172.21.15.106:0/972223510 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd910009840 con 0x7fd918154c80 2024-04-25T10:31:13.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.275+0000 7fd916ffd640 1 --2- 172.21.15.106:0/972223510 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fd9040744b0 0x7fd9040769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:13.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.275+0000 7fd91e4e2640 1 --2- 172.21.15.106:0/972223510 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fd9040744b0 0x7fd9040769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:13.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.275+0000 7fd916ffd640 1 -- 172.21.15.106:0/972223510 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5825+0+0 (secure 0 0 0) 0x7fd9100954d0 con 0x7fd918154c80 2024-04-25T10:31:13.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.276+0000 7fd91e4e2640 1 --2- 172.21.15.106:0/972223510 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fd9040744b0 0x7fd9040769c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd908034000 tx=0x7fd90802f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:13.282 INFO:tasks.workunit.client.0.smithi106.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-25T10:31:13.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.280+0000 7fd916ffd640 1 -- 172.21.15.106:0/972223510 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd91005f590 con 0x7fd918154c80 2024-04-25T10:31:13.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.512+0000 7fd920746640 1 -- 172.21.15.106:0/972223510 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd9181572f0 con 0x7fd918154c80 2024-04-25T10:31:13.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.512+0000 7fd916ffd640 1 -- 172.21.15.106:0/972223510 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+6922 (secure 0 0 0) 0x7fd910063890 con 0x7fd918154c80 2024-04-25T10:31:13.515 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:31:13.515 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/2834155852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":2834155852},{"type":"v1","addr":"172.21.15.106:6835","nonce":2834155852}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:30:31.761107+0000","modified":"2024-04-25T10:31:00.589522+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4290,"mds_1":4287,"mds_2":4284,"mds_3":4269,"mds_4":4266},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4266":{"gid":4266,"name":"c","rank":4,"incarnation":16,"state":"up:active","state_seq":10,"addr":"172.21.15.106:6837/2632121774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":2632121774},{"type":"v1","addr":"172.21.15.106:6837","nonce":2632121774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4269":{"gid":4269,"name":"e","rank":3,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.106:6839/486339834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":486339834},{"type":"v1","addr":"172.21.15.106:6839","nonce":486339834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4284":{"gid":4284,"name":"b","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.134:6836/2781656864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6833","nonce":2781656864},{"type":"v1","addr":"172.21.15.134:6836","nonce":2781656864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4287":{"gid":4287,"name":"f","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.134:6835/1993918351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1993918351},{"type":"v1","addr":"172.21.15.134:6835","nonce":1993918351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4290":{"gid":4290,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.134:6837/2662003420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2662003420},{"type":"v1","addr":"172.21.15.134:6837","nonce":2662003420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4290,"qdb_cluster":[4266,4269,4284,4287,4290]},"id":1}]} 2024-04-25T10:31:13.515 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 17 2024-04-25T10:31:13.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.514+0000 7fd920746640 1 -- 172.21.15.106:0/972223510 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fd9040744b0 msgr2=0x7fd9040769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:13.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.514+0000 7fd920746640 1 --2- 172.21.15.106:0/972223510 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fd9040744b0 0x7fd9040769c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd908034000 tx=0x7fd90802f000 comp rx=0 tx=0).stop 2024-04-25T10:31:13.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.514+0000 7fd920746640 1 -- 172.21.15.106:0/972223510 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd918154c80 msgr2=0x7fd91814ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:13.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.514+0000 7fd920746640 1 --2- 172.21.15.106:0/972223510 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd918154c80 0x7fd91814ea70 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd91000d800 tx=0x7fd91000dcc0 comp rx=0 tx=0).stop 2024-04-25T10:31:13.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.514+0000 7fd920746640 1 -- 172.21.15.106:0/972223510 shutdown_connections 2024-04-25T10:31:13.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.514+0000 7fd920746640 1 --2- 172.21.15.106:0/972223510 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fd9040744b0 0x7fd9040769c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:13.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.514+0000 7fd920746640 1 --2- 172.21.15.106:0/972223510 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd918154c80 0x7fd91814ea70 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:13.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.514+0000 7fd920746640 1 --2- 172.21.15.106:0/972223510 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd9180cc6b0 0x7fd91814e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:13.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.514+0000 7fd920746640 1 --2- 172.21.15.106:0/972223510 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9180c6a80 0x7fd9180ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:13.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.514+0000 7fd920746640 1 -- 172.21.15.106:0/972223510 >> 172.21.15.106:0/972223510 conn(0x7fd91806d6d0 msgr2=0x7fd918074430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:13.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.514+0000 7fd920746640 1 -- 172.21.15.106:0/972223510 shutdown_connections 2024-04-25T10:31:13.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.514+0000 7fd920746640 1 -- 172.21.15.106:0/972223510 wait complete. 2024-04-25T10:31:13.528 INFO:tasks.mon_thrash.mon_thrasher:start thrashing 2024-04-25T10:31:13.528 INFO:tasks.mon_thrash.mon_thrasher:seed: 1714041071, revive delay: 20.0, thrash delay: 10.0 thrash many: False, maintain quorum: True store thrash: False, probability: 50 freeze mon: prob 10.0 duration 15.0 2024-04-25T10:31:13.529 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-04-25T10:31:13.529 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T10:31:13.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.680+0000 7f05d526e640 1 Processor -- start 2024-04-25T10:31:13.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.680+0000 7f05d526e640 1 -- start start 2024-04-25T10:31:13.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.682+0000 7f05d526e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f05d00738d0 0x7f05d006c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:13.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.682+0000 7f05d526e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05d006cf10 con 0x7f05d0154ca0 2024-04-25T10:31:13.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.682+0000 7f05d526e640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05d00ca310 con 0x7f05d015b5e0 2024-04-25T10:31:13.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.682+0000 7f05d526e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f05d006cbb0 con 0x7f05d00738d0 2024-04-25T10:31:13.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.682+0000 7f05cf7fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f05d0154ca0 0x7f05d00c8630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:34478/0 (socket says 172.21.15.106:34478) 2024-04-25T10:31:13.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.682+0000 7f05cf7fe640 1 -- 172.21.15.106:0/2885108008 learned_addr learned my addr 172.21.15.106:0/2885108008 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:31:13.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.682+0000 7f05ceffd640 1 --2- 172.21.15.106:0/2885108008 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f05d00738d0 0x7f05d006c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:13.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.682+0000 7f05cdffb640 1 -- 172.21.15.106:0/2885108008 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2155776943 0 0) 0x7f05d006cf10 con 0x7f05d0154ca0 2024-04-25T10:31:13.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.682+0000 7f05cdffb640 1 -- 172.21.15.106:0/2885108008 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f05b8003520 con 0x7f05d0154ca0 2024-04-25T10:31:13.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.682+0000 7f05cdffb640 1 -- 172.21.15.106:0/2885108008 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 759924377 0 0) 0x7f05d00ca310 con 0x7f05d015b5e0 2024-04-25T10:31:13.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.682+0000 7f05cdffb640 1 -- 172.21.15.106:0/2885108008 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f05d006cf10 con 0x7f05d015b5e0 2024-04-25T10:31:13.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.682+0000 7f05cdffb640 1 -- 172.21.15.106:0/2885108008 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3298029133 0 0) 0x7f05b8003520 con 0x7f05d0154ca0 2024-04-25T10:31:13.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.682+0000 7f05cdffb640 1 -- 172.21.15.106:0/2885108008 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f05d00738d0 msgr2=0x7f05d006c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:13.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.682+0000 7f05cdffb640 1 --2- 172.21.15.106:0/2885108008 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f05d00738d0 0x7f05d006c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:13.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.682+0000 7f05cdffb640 1 -- 172.21.15.106:0/2885108008 >> v1:172.21.15.134:6789/0 conn(0x7f05d015b5e0 legacy=0x7f05d00cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:13.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.683+0000 7f05cdffb640 1 -- 172.21.15.106:0/2885108008 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05d00ca5d0 con 0x7f05d0154ca0 2024-04-25T10:31:13.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.683+0000 7f05cdffb640 1 -- 172.21.15.106:0/2885108008 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3776459350 0 0) 0x7f05c80019f0 con 0x7f05d0154ca0 2024-04-25T10:31:13.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.683+0000 7f05ceffd640 1 --2- 172.21.15.106:0/2885108008 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f05d00738d0 0x7f05d006c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:31:13.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.683+0000 7f05cdffb640 1 -- 172.21.15.106:0/2885108008 >> v1:172.21.15.106:6789/0 conn(0x7f05d0154ca0 legacy=0x7f05d00c8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:13.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.683+0000 7f05cdffb640 1 --2- 172.21.15.106:0/2885108008 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05b8003b70 0x7f05b8003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:13.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.683+0000 7f05cdffb640 1 --2- 172.21.15.106:0/2885108008 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f05d00738d0 0x7f05b8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:13.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.683+0000 7f05cdffb640 1 --2- 172.21.15.106:0/2885108008 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05b8004d40 0x7f05b8005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:13.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.683+0000 7f05cdffb640 1 -- 172.21.15.106:0/2885108008 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f05d006cbb0 con 0x7f05b8003b70 2024-04-25T10:31:13.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.683+0000 7f05cdffb640 1 -- 172.21.15.106:0/2885108008 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f05b800baa0 con 0x7f05b8004d40 2024-04-25T10:31:13.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.683+0000 7f05cdffb640 1 -- 172.21.15.106:0/2885108008 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f05b800bca0 con 0x7f05d00738d0 2024-04-25T10:31:13.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.683+0000 7f05ceffd640 1 --2- 172.21.15.106:0/2885108008 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05b8003b70 0x7f05b8003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:13.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.683+0000 7f05ce7fc640 1 --2- 172.21.15.106:0/2885108008 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f05d00738d0 0x7f05b8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:13.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.683+0000 7f05cf7fe640 1 --2- 172.21.15.106:0/2885108008 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05b8004d40 0x7f05b8005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:13.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.683+0000 7f05ceffd640 1 -- 172.21.15.106:0/2885108008 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f05d00738d0 msgr2=0x7f05b8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:13.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.683+0000 7f05ceffd640 1 --2- 172.21.15.106:0/2885108008 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f05d00738d0 0x7f05b8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:13.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.683+0000 7f05ceffd640 1 -- 172.21.15.106:0/2885108008 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05b8004d40 msgr2=0x7f05b8005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:13.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.683+0000 7f05ceffd640 1 --2- 172.21.15.106:0/2885108008 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05b8004d40 0x7f05b8005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:13.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.683+0000 7f05ceffd640 1 -- 172.21.15.106:0/2885108008 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05b8003520 con 0x7f05b8003b70 2024-04-25T10:31:13.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.684+0000 7f05ceffd640 1 --2- 172.21.15.106:0/2885108008 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05b8003b70 0x7f05b8003fe0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f05c0009c90 tx=0x7f05b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7621e2297cb165b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:13.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.684+0000 7f05cdffb640 1 -- 172.21.15.106:0/2885108008 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c0036070 con 0x7f05b8003b70 2024-04-25T10:31:13.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.684+0000 7f05cdffb640 1 -- 172.21.15.106:0/2885108008 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05c0031400 con 0x7f05b8003b70 2024-04-25T10:31:13.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.684+0000 7f05cdffb640 1 -- 172.21.15.106:0/2885108008 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c0009860 con 0x7f05b8003b70 2024-04-25T10:31:13.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.685+0000 7f05d526e640 1 -- 172.21.15.106:0/2885108008 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05b8003b70 msgr2=0x7f05b8003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:13.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.685+0000 7f05d526e640 1 --2- 172.21.15.106:0/2885108008 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05b8003b70 0x7f05b8003fe0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f05c0009c90 tx=0x7f05b8005830 comp rx=0 tx=0).stop 2024-04-25T10:31:13.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.685+0000 7f05cf7fe640 1 -- 172.21.15.106:0/2885108008 reap_dead start 2024-04-25T10:31:13.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.685+0000 7f05d526e640 1 -- 172.21.15.106:0/2885108008 shutdown_connections 2024-04-25T10:31:13.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.685+0000 7f05d526e640 1 -- 172.21.15.106:0/2885108008 >> 172.21.15.106:0/2885108008 conn(0x7f05d006d6d0 msgr2=0x7f05d006dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:13.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.685+0000 7f05d526e640 1 -- 172.21.15.106:0/2885108008 shutdown_connections 2024-04-25T10:31:13.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.685+0000 7f05d526e640 1 -- 172.21.15.106:0/2885108008 wait complete. 2024-04-25T10:31:13.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.687+0000 7f05d526e640 1 Processor -- start 2024-04-25T10:31:13.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.687+0000 7f05d526e640 1 -- start start 2024-04-25T10:31:13.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.688+0000 7f05d526e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f05b8003b70 0x7f05d0147e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:13.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.688+0000 7f05d526e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05d0149c00 0x7f05d00ce0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:13.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.688+0000 7f05d526e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05d0157410 0x7f05d00ce5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:13.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.688+0000 7f05d526e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f05d00ca8b0 con 0x7f05d0157410 2024-04-25T10:31:13.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.688+0000 7f05d526e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f05d00cba80 con 0x7f05d0149c00 2024-04-25T10:31:13.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.688+0000 7f05d526e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f05d015b9f0 con 0x7f05b8003b70 2024-04-25T10:31:13.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.688+0000 7f05cf7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05d0157410 0x7f05d00ce5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:13.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.688+0000 7f05cf7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05d0157410 0x7f05d00ce5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35588/0 (socket says 172.21.15.106:35588) 2024-04-25T10:31:13.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.688+0000 7f05cf7fe640 1 -- 172.21.15.106:0/1996921885 learned_addr learned my addr 172.21.15.106:0/1996921885 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:31:13.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.688+0000 7f05ceffd640 1 --2- 172.21.15.106:0/1996921885 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f05b8003b70 0x7f05d0147e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:13.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.688+0000 7f05ce7fc640 1 --2- 172.21.15.106:0/1996921885 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05d0149c00 0x7f05d00ce0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:13.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.688+0000 7f05cf7fe640 1 -- 172.21.15.106:0/1996921885 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f05b8003b70 msgr2=0x7f05d0147e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:13.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.688+0000 7f05cf7fe640 1 --2- 172.21.15.106:0/1996921885 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f05b8003b70 0x7f05d0147e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:13.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.688+0000 7f05cf7fe640 1 -- 172.21.15.106:0/1996921885 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05d0149c00 msgr2=0x7f05d00ce0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:13.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.688+0000 7f05cf7fe640 1 --2- 172.21.15.106:0/1996921885 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05d0149c00 0x7f05d00ce0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:13.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.688+0000 7f05cf7fe640 1 -- 172.21.15.106:0/1996921885 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05d0148c30 con 0x7f05d0157410 2024-04-25T10:31:13.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.688+0000 7f05ceffd640 1 --2- 172.21.15.106:0/1996921885 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f05b8003b70 0x7f05d0147e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:31:13.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.689+0000 7f05cf7fe640 1 --2- 172.21.15.106:0/1996921885 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05d0157410 0x7f05d00ce5f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f05c8002230 tx=0x7f05c800a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:13.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.689+0000 7f05b77fe640 1 -- 172.21.15.106:0/1996921885 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c800aca0 con 0x7f05d0157410 2024-04-25T10:31:13.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.689+0000 7f05b77fe640 1 -- 172.21.15.106:0/1996921885 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05c8018070 con 0x7f05d0157410 2024-04-25T10:31:13.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.689+0000 7f05d526e640 1 -- 172.21.15.106:0/1996921885 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05d014aaa0 con 0x7f05d0157410 2024-04-25T10:31:13.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.689+0000 7f05b77fe640 1 -- 172.21.15.106:0/1996921885 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c800ae60 con 0x7f05d0157410 2024-04-25T10:31:13.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.689+0000 7f05d526e640 1 -- 172.21.15.106:0/1996921885 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05d0149400 con 0x7f05d0157410 2024-04-25T10:31:13.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.690+0000 7f05d526e640 1 -- 172.21.15.106:0/1996921885 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05d0070fb0 con 0x7f05d0157410 2024-04-25T10:31:13.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.691+0000 7f05b77fe640 1 -- 172.21.15.106:0/1996921885 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f05c8013630 con 0x7f05d0157410 2024-04-25T10:31:13.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.691+0000 7f05b77fe640 1 --2- 172.21.15.106:0/1996921885 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f05bc0744e0 0x7f05bc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:13.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.691+0000 7f05b77fe640 1 -- 172.21.15.106:0/1996921885 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f05c8095840 con 0x7f05d0157410 2024-04-25T10:31:13.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.691+0000 7f05ceffd640 1 --2- 172.21.15.106:0/1996921885 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f05bc0744e0 0x7f05bc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:13.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.692+0000 7f05ceffd640 1 --2- 172.21.15.106:0/1996921885 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f05bc0744e0 0x7f05bc0769f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f05c0033000 tx=0x7f05c0008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:13.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.696+0000 7f05b77fe640 1 -- 172.21.15.106:0/1996921885 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f05d0070fb0 con 0x7f05d0157410 2024-04-25T10:31:13.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.942+0000 7f05d526e640 1 -- 172.21.15.106:0/1996921885 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f05d00c9cc0 con 0x7f05d0157410 2024-04-25T10:31:13.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.942+0000 7f05b77fe640 1 -- 172.21.15.106:0/1996921885 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1568 (secure 0 0 0) 0x7f05d00c9cc0 con 0x7f05d0157410 2024-04-25T10:31:13.945 INFO:teuthology.orchestra.run.smithi106.stdout:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":65,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"3ee77985-e8e4-41cf-81e9-7c774cf7891f","modified":"2024-04-25T10:29:33.446228Z","created":"2024-04-25T10:29:33.446228Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:3300","nonce":0},{"type":"v1","addr":"172.21.15.106:6789","nonce":0}]},"addr":"172.21.15.106:6789/0","public_addr":"172.21.15.106:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T10:31:13.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.944+0000 7f05d526e640 1 -- 172.21.15.106:0/1996921885 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f05bc0744e0 msgr2=0x7f05bc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:13.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.944+0000 7f05d526e640 1 --2- 172.21.15.106:0/1996921885 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f05bc0744e0 0x7f05bc0769f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f05c0033000 tx=0x7f05c0008030 comp rx=0 tx=0).stop 2024-04-25T10:31:13.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.944+0000 7f05d526e640 1 -- 172.21.15.106:0/1996921885 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05d0157410 msgr2=0x7f05d00ce5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:13.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.944+0000 7f05d526e640 1 --2- 172.21.15.106:0/1996921885 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05d0157410 0x7f05d00ce5f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f05c8002230 tx=0x7f05c800a6a0 comp rx=0 tx=0).stop 2024-04-25T10:31:13.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.944+0000 7f05d526e640 1 -- 172.21.15.106:0/1996921885 shutdown_connections 2024-04-25T10:31:13.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.944+0000 7f05d526e640 1 --2- 172.21.15.106:0/1996921885 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f05bc0744e0 0x7f05bc0769f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:13.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.944+0000 7f05d526e640 1 --2- 172.21.15.106:0/1996921885 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05d0157410 0x7f05d00ce5f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:13.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.944+0000 7f05d526e640 1 --2- 172.21.15.106:0/1996921885 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05d0149c00 0x7f05d00ce0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:13.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.944+0000 7f05d526e640 1 --2- 172.21.15.106:0/1996921885 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f05b8003b70 0x7f05d0147e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:13.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.944+0000 7f05d526e640 1 -- 172.21.15.106:0/1996921885 >> 172.21.15.106:0/1996921885 conn(0x7f05d006d6d0 msgr2=0x7f05d0158710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:13.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.944+0000 7f05d526e640 1 -- 172.21.15.106:0/1996921885 shutdown_connections 2024-04-25T10:31:13.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:13.944+0000 7f05d526e640 1 -- 172.21.15.106:0/1996921885 wait complete. 2024-04-25T10:31:13.958 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-04-25T10:31:13.958 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-04-25T10:31:13.958 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-04-25T10:31:13.958 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-25T10:31:14.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.064+0000 7fbe6184c640 1 Processor -- start 2024-04-25T10:31:14.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.064+0000 7fbe6184c640 1 -- start start 2024-04-25T10:31:14.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.065+0000 7fbe6184c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe5c06dd50 0x7fbe5c0c6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:14.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.065+0000 7fbe6184c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe5c15d750 con 0x7fbe5c1570d0 2024-04-25T10:31:14.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.065+0000 7fbe6184c640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe5c15da10 con 0x7fbe5c06f390 2024-04-25T10:31:14.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.065+0000 7fbe6184c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fbe5c15dcf0 con 0x7fbe5c06dd50 2024-04-25T10:31:14.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.065+0000 7fbe53fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fbe5c1570d0 0x7fbe5c0c6250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:34488/0 (socket says 172.21.15.106:34488) 2024-04-25T10:31:14.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.066+0000 7fbe53fff640 1 -- 172.21.15.106:0/3538975758 learned_addr learned my addr 172.21.15.106:0/3538975758 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:31:14.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.066+0000 7fbe5b7fe640 1 --2- 172.21.15.106:0/3538975758 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe5c06dd50 0x7fbe5c0c6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:14.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.066+0000 7fbe5a7fc640 1 -- 172.21.15.106:0/3538975758 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2367146065 0 0) 0x7fbe5c15d750 con 0x7fbe5c1570d0 2024-04-25T10:31:14.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.066+0000 7fbe5a7fc640 1 -- 172.21.15.106:0/3538975758 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbe48003520 con 0x7fbe5c1570d0 2024-04-25T10:31:14.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.066+0000 7fbe5a7fc640 1 -- 172.21.15.106:0/3538975758 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2201916342 0 0) 0x7fbe5c15da10 con 0x7fbe5c06f390 2024-04-25T10:31:14.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.066+0000 7fbe5a7fc640 1 -- 172.21.15.106:0/3538975758 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbe5c15d750 con 0x7fbe5c06f390 2024-04-25T10:31:14.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.066+0000 7fbe5a7fc640 1 -- 172.21.15.106:0/3538975758 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 401833184 0 0) 0x7fbe48003520 con 0x7fbe5c1570d0 2024-04-25T10:31:14.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.066+0000 7fbe5b7fe640 1 -- 172.21.15.106:0/3538975758 >> v1:172.21.15.134:6789/0 conn(0x7fbe5c06f390 legacy=0x7fbe5c0ca960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:14.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.066+0000 7fbe5b7fe640 1 -- 172.21.15.106:0/3538975758 >> v1:172.21.15.106:6789/0 conn(0x7fbe5c1570d0 legacy=0x7fbe5c0c6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:14.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.066+0000 7fbe5b7fe640 1 -- 172.21.15.106:0/3538975758 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe5c06e5d0 con 0x7fbe5c06dd50 2024-04-25T10:31:14.070 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.067+0000 7fbe5b7fe640 1 --2- 172.21.15.106:0/3538975758 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe5c06dd50 0x7fbe5c0c6960 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbe54009dc0 tx=0x7fbe5402a6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95dd1c89f7874328 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:14.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.067+0000 7fbe5a7fc640 1 -- 172.21.15.106:0/3538975758 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe44001670 con 0x7fbe5c06dd50 2024-04-25T10:31:14.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.067+0000 7fbe5a7fc640 1 -- 172.21.15.106:0/3538975758 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe5402ae50 con 0x7fbe5c06dd50 2024-04-25T10:31:14.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.067+0000 7fbe5a7fc640 1 -- 172.21.15.106:0/3538975758 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe54007740 con 0x7fbe5c06dd50 2024-04-25T10:31:14.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.068+0000 7fbe6184c640 1 -- 172.21.15.106:0/3538975758 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe5c06dd50 msgr2=0x7fbe5c0c6960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:14.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.068+0000 7fbe6184c640 1 --2- 172.21.15.106:0/3538975758 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe5c06dd50 0x7fbe5c0c6960 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbe54009dc0 tx=0x7fbe5402a6f0 comp rx=0 tx=0).stop 2024-04-25T10:31:14.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.068+0000 7fbe6184c640 1 -- 172.21.15.106:0/3538975758 shutdown_connections 2024-04-25T10:31:14.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.068+0000 7fbe6184c640 1 --2- 172.21.15.106:0/3538975758 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe5c06dd50 0x7fbe5c0c6960 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:14.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.068+0000 7fbe6184c640 1 -- 172.21.15.106:0/3538975758 >> 172.21.15.106:0/3538975758 conn(0x7fbe5c0cff60 msgr2=0x7fbe5c0d0370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:14.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.068+0000 7fbe6184c640 1 -- 172.21.15.106:0/3538975758 shutdown_connections 2024-04-25T10:31:14.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.068+0000 7fbe6184c640 1 -- 172.21.15.106:0/3538975758 wait complete. 2024-04-25T10:31:14.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.069+0000 7fbe6184c640 1 Processor -- start 2024-04-25T10:31:14.072 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.069+0000 7fbe6184c640 1 -- start start 2024-04-25T10:31:14.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.071+0000 7fbe6184c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe5c06dd50 0x7fbe5c149bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:14.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.071+0000 7fbe6184c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe5c06f390 0x7fbe5c1539b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:14.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.071+0000 7fbe6184c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe5c1570d0 0x7fbe5c153ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:14.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.071+0000 7fbe6184c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbe5c06eb90 con 0x7fbe5c1570d0 2024-04-25T10:31:14.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.071+0000 7fbe6184c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbe5c06ef00 con 0x7fbe5c06f390 2024-04-25T10:31:14.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.071+0000 7fbe6184c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fbe5c06e160 con 0x7fbe5c06dd50 2024-04-25T10:31:14.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.071+0000 7fbe5b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe5c1570d0 0x7fbe5c153ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:14.074 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.071+0000 7fbe5b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe5c1570d0 0x7fbe5c153ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35604/0 (socket says 172.21.15.106:35604) 2024-04-25T10:31:14.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.071+0000 7fbe5b7fe640 1 -- 172.21.15.106:0/1282447119 learned_addr learned my addr 172.21.15.106:0/1282447119 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:31:14.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.071+0000 7fbe53fff640 1 --2- 172.21.15.106:0/1282447119 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe5c06f390 0x7fbe5c1539b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:14.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.071+0000 7fbe5affd640 1 --2- 172.21.15.106:0/1282447119 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe5c06dd50 0x7fbe5c149bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:14.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.071+0000 7fbe5b7fe640 1 -- 172.21.15.106:0/1282447119 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe5c06dd50 msgr2=0x7fbe5c149bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:14.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.071+0000 7fbe5b7fe640 1 --2- 172.21.15.106:0/1282447119 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe5c06dd50 0x7fbe5c149bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:14.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.071+0000 7fbe5b7fe640 1 -- 172.21.15.106:0/1282447119 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe5c06f390 msgr2=0x7fbe5c1539b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:14.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.071+0000 7fbe5b7fe640 1 --2- 172.21.15.106:0/1282447119 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe5c06f390 0x7fbe5c1539b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:14.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.071+0000 7fbe5b7fe640 1 -- 172.21.15.106:0/1282447119 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe5c152ec0 con 0x7fbe5c1570d0 2024-04-25T10:31:14.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.071+0000 7fbe5b7fe640 1 --2- 172.21.15.106:0/1282447119 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe5c1570d0 0x7fbe5c153ef0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fbe54009ee0 tx=0x7fbe54004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:14.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.071+0000 7fbe58ff9640 1 -- 172.21.15.106:0/1282447119 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe5403f070 con 0x7fbe5c1570d0 2024-04-25T10:31:14.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.071+0000 7fbe58ff9640 1 -- 172.21.15.106:0/1282447119 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe5403b820 con 0x7fbe5c1570d0 2024-04-25T10:31:14.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.071+0000 7fbe6184c640 1 -- 172.21.15.106:0/1282447119 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe5c15c7e0 con 0x7fbe5c1570d0 2024-04-25T10:31:14.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.071+0000 7fbe58ff9640 1 -- 172.21.15.106:0/1282447119 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe5403bbf0 con 0x7fbe5c1570d0 2024-04-25T10:31:14.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.071+0000 7fbe6184c640 1 -- 172.21.15.106:0/1282447119 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe5c1526c0 con 0x7fbe5c1570d0 2024-04-25T10:31:14.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.072+0000 7fbe6184c640 1 --2- 172.21.15.106:0/1282447119 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe5c1575c0 0x7fbe5c15a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:14.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.072+0000 7fbe6184c640 1 -- 172.21.15.106:0/1282447119 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbe5c15c570 con 0x7fbe5c1575c0 2024-04-25T10:31:14.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.073+0000 7fbe5affd640 1 --2- 172.21.15.106:0/1282447119 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe5c1575c0 0x7fbe5c15a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:14.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.073+0000 7fbe5affd640 1 -- 172.21.15.106:0/1282447119 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbe5c06a3d0 con 0x7fbe5c1575c0 2024-04-25T10:31:14.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.073+0000 7fbe5affd640 1 --2- 172.21.15.106:0/1282447119 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe5c1575c0 0x7fbe5c15a320 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c00e390 tx=0x7fbe4c00e850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:14.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.073+0000 7fbe58ff9640 1 -- 172.21.15.106:0/1282447119 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbe54031400 con 0x7fbe5c1570d0 2024-04-25T10:31:14.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.073+0000 7fbe58ff9640 1 --2- 172.21.15.106:0/1282447119 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fbe3c0744b0 0x7fbe3c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:14.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.073+0000 7fbe53fff640 1 --2- 172.21.15.106:0/1282447119 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fbe3c0744b0 0x7fbe3c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:14.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.074+0000 7fbe58ff9640 1 -- 172.21.15.106:0/1282447119 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fbe540b6400 con 0x7fbe5c1570d0 2024-04-25T10:31:14.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.074+0000 7fbe58ff9640 1 -- 172.21.15.106:0/1282447119 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c019070 con 0x7fbe5c1575c0 2024-04-25T10:31:14.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.074+0000 7fbe58ff9640 1 -- 172.21.15.106:0/1282447119 <== mon.0 v2:172.21.15.106:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fbe4c0052e0 con 0x7fbe5c1575c0 2024-04-25T10:31:14.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.074+0000 7fbe58ff9640 1 -- 172.21.15.106:0/1282447119 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe5c1575c0 msgr2=0x7fbe5c15a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:14.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.074+0000 7fbe58ff9640 1 --2- 172.21.15.106:0/1282447119 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe5c1575c0 0x7fbe5c15a320 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c00e390 tx=0x7fbe4c00e850 comp rx=0 tx=0).stop 2024-04-25T10:31:14.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.074+0000 7fbe58ff9640 1 -- 172.21.15.106:0/1282447119 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe5c1575c0 msgr2=0x7fbe5c15a320 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T10:31:14.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.074+0000 7fbe58ff9640 1 --2- 172.21.15.106:0/1282447119 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe5c1575c0 0x7fbe5c15a320 secure :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c00e390 tx=0x7fbe4c00e850 comp rx=0 tx=0).stop 2024-04-25T10:31:14.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.074+0000 7fbe53fff640 1 --2- 172.21.15.106:0/1282447119 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fbe3c0744b0 0x7fbe3c0769c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c156ac0 tx=0x7fbe44009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:14.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.080+0000 7fbe6184c640 1 --2- 172.21.15.106:0/1282447119 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe5c1f64f0 0x7fbe5c1f6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:14.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.080+0000 7fbe6184c640 1 -- 172.21.15.106:0/1282447119 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbe5c0ca430 con 0x7fbe5c1f64f0 2024-04-25T10:31:14.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.080+0000 7fbe5affd640 1 --2- 172.21.15.106:0/1282447119 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe5c1f64f0 0x7fbe5c1f6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:14.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.081+0000 7fbe5affd640 1 -- 172.21.15.106:0/1282447119 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fbe5c0cbe80 con 0x7fbe5c1f64f0 2024-04-25T10:31:14.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.081+0000 7fbe5affd640 1 --2- 172.21.15.106:0/1282447119 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe5c1f64f0 0x7fbe5c1f6900 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c00e390 tx=0x7fbe4c000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:14.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.081+0000 7fbe58ff9640 1 -- 172.21.15.106:0/1282447119 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c019030 con 0x7fbe5c1f64f0 2024-04-25T10:31:14.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.081+0000 7fbe58ff9640 1 -- 172.21.15.106:0/1282447119 <== mon.0 v2:172.21.15.106:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5070 (secure 0 0 0) 0x7fbe4c01c2c0 con 0x7fbe5c1f64f0 2024-04-25T10:31:14.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.081+0000 7fbe58ff9640 1 -- 172.21.15.106:0/1282447119 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe5c1f64f0 msgr2=0x7fbe5c1f6900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:14.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.081+0000 7fbe58ff9640 1 --2- 172.21.15.106:0/1282447119 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe5c1f64f0 0x7fbe5c1f6900 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c00e390 tx=0x7fbe4c000ed0 comp rx=0 tx=0).stop 2024-04-25T10:31:14.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.081+0000 7fbe58ff9640 1 -- 172.21.15.106:0/1282447119 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe5c1f64f0 msgr2=0x7fbe5c1f6900 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T10:31:14.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.081+0000 7fbe58ff9640 1 --2- 172.21.15.106:0/1282447119 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe5c1f64f0 0x7fbe5c1f6900 secure :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c00e390 tx=0x7fbe4c000ed0 comp rx=0 tx=0).stop 2024-04-25T10:31:14.084 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-25T10:31:14.084 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "a", 2024-04-25T10:31:14.084 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 0, 2024-04-25T10:31:14.084 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "leader", 2024-04-25T10:31:14.085 INFO:teuthology.orchestra.run.smithi106.stdout: "election_epoch": 4, 2024-04-25T10:31:14.085 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 66320, 2024-04-25T10:31:14.085 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum": [ 2024-04-25T10:31:14.085 INFO:teuthology.orchestra.run.smithi106.stdout: 0, 2024-04-25T10:31:14.085 INFO:teuthology.orchestra.run.smithi106.stdout: 1, 2024-04-25T10:31:14.085 INFO:teuthology.orchestra.run.smithi106.stdout: 2 2024-04-25T10:31:14.085 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:31:14.085 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_age": 66, 2024-04-25T10:31:14.085 INFO:teuthology.orchestra.run.smithi106.stdout: "features": { 2024-04-25T10:31:14.085 INFO:teuthology.orchestra.run.smithi106.stdout: "required_con": "2450521708007866372", 2024-04-25T10:31:14.085 INFO:teuthology.orchestra.run.smithi106.stdout: "required_mon": [ 2024-04-25T10:31:14.085 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:31:14.085 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:31:14.086 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:31:14.086 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:31:14.086 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:31:14.086 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:31:14.086 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:31:14.086 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:31:14.086 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:31:14.086 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:31:14.086 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:31:14.086 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:31:14.086 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_con": "4540701547738038271", 2024-04-25T10:31:14.086 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_mon": [ 2024-04-25T10:31:14.086 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:31:14.086 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:31:14.086 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:31:14.087 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:31:14.087 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:31:14.087 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:31:14.087 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:31:14.087 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:31:14.087 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:31:14.087 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:31:14.087 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:31:14.087 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:31:14.087 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:14.087 INFO:teuthology.orchestra.run.smithi106.stdout: "outside_quorum": [], 2024-04-25T10:31:14.087 INFO:teuthology.orchestra.run.smithi106.stdout: "extra_probe_peers": [], 2024-04-25T10:31:14.087 INFO:teuthology.orchestra.run.smithi106.stdout: "sync_provider": [], 2024-04-25T10:31:14.087 INFO:teuthology.orchestra.run.smithi106.stdout: "monmap": { 2024-04-25T10:31:14.088 INFO:teuthology.orchestra.run.smithi106.stdout: "epoch": 1, 2024-04-25T10:31:14.088 INFO:teuthology.orchestra.run.smithi106.stdout: "fsid": "3ee77985-e8e4-41cf-81e9-7c774cf7891f", 2024-04-25T10:31:14.088 INFO:teuthology.orchestra.run.smithi106.stdout: "modified": "2024-04-25T10:29:33.446228Z", 2024-04-25T10:31:14.088 INFO:teuthology.orchestra.run.smithi106.stdout: "created": "2024-04-25T10:29:33.446228Z", 2024-04-25T10:31:14.088 INFO:teuthology.orchestra.run.smithi106.stdout: "min_mon_release": 19, 2024-04-25T10:31:14.088 INFO:teuthology.orchestra.run.smithi106.stdout: "min_mon_release_name": "squid", 2024-04-25T10:31:14.088 INFO:teuthology.orchestra.run.smithi106.stdout: "election_strategy": 1, 2024-04-25T10:31:14.088 INFO:teuthology.orchestra.run.smithi106.stdout: "disallowed_leaders: ": "", 2024-04-25T10:31:14.088 INFO:teuthology.orchestra.run.smithi106.stdout: "stretch_mode": false, 2024-04-25T10:31:14.088 INFO:teuthology.orchestra.run.smithi106.stdout: "tiebreaker_mon": "", 2024-04-25T10:31:14.088 INFO:teuthology.orchestra.run.smithi106.stdout: "removed_ranks: ": "", 2024-04-25T10:31:14.088 INFO:teuthology.orchestra.run.smithi106.stdout: "features": { 2024-04-25T10:31:14.088 INFO:teuthology.orchestra.run.smithi106.stdout: "persistent": [ 2024-04-25T10:31:14.088 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:31:14.088 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:31:14.088 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:31:14.089 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:31:14.089 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:31:14.089 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:31:14.089 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:31:14.089 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:31:14.089 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:31:14.089 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:31:14.089 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:31:14.089 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:31:14.089 INFO:teuthology.orchestra.run.smithi106.stdout: "optional": [] 2024-04-25T10:31:14.089 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:14.089 INFO:teuthology.orchestra.run.smithi106.stdout: "mons": [ 2024-04-25T10:31:14.089 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:14.089 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 0, 2024-04-25T10:31:14.089 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "a", 2024-04-25T10:31:14.090 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:31:14.090 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:31:14.090 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:14.090 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:31:14.090 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:3300", 2024-04-25T10:31:14.090 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:31:14.090 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:14.090 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:14.090 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:31:14.090 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:6789", 2024-04-25T10:31:14.090 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:31:14.090 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:31:14.090 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:31:14.090 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:14.091 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:6789/0", 2024-04-25T10:31:14.091 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.106:6789/0", 2024-04-25T10:31:14.091 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:31:14.091 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:31:14.091 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:31:14.091 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:14.091 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:14.091 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 1, 2024-04-25T10:31:14.091 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "b", 2024-04-25T10:31:14.091 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:31:14.091 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:31:14.091 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:14.091 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:31:14.091 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:3300", 2024-04-25T10:31:14.091 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:31:14.092 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:14.092 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:14.092 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:31:14.092 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6789", 2024-04-25T10:31:14.092 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:31:14.092 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:31:14.092 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:31:14.092 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:14.092 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6789/0", 2024-04-25T10:31:14.092 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.134:6789/0", 2024-04-25T10:31:14.092 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:31:14.092 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:31:14.092 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:31:14.092 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:14.092 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:14.093 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 2, 2024-04-25T10:31:14.093 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "c", 2024-04-25T10:31:14.093 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:31:14.093 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:31:14.093 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:14.093 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:31:14.093 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:3301", 2024-04-25T10:31:14.093 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:31:14.093 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:14.093 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:14.093 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:31:14.093 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6790", 2024-04-25T10:31:14.093 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:31:14.093 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:31:14.094 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:31:14.094 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:14.094 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6790/0", 2024-04-25T10:31:14.094 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.134:6790/0", 2024-04-25T10:31:14.094 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:31:14.094 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:31:14.094 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:31:14.094 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:31:14.094 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:31:14.094 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:14.094 INFO:teuthology.orchestra.run.smithi106.stdout: "feature_map": { 2024-04-25T10:31:14.094 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": [ 2024-04-25T10:31:14.094 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:14.094 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:31:14.095 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:31:14.095 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 1 2024-04-25T10:31:14.095 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:31:14.095 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:31:14.095 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": [ 2024-04-25T10:31:14.095 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:14.095 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:31:14.095 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:31:14.095 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 6 2024-04-25T10:31:14.095 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:31:14.095 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:31:14.095 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": [ 2024-04-25T10:31:14.095 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:14.095 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:31:14.095 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:31:14.096 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 1 2024-04-25T10:31:14.096 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:31:14.096 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:31:14.096 INFO:teuthology.orchestra.run.smithi106.stdout: "client": [ 2024-04-25T10:31:14.096 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:14.096 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:31:14.096 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:31:14.096 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 7 2024-04-25T10:31:14.096 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:31:14.096 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:31:14.096 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": [ 2024-04-25T10:31:14.096 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:14.096 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:31:14.096 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:31:14.096 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 1 2024-04-25T10:31:14.097 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:31:14.097 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:31:14.097 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:14.097 INFO:teuthology.orchestra.run.smithi106.stdout: "stretch_mode": false 2024-04-25T10:31:14.097 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-25T10:31:14.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.081+0000 7fbe6184c640 1 -- 172.21.15.106:0/1282447119 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fbe3c0744b0 msgr2=0x7fbe3c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:14.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.081+0000 7fbe6184c640 1 --2- 172.21.15.106:0/1282447119 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fbe3c0744b0 0x7fbe3c0769c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c156ac0 tx=0x7fbe44009040 comp rx=0 tx=0).stop 2024-04-25T10:31:14.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.081+0000 7fbe6184c640 1 -- 172.21.15.106:0/1282447119 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe5c1570d0 msgr2=0x7fbe5c153ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:14.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.081+0000 7fbe6184c640 1 --2- 172.21.15.106:0/1282447119 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe5c1570d0 0x7fbe5c153ef0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fbe54009ee0 tx=0x7fbe54004a40 comp rx=0 tx=0).stop 2024-04-25T10:31:14.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.081+0000 7fbe5b7fe640 1 -- 172.21.15.106:0/1282447119 reap_dead start 2024-04-25T10:31:14.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.081+0000 7fbe5b7fe640 1 -- 172.21.15.106:0/1282447119 reap_dead start 2024-04-25T10:31:14.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.081+0000 7fbe6184c640 1 -- 172.21.15.106:0/1282447119 shutdown_connections 2024-04-25T10:31:14.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.081+0000 7fbe6184c640 1 -- 172.21.15.106:0/1282447119 >> 172.21.15.106:0/1282447119 conn(0x7fbe5c0cff60 msgr2=0x7fbe5c0cd260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:14.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.082+0000 7fbe6184c640 1 -- 172.21.15.106:0/1282447119 shutdown_connections 2024-04-25T10:31:14.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.082+0000 7fbe6184c640 1 -- 172.21.15.106:0/1282447119 wait complete. 2024-04-25T10:31:14.099 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T10:31:14.099 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-04-25T10:31:14.099 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-25T10:31:14.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.206+0000 7fd070934640 1 Processor -- start 2024-04-25T10:31:14.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.206+0000 7fd070934640 1 -- start start 2024-04-25T10:31:14.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.207+0000 7fd070934640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd06c15d7c0 0x7fd06c0c61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:14.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.207+0000 7fd070934640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd06c1572d0 con 0x7fd06c06dc00 2024-04-25T10:31:14.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.207+0000 7fd070934640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd06c160ae0 con 0x7fd06c156ec0 2024-04-25T10:31:14.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.207+0000 7fd070934640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fd06c15dbd0 con 0x7fd06c15d7c0 2024-04-25T10:31:14.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.207+0000 7fd06a59c640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fd06c06dc00 0x7fd06c1601c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:34500/0 (socket says 172.21.15.106:34500) 2024-04-25T10:31:14.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.207+0000 7fd06a59c640 1 -- 172.21.15.106:0/139446838 learned_addr learned my addr 172.21.15.106:0/139446838 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:31:14.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.207+0000 7fd063fff640 1 --2- 172.21.15.106:0/139446838 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd06c15d7c0 0x7fd06c0c61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:14.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.207+0000 7fd069d9b640 1 -- 172.21.15.106:0/139446838 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2443556341 0 0) 0x7fd06c1572d0 con 0x7fd06c06dc00 2024-04-25T10:31:14.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.207+0000 7fd069d9b640 1 -- 172.21.15.106:0/139446838 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd058003520 con 0x7fd06c06dc00 2024-04-25T10:31:14.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.208+0000 7fd069d9b640 1 -- 172.21.15.106:0/139446838 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1793245840 0 0) 0x7fd06c160ae0 con 0x7fd06c156ec0 2024-04-25T10:31:14.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.208+0000 7fd069d9b640 1 -- 172.21.15.106:0/139446838 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd06c1572d0 con 0x7fd06c156ec0 2024-04-25T10:31:14.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.208+0000 7fd063fff640 1 -- 172.21.15.106:0/139446838 >> v1:172.21.15.134:6789/0 conn(0x7fd06c156ec0 legacy=0x7fd06c0ccde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:14.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.208+0000 7fd063fff640 1 -- 172.21.15.106:0/139446838 >> v1:172.21.15.106:6789/0 conn(0x7fd06c06dc00 legacy=0x7fd06c1601c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:14.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.208+0000 7fd063fff640 1 -- 172.21.15.106:0/139446838 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd06c160dc0 con 0x7fd06c15d7c0 2024-04-25T10:31:14.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.208+0000 7fd063fff640 1 --2- 172.21.15.106:0/139446838 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd06c15d7c0 0x7fd06c0c61a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd0540097d0 tx=0x7fd05402a290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ed5272d5caaf31d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:14.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.209+0000 7fd069d9b640 1 -- 172.21.15.106:0/139446838 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd054037070 con 0x7fd06c15d7c0 2024-04-25T10:31:14.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.209+0000 7fd069d9b640 1 -- 172.21.15.106:0/139446838 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd054032440 con 0x7fd06c15d7c0 2024-04-25T10:31:14.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.209+0000 7fd069d9b640 1 -- 172.21.15.106:0/139446838 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0540327f0 con 0x7fd06c15d7c0 2024-04-25T10:31:14.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.209+0000 7fd070934640 1 -- 172.21.15.106:0/139446838 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd06c15d7c0 msgr2=0x7fd06c0c61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:14.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.209+0000 7fd070934640 1 --2- 172.21.15.106:0/139446838 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd06c15d7c0 0x7fd06c0c61a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd0540097d0 tx=0x7fd05402a290 comp rx=0 tx=0).stop 2024-04-25T10:31:14.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.209+0000 7fd070934640 1 -- 172.21.15.106:0/139446838 shutdown_connections 2024-04-25T10:31:14.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.209+0000 7fd070934640 1 --2- 172.21.15.106:0/139446838 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd06c15d7c0 0x7fd06c0c61a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:14.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.209+0000 7fd070934640 1 -- 172.21.15.106:0/139446838 >> 172.21.15.106:0/139446838 conn(0x7fd06c0727d0 msgr2=0x7fd06c072c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:14.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.209+0000 7fd070934640 1 -- 172.21.15.106:0/139446838 shutdown_connections 2024-04-25T10:31:14.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.209+0000 7fd070934640 1 -- 172.21.15.106:0/139446838 wait complete. 2024-04-25T10:31:14.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.211+0000 7fd070934640 1 Processor -- start 2024-04-25T10:31:14.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.211+0000 7fd070934640 1 -- start start 2024-04-25T10:31:14.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.212+0000 7fd070934640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd06c06dc00 0x7fd06c1f2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:14.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.212+0000 7fd070934640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd06c156ec0 0x7fd06c155dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:14.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.212+0000 7fd070934640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd06c15d7c0 0x7fd06c150b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:14.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.212+0000 7fd070934640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd06c15cfc0 con 0x7fd06c156ec0 2024-04-25T10:31:14.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.212+0000 7fd070934640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd06c15cce0 con 0x7fd06c06dc00 2024-04-25T10:31:14.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.212+0000 7fd070934640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fd06c15d2a0 con 0x7fd06c15d7c0 2024-04-25T10:31:14.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.212+0000 7fd063fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd06c156ec0 0x7fd06c155dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:14.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.212+0000 7fd063fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd06c156ec0 0x7fd06c155dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35622/0 (socket says 172.21.15.106:35622) 2024-04-25T10:31:14.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.212+0000 7fd063fff640 1 -- 172.21.15.106:0/3054659499 learned_addr learned my addr 172.21.15.106:0/3054659499 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:31:14.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.213+0000 7fd06a59c640 1 --2- 172.21.15.106:0/3054659499 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd06c06dc00 0x7fd06c1f2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:14.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.213+0000 7fd06ad9d640 1 --2- 172.21.15.106:0/3054659499 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd06c15d7c0 0x7fd06c150b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:14.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.213+0000 7fd063fff640 1 -- 172.21.15.106:0/3054659499 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd06c15d7c0 msgr2=0x7fd06c150b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:14.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.213+0000 7fd063fff640 1 --2- 172.21.15.106:0/3054659499 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd06c15d7c0 0x7fd06c150b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:14.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.213+0000 7fd063fff640 1 -- 172.21.15.106:0/3054659499 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd06c06dc00 msgr2=0x7fd06c1f2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:14.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.213+0000 7fd063fff640 1 --2- 172.21.15.106:0/3054659499 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd06c06dc00 0x7fd06c1f2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:14.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.213+0000 7fd063fff640 1 -- 172.21.15.106:0/3054659499 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd06c156450 con 0x7fd06c156ec0 2024-04-25T10:31:14.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.213+0000 7fd063fff640 1 --2- 172.21.15.106:0/3054659499 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd06c156ec0 0x7fd06c155dc0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd05402a2d0 tx=0x7fd0540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:14.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.213+0000 7fd062ffd640 1 -- 172.21.15.106:0/3054659499 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd054041020 con 0x7fd06c156ec0 2024-04-25T10:31:14.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.213+0000 7fd062ffd640 1 -- 172.21.15.106:0/3054659499 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd05403d6e0 con 0x7fd06c156ec0 2024-04-25T10:31:14.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.213+0000 7fd070934640 1 -- 172.21.15.106:0/3054659499 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd06c1f6f30 con 0x7fd06c156ec0 2024-04-25T10:31:14.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.213+0000 7fd062ffd640 1 -- 172.21.15.106:0/3054659499 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd054037040 con 0x7fd06c156ec0 2024-04-25T10:31:14.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.213+0000 7fd070934640 1 -- 172.21.15.106:0/3054659499 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd06c1f6170 con 0x7fd06c156ec0 2024-04-25T10:31:14.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.214+0000 7fd070934640 1 --2- 172.21.15.106:0/3054659499 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd06c1504c0 0x7fd06c1f63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:14.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.214+0000 7fd070934640 1 -- 172.21.15.106:0/3054659499 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd06c1f33b0 con 0x7fd06c1504c0 2024-04-25T10:31:14.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.214+0000 7fd06a59c640 1 --2- 172.21.15.106:0/3054659499 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd06c1504c0 0x7fd06c1f63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:14.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.215+0000 7fd062ffd640 1 -- 172.21.15.106:0/3054659499 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd054044020 con 0x7fd06c156ec0 2024-04-25T10:31:14.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.215+0000 7fd06a59c640 1 -- 172.21.15.106:0/3054659499 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd06c06a3d0 con 0x7fd06c1504c0 2024-04-25T10:31:14.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.215+0000 7fd06a59c640 1 --2- 172.21.15.106:0/3054659499 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd06c1504c0 0x7fd06c1f63b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd05c0011e0 tx=0x7fd05c001910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:14.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.215+0000 7fd062ffd640 1 --2- 172.21.15.106:0/3054659499 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fd04c0744b0 0x7fd04c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:14.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.215+0000 7fd062ffd640 1 -- 172.21.15.106:0/3054659499 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fd0540b5940 con 0x7fd06c156ec0 2024-04-25T10:31:14.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.215+0000 7fd06ad9d640 1 --2- 172.21.15.106:0/3054659499 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fd04c0744b0 0x7fd04c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:14.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.215+0000 7fd062ffd640 1 -- 172.21.15.106:0/3054659499 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05c019070 con 0x7fd06c1504c0 2024-04-25T10:31:14.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.215+0000 7fd062ffd640 1 -- 172.21.15.106:0/3054659499 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fd05c005a60 con 0x7fd06c1504c0 2024-04-25T10:31:14.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.215+0000 7fd062ffd640 1 -- 172.21.15.106:0/3054659499 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd06c1504c0 msgr2=0x7fd06c1f63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:14.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.215+0000 7fd062ffd640 1 --2- 172.21.15.106:0/3054659499 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd06c1504c0 0x7fd06c1f63b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd05c0011e0 tx=0x7fd05c001910 comp rx=0 tx=0).stop 2024-04-25T10:31:14.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.215+0000 7fd062ffd640 1 -- 172.21.15.106:0/3054659499 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd06c1504c0 msgr2=0x7fd06c1f63b0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T10:31:14.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.215+0000 7fd062ffd640 1 --2- 172.21.15.106:0/3054659499 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd06c1504c0 0x7fd06c1f63b0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:14.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.216+0000 7fd06ad9d640 1 --2- 172.21.15.106:0/3054659499 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fd04c0744b0 0x7fd04c0769c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd06c1f30a0 tx=0x7fd064009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:14.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.222+0000 7fd070934640 1 --2- 172.21.15.106:0/3054659499 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd06c151560 0x7fd06c151970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:14.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.222+0000 7fd070934640 1 -- 172.21.15.106:0/3054659499 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd06c1f6aa0 con 0x7fd06c151560 2024-04-25T10:31:14.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.222+0000 7fd06a59c640 1 --2- 172.21.15.106:0/3054659499 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd06c151560 0x7fd06c151970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:14.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.222+0000 7fd06a59c640 1 -- 172.21.15.106:0/3054659499 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fd06c0cbdf0 con 0x7fd06c151560 2024-04-25T10:31:14.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.223+0000 7fd06a59c640 1 --2- 172.21.15.106:0/3054659499 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd06c151560 0x7fd06c151970 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd05c016000 tx=0x7fd05c001150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:14.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.223+0000 7fd062ffd640 1 -- 172.21.15.106:0/3054659499 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05c019070 con 0x7fd06c151560 2024-04-25T10:31:14.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.223+0000 7fd062ffd640 1 -- 172.21.15.106:0/3054659499 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4728 (secure 0 0 0) 0x7fd05c01a040 con 0x7fd06c151560 2024-04-25T10:31:14.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.223+0000 7fd062ffd640 1 -- 172.21.15.106:0/3054659499 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd06c151560 msgr2=0x7fd06c151970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:14.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.223+0000 7fd062ffd640 1 --2- 172.21.15.106:0/3054659499 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd06c151560 0x7fd06c151970 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd05c016000 tx=0x7fd05c001150 comp rx=0 tx=0).stop 2024-04-25T10:31:14.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.223+0000 7fd062ffd640 1 -- 172.21.15.106:0/3054659499 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd06c151560 msgr2=0x7fd06c151970 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T10:31:14.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.223+0000 7fd062ffd640 1 --2- 172.21.15.106:0/3054659499 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd06c151560 0x7fd06c151970 secure :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd05c016000 tx=0x7fd05c001150 comp rx=0 tx=0).stop 2024-04-25T10:31:14.226 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-25T10:31:14.226 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "b", 2024-04-25T10:31:14.226 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 1, 2024-04-25T10:31:14.226 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "peon", 2024-04-25T10:31:14.226 INFO:teuthology.orchestra.run.smithi106.stdout: "election_epoch": 4, 2024-04-25T10:31:14.226 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 66419, 2024-04-25T10:31:14.226 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum": [ 2024-04-25T10:31:14.226 INFO:teuthology.orchestra.run.smithi106.stdout: 0, 2024-04-25T10:31:14.227 INFO:teuthology.orchestra.run.smithi106.stdout: 1, 2024-04-25T10:31:14.227 INFO:teuthology.orchestra.run.smithi106.stdout: 2 2024-04-25T10:31:14.227 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:31:14.227 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_age": 66, 2024-04-25T10:31:14.227 INFO:teuthology.orchestra.run.smithi106.stdout: "features": { 2024-04-25T10:31:14.227 INFO:teuthology.orchestra.run.smithi106.stdout: "required_con": "2450521708007866372", 2024-04-25T10:31:14.227 INFO:teuthology.orchestra.run.smithi106.stdout: "required_mon": [ 2024-04-25T10:31:14.227 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:31:14.227 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:31:14.227 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:31:14.227 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:31:14.227 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:31:14.227 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:31:14.227 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:31:14.227 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:31:14.227 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:31:14.228 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:31:14.228 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:31:14.228 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:31:14.228 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_con": "4540701547738038271", 2024-04-25T10:31:14.228 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_mon": [ 2024-04-25T10:31:14.228 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:31:14.228 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:31:14.228 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:31:14.228 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:31:14.228 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:31:14.228 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:31:14.228 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:31:14.228 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:31:14.228 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:31:14.228 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:31:14.229 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:31:14.229 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:31:14.229 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:14.229 INFO:teuthology.orchestra.run.smithi106.stdout: "outside_quorum": [], 2024-04-25T10:31:14.229 INFO:teuthology.orchestra.run.smithi106.stdout: "extra_probe_peers": [], 2024-04-25T10:31:14.229 INFO:teuthology.orchestra.run.smithi106.stdout: "sync_provider": [], 2024-04-25T10:31:14.229 INFO:teuthology.orchestra.run.smithi106.stdout: "monmap": { 2024-04-25T10:31:14.229 INFO:teuthology.orchestra.run.smithi106.stdout: "epoch": 1, 2024-04-25T10:31:14.229 INFO:teuthology.orchestra.run.smithi106.stdout: "fsid": "3ee77985-e8e4-41cf-81e9-7c774cf7891f", 2024-04-25T10:31:14.229 INFO:teuthology.orchestra.run.smithi106.stdout: "modified": "2024-04-25T10:29:33.446228Z", 2024-04-25T10:31:14.229 INFO:teuthology.orchestra.run.smithi106.stdout: "created": "2024-04-25T10:29:33.446228Z", 2024-04-25T10:31:14.229 INFO:teuthology.orchestra.run.smithi106.stdout: "min_mon_release": 19, 2024-04-25T10:31:14.229 INFO:teuthology.orchestra.run.smithi106.stdout: "min_mon_release_name": "squid", 2024-04-25T10:31:14.229 INFO:teuthology.orchestra.run.smithi106.stdout: "election_strategy": 1, 2024-04-25T10:31:14.229 INFO:teuthology.orchestra.run.smithi106.stdout: "disallowed_leaders: ": "", 2024-04-25T10:31:14.229 INFO:teuthology.orchestra.run.smithi106.stdout: "stretch_mode": false, 2024-04-25T10:31:14.230 INFO:teuthology.orchestra.run.smithi106.stdout: "tiebreaker_mon": "", 2024-04-25T10:31:14.230 INFO:teuthology.orchestra.run.smithi106.stdout: "removed_ranks: ": "", 2024-04-25T10:31:14.230 INFO:teuthology.orchestra.run.smithi106.stdout: "features": { 2024-04-25T10:31:14.230 INFO:teuthology.orchestra.run.smithi106.stdout: "persistent": [ 2024-04-25T10:31:14.230 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:31:14.230 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:31:14.230 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:31:14.230 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:31:14.230 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:31:14.230 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:31:14.230 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:31:14.230 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:31:14.230 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:31:14.231 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:31:14.231 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:31:14.231 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:31:14.231 INFO:teuthology.orchestra.run.smithi106.stdout: "optional": [] 2024-04-25T10:31:14.231 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:14.231 INFO:teuthology.orchestra.run.smithi106.stdout: "mons": [ 2024-04-25T10:31:14.231 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:14.231 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 0, 2024-04-25T10:31:14.231 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "a", 2024-04-25T10:31:14.231 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:31:14.231 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:31:14.231 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:14.231 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:31:14.231 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:3300", 2024-04-25T10:31:14.232 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:31:14.232 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:14.232 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:14.232 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:31:14.232 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:6789", 2024-04-25T10:31:14.232 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:31:14.232 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:31:14.232 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:31:14.232 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:14.232 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:6789/0", 2024-04-25T10:31:14.232 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.106:6789/0", 2024-04-25T10:31:14.232 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:31:14.232 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:31:14.232 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:31:14.233 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:14.233 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:14.233 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 1, 2024-04-25T10:31:14.233 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "b", 2024-04-25T10:31:14.233 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:31:14.233 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:31:14.233 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:14.233 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:31:14.233 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:3300", 2024-04-25T10:31:14.233 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:31:14.233 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:14.233 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:14.233 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:31:14.233 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6789", 2024-04-25T10:31:14.234 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:31:14.234 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:31:14.234 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:31:14.234 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:14.234 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6789/0", 2024-04-25T10:31:14.234 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.134:6789/0", 2024-04-25T10:31:14.234 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:31:14.234 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:31:14.234 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:31:14.234 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:14.234 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:14.234 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 2, 2024-04-25T10:31:14.234 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "c", 2024-04-25T10:31:14.234 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:31:14.235 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:31:14.235 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:14.235 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:31:14.235 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:3301", 2024-04-25T10:31:14.235 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:31:14.235 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:14.235 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:14.235 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:31:14.235 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6790", 2024-04-25T10:31:14.235 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:31:14.235 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:31:14.235 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:31:14.235 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:14.235 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6790/0", 2024-04-25T10:31:14.235 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.134:6790/0", 2024-04-25T10:31:14.236 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:31:14.236 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:31:14.236 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:31:14.236 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:31:14.236 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:31:14.236 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:14.236 INFO:teuthology.orchestra.run.smithi106.stdout: "feature_map": { 2024-04-25T10:31:14.236 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": [ 2024-04-25T10:31:14.236 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:14.236 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:31:14.236 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:31:14.236 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 1 2024-04-25T10:31:14.236 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:31:14.236 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:31:14.236 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": [ 2024-04-25T10:31:14.237 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:14.237 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:31:14.237 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:31:14.237 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 2 2024-04-25T10:31:14.237 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:31:14.237 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:31:14.237 INFO:teuthology.orchestra.run.smithi106.stdout: "client": [ 2024-04-25T10:31:14.237 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:14.237 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:31:14.237 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:31:14.237 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 1 2024-04-25T10:31:14.237 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:31:14.237 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:31:14.237 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:14.237 INFO:teuthology.orchestra.run.smithi106.stdout: "stretch_mode": false 2024-04-25T10:31:14.238 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-25T10:31:14.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.223+0000 7fd070934640 1 -- 172.21.15.106:0/3054659499 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fd04c0744b0 msgr2=0x7fd04c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:14.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.223+0000 7fd070934640 1 --2- 172.21.15.106:0/3054659499 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fd04c0744b0 0x7fd04c0769c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd06c1f30a0 tx=0x7fd064009040 comp rx=0 tx=0).stop 2024-04-25T10:31:14.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.223+0000 7fd070934640 1 -- 172.21.15.106:0/3054659499 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd06c156ec0 msgr2=0x7fd06c155dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:14.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.223+0000 7fd070934640 1 --2- 172.21.15.106:0/3054659499 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd06c156ec0 0x7fd06c155dc0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd05402a2d0 tx=0x7fd0540046c0 comp rx=0 tx=0).stop 2024-04-25T10:31:14.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.223+0000 7fd06ad9d640 1 -- 172.21.15.106:0/3054659499 reap_dead start 2024-04-25T10:31:14.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.223+0000 7fd06ad9d640 1 -- 172.21.15.106:0/3054659499 reap_dead start 2024-04-25T10:31:14.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.223+0000 7fd070934640 1 -- 172.21.15.106:0/3054659499 shutdown_connections 2024-04-25T10:31:14.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.223+0000 7fd070934640 1 -- 172.21.15.106:0/3054659499 >> 172.21.15.106:0/3054659499 conn(0x7fd06c0727d0 msgr2=0x7fd06c154540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:14.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.223+0000 7fd070934640 1 -- 172.21.15.106:0/3054659499 shutdown_connections 2024-04-25T10:31:14.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.223+0000 7fd070934640 1 -- 172.21.15.106:0/3054659499 wait complete. 2024-04-25T10:31:14.240 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T10:31:14.240 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-04-25T10:31:14.240 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-25T10:31:14.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.346+0000 7f6c65f1f640 1 Processor -- start 2024-04-25T10:31:14.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.346+0000 7f6c65f1f640 1 -- start start 2024-04-25T10:31:14.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.347+0000 7f6c65f1f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c600c7e30 0x7f6c600c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:14.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.347+0000 7f6c65f1f640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f6c600b4ce0 0x7f6c6015ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:14.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.347+0000 7f6c65f1f640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c600ce630 con 0x7f6c60156ef0 2024-04-25T10:31:14.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.347+0000 7f6c65f1f640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f6c600b50f0 con 0x7f6c600b4ce0 2024-04-25T10:31:14.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.347+0000 7f6c65f1f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6c600c7c60 con 0x7f6c600c7e30 2024-04-25T10:31:14.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.347+0000 7f6c5f7fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c600c7e30 0x7f6c600c8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:14.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.347+0000 7f6c5ffff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f6c600b4ce0 0x7f6c6015ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:14.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.347+0000 7f6c5ffff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f6c600b4ce0 0x7f6c6015ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.106:49366/0 (socket says 172.21.15.106:49366) 2024-04-25T10:31:14.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.347+0000 7f6c5effd640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f6c60156ef0 0x7f6c6006d000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.106:53290/0 (socket says 172.21.15.106:53290) 2024-04-25T10:31:14.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.347+0000 7f6c5f7fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c600c7e30 0x7f6c600c8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.106:46052/0 (socket says 172.21.15.106:46052) 2024-04-25T10:31:14.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.347+0000 7f6c5ffff640 1 -- 172.21.15.106:0/2860459204 learned_addr learned my addr 172.21.15.106:0/2860459204 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:31:14.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.348+0000 7f6c5ffff640 1 -- 172.21.15.106:0/2860459204 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c600c7e30 msgr2=0x7f6c600c8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:14.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.348+0000 7f6c5ffff640 1 --2- 172.21.15.106:0/2860459204 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c600c7e30 0x7f6c600c8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:14.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.348+0000 7f6c5ffff640 1 -- 172.21.15.106:0/2860459204 >> v1:172.21.15.134:6789/0 conn(0x7f6c60156ef0 legacy=0x7f6c6006d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:14.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.348+0000 7f6c5ffff640 1 -- 172.21.15.106:0/2860459204 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c600c7980 con 0x7f6c600b4ce0 2024-04-25T10:31:14.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.349+0000 7f6c5ffff640 1 --2- 172.21.15.106:0/2860459204 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c600b4ce0 0x7f6c6015ce30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6c58009e50 tx=0x7f6c5802a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4617c3026bdcfcd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:14.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.349+0000 7f6c5e7fc640 1 -- 172.21.15.106:0/2860459204 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c58036070 con 0x7f6c600b4ce0 2024-04-25T10:31:14.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.349+0000 7f6c5e7fc640 1 -- 172.21.15.106:0/2860459204 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c58031600 con 0x7f6c600b4ce0 2024-04-25T10:31:14.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.349+0000 7f6c5e7fc640 1 -- 172.21.15.106:0/2860459204 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c580319d0 con 0x7f6c600b4ce0 2024-04-25T10:31:14.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.349+0000 7f6c65f1f640 1 -- 172.21.15.106:0/2860459204 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c600b4ce0 msgr2=0x7f6c6015ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:14.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.349+0000 7f6c65f1f640 1 --2- 172.21.15.106:0/2860459204 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c600b4ce0 0x7f6c6015ce30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6c58009e50 tx=0x7f6c5802a680 comp rx=0 tx=0).stop 2024-04-25T10:31:14.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.349+0000 7f6c65f1f640 1 -- 172.21.15.106:0/2860459204 shutdown_connections 2024-04-25T10:31:14.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.349+0000 7f6c65f1f640 1 --2- 172.21.15.106:0/2860459204 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c600b4ce0 0x7f6c6015ce30 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:14.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.349+0000 7f6c65f1f640 1 --2- 172.21.15.106:0/2860459204 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c600c7e30 0x7f6c600c8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:14.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.349+0000 7f6c65f1f640 1 -- 172.21.15.106:0/2860459204 >> 172.21.15.106:0/2860459204 conn(0x7f6c60068b10 msgr2=0x7f6c600d1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:14.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.349+0000 7f6c65f1f640 1 -- 172.21.15.106:0/2860459204 shutdown_connections 2024-04-25T10:31:14.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.349+0000 7f6c65f1f640 1 -- 172.21.15.106:0/2860459204 wait complete. 2024-04-25T10:31:14.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.351+0000 7f6c65f1f640 1 Processor -- start 2024-04-25T10:31:14.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.351+0000 7f6c65f1f640 1 -- start start 2024-04-25T10:31:14.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.352+0000 7f6c65f1f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c600b4ce0 0x7f6c600d0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:14.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.352+0000 7f6c65f1f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c600c7e30 0x7f6c6014ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:14.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.352+0000 7f6c65f1f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c60156ef0 0x7f6c6015e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:14.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.352+0000 7f6c65f1f640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6c6006d710 con 0x7f6c600c7e30 2024-04-25T10:31:14.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.352+0000 7f6c65f1f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6c600c72f0 con 0x7f6c60156ef0 2024-04-25T10:31:14.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.352+0000 7f6c65f1f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6c6006d890 con 0x7f6c600b4ce0 2024-04-25T10:31:14.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.353+0000 7f6c5effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c600c7e30 0x7f6c6014ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:14.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.353+0000 7f6c5effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c600c7e30 0x7f6c6014ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35628/0 (socket says 172.21.15.106:35628) 2024-04-25T10:31:14.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.353+0000 7f6c5effd640 1 -- 172.21.15.106:0/1998311589 learned_addr learned my addr 172.21.15.106:0/1998311589 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:31:14.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.353+0000 7f6c5ffff640 1 --2- 172.21.15.106:0/1998311589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c60156ef0 0x7f6c6015e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:14.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.353+0000 7f6c5f7fe640 1 --2- 172.21.15.106:0/1998311589 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c600b4ce0 0x7f6c600d0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:14.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.353+0000 7f6c5effd640 1 -- 172.21.15.106:0/1998311589 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c600b4ce0 msgr2=0x7f6c600d0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:14.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.353+0000 7f6c5effd640 1 --2- 172.21.15.106:0/1998311589 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c600b4ce0 0x7f6c600d0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:14.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.353+0000 7f6c5effd640 1 -- 172.21.15.106:0/1998311589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c60156ef0 msgr2=0x7f6c6015e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:14.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.353+0000 7f6c5effd640 1 --2- 172.21.15.106:0/1998311589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c60156ef0 0x7f6c6015e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:14.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.353+0000 7f6c5effd640 1 -- 172.21.15.106:0/1998311589 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c600d0790 con 0x7f6c600c7e30 2024-04-25T10:31:14.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.353+0000 7f6c5effd640 1 --2- 172.21.15.106:0/1998311589 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c600c7e30 0x7f6c6014ab80 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6c5400c3c0 tx=0x7f6c54001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:14.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.353+0000 7f6c47fff640 1 -- 172.21.15.106:0/1998311589 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c540074e0 con 0x7f6c600c7e30 2024-04-25T10:31:14.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.353+0000 7f6c47fff640 1 -- 172.21.15.106:0/1998311589 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c540076a0 con 0x7f6c600c7e30 2024-04-25T10:31:14.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.353+0000 7f6c65f1f640 1 -- 172.21.15.106:0/1998311589 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c60154ab0 con 0x7f6c600c7e30 2024-04-25T10:31:14.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.353+0000 7f6c47fff640 1 -- 172.21.15.106:0/1998311589 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c54004a20 con 0x7f6c600c7e30 2024-04-25T10:31:14.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.353+0000 7f6c65f1f640 1 -- 172.21.15.106:0/1998311589 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c60155700 con 0x7f6c600c7e30 2024-04-25T10:31:14.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.354+0000 7f6c65f1f640 1 --2- 172.21.15.106:0/1998311589 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c600c7530 0x7f6c6015dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:14.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.354+0000 7f6c65f1f640 1 -- 172.21.15.106:0/1998311589 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6c600d0c60 con 0x7f6c600c7530 2024-04-25T10:31:14.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.354+0000 7f6c5f7fe640 1 --2- 172.21.15.106:0/1998311589 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c600c7530 0x7f6c6015dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:14.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.355+0000 7f6c5f7fe640 1 -- 172.21.15.106:0/1998311589 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6c6006a3d0 con 0x7f6c600c7530 2024-04-25T10:31:14.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.355+0000 7f6c5f7fe640 1 --2- 172.21.15.106:0/1998311589 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c600c7530 0x7f6c6015dd40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6c50011890 tx=0x7f6c50011d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:14.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.355+0000 7f6c47fff640 1 -- 172.21.15.106:0/1998311589 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6c5401a020 con 0x7f6c600c7e30 2024-04-25T10:31:14.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.355+0000 7f6c47fff640 1 --2- 172.21.15.106:0/1998311589 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f6c4c0744b0 0x7f6c4c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:14.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.355+0000 7f6c47fff640 1 -- 172.21.15.106:0/1998311589 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f6c540949c0 con 0x7f6c600c7e30 2024-04-25T10:31:14.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.355+0000 7f6c5ffff640 1 --2- 172.21.15.106:0/1998311589 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f6c4c0744b0 0x7f6c4c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:14.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.356+0000 7f6c47fff640 1 -- 172.21.15.106:0/1998311589 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c50019070 con 0x7f6c600c7530 2024-04-25T10:31:14.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.356+0000 7f6c47fff640 1 -- 172.21.15.106:0/1998311589 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f6c50004240 con 0x7f6c600c7530 2024-04-25T10:31:14.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.356+0000 7f6c47fff640 1 -- 172.21.15.106:0/1998311589 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c600c7530 msgr2=0x7f6c6015dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:14.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.356+0000 7f6c47fff640 1 --2- 172.21.15.106:0/1998311589 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c600c7530 0x7f6c6015dd40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6c50011890 tx=0x7f6c50011d50 comp rx=0 tx=0).stop 2024-04-25T10:31:14.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.356+0000 7f6c47fff640 1 -- 172.21.15.106:0/1998311589 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c600c7530 msgr2=0x7f6c6015dd40 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T10:31:14.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.356+0000 7f6c47fff640 1 --2- 172.21.15.106:0/1998311589 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c600c7530 0x7f6c6015dd40 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:14.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.356+0000 7f6c5ffff640 1 --2- 172.21.15.106:0/1998311589 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f6c4c0744b0 0x7f6c4c0769c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6c58033000 tx=0x7f6c58009000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:14.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.362+0000 7f6c65f1f640 1 --2- 172.21.15.106:0/1998311589 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c6015e7d0 0x7f6c6015f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:14.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.362+0000 7f6c65f1f640 1 -- 172.21.15.106:0/1998311589 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6c6015c960 con 0x7f6c6015e7d0 2024-04-25T10:31:14.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.362+0000 7f6c5f7fe640 1 --2- 172.21.15.106:0/1998311589 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c6015e7d0 0x7f6c6015f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:14.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.363+0000 7f6c5f7fe640 1 -- 172.21.15.106:0/1998311589 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f6c6015a5a0 con 0x7f6c6015e7d0 2024-04-25T10:31:14.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.363+0000 7f6c5f7fe640 1 --2- 172.21.15.106:0/1998311589 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c6015e7d0 0x7f6c6015f7d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6c50011890 tx=0x7f6c500046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:14.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.363+0000 7f6c47fff640 1 -- 172.21.15.106:0/1998311589 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c50019040 con 0x7f6c6015e7d0 2024-04-25T10:31:14.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.363+0000 7f6c47fff640 1 -- 172.21.15.106:0/1998311589 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4898 (secure 0 0 0) 0x7f6c50003da0 con 0x7f6c6015e7d0 2024-04-25T10:31:14.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.363+0000 7f6c47fff640 1 -- 172.21.15.106:0/1998311589 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c6015e7d0 msgr2=0x7f6c6015f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:14.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.363+0000 7f6c47fff640 1 --2- 172.21.15.106:0/1998311589 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c6015e7d0 0x7f6c6015f7d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6c50011890 tx=0x7f6c500046c0 comp rx=0 tx=0).stop 2024-04-25T10:31:14.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.363+0000 7f6c47fff640 1 -- 172.21.15.106:0/1998311589 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c6015e7d0 msgr2=0x7f6c6015f7d0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T10:31:14.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.363+0000 7f6c47fff640 1 --2- 172.21.15.106:0/1998311589 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c6015e7d0 0x7f6c6015f7d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:14.366 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-25T10:31:14.367 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "c", 2024-04-25T10:31:14.367 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 2, 2024-04-25T10:31:14.367 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "peon", 2024-04-25T10:31:14.367 INFO:teuthology.orchestra.run.smithi106.stdout: "election_epoch": 4, 2024-04-25T10:31:14.367 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 66559, 2024-04-25T10:31:14.367 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum": [ 2024-04-25T10:31:14.367 INFO:teuthology.orchestra.run.smithi106.stdout: 0, 2024-04-25T10:31:14.367 INFO:teuthology.orchestra.run.smithi106.stdout: 1, 2024-04-25T10:31:14.367 INFO:teuthology.orchestra.run.smithi106.stdout: 2 2024-04-25T10:31:14.368 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:31:14.368 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_age": 66, 2024-04-25T10:31:14.368 INFO:teuthology.orchestra.run.smithi106.stdout: "features": { 2024-04-25T10:31:14.368 INFO:teuthology.orchestra.run.smithi106.stdout: "required_con": "2450521708007866372", 2024-04-25T10:31:14.368 INFO:teuthology.orchestra.run.smithi106.stdout: "required_mon": [ 2024-04-25T10:31:14.368 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:31:14.368 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:31:14.368 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:31:14.368 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:31:14.368 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:31:14.368 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:31:14.368 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:31:14.368 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:31:14.368 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:31:14.369 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:31:14.369 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:31:14.369 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:31:14.369 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_con": "4540701547738038271", 2024-04-25T10:31:14.369 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_mon": [ 2024-04-25T10:31:14.369 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:31:14.369 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:31:14.369 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:31:14.369 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:31:14.369 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:31:14.369 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:31:14.369 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:31:14.369 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:31:14.369 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:31:14.370 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:31:14.370 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:31:14.370 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:31:14.370 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:14.370 INFO:teuthology.orchestra.run.smithi106.stdout: "outside_quorum": [], 2024-04-25T10:31:14.370 INFO:teuthology.orchestra.run.smithi106.stdout: "extra_probe_peers": [], 2024-04-25T10:31:14.370 INFO:teuthology.orchestra.run.smithi106.stdout: "sync_provider": [], 2024-04-25T10:31:14.370 INFO:teuthology.orchestra.run.smithi106.stdout: "monmap": { 2024-04-25T10:31:14.370 INFO:teuthology.orchestra.run.smithi106.stdout: "epoch": 1, 2024-04-25T10:31:14.370 INFO:teuthology.orchestra.run.smithi106.stdout: "fsid": "3ee77985-e8e4-41cf-81e9-7c774cf7891f", 2024-04-25T10:31:14.370 INFO:teuthology.orchestra.run.smithi106.stdout: "modified": "2024-04-25T10:29:33.446228Z", 2024-04-25T10:31:14.370 INFO:teuthology.orchestra.run.smithi106.stdout: "created": "2024-04-25T10:29:33.446228Z", 2024-04-25T10:31:14.370 INFO:teuthology.orchestra.run.smithi106.stdout: "min_mon_release": 19, 2024-04-25T10:31:14.370 INFO:teuthology.orchestra.run.smithi106.stdout: "min_mon_release_name": "squid", 2024-04-25T10:31:14.371 INFO:teuthology.orchestra.run.smithi106.stdout: "election_strategy": 1, 2024-04-25T10:31:14.371 INFO:teuthology.orchestra.run.smithi106.stdout: "disallowed_leaders: ": "", 2024-04-25T10:31:14.371 INFO:teuthology.orchestra.run.smithi106.stdout: "stretch_mode": false, 2024-04-25T10:31:14.371 INFO:teuthology.orchestra.run.smithi106.stdout: "tiebreaker_mon": "", 2024-04-25T10:31:14.371 INFO:teuthology.orchestra.run.smithi106.stdout: "removed_ranks: ": "", 2024-04-25T10:31:14.371 INFO:teuthology.orchestra.run.smithi106.stdout: "features": { 2024-04-25T10:31:14.371 INFO:teuthology.orchestra.run.smithi106.stdout: "persistent": [ 2024-04-25T10:31:14.371 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:31:14.371 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:31:14.371 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:31:14.371 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:31:14.371 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:31:14.371 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:31:14.371 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:31:14.372 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:31:14.372 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:31:14.372 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:31:14.372 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:31:14.372 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:31:14.372 INFO:teuthology.orchestra.run.smithi106.stdout: "optional": [] 2024-04-25T10:31:14.372 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:14.372 INFO:teuthology.orchestra.run.smithi106.stdout: "mons": [ 2024-04-25T10:31:14.372 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:14.372 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 0, 2024-04-25T10:31:14.372 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "a", 2024-04-25T10:31:14.372 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:31:14.372 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:31:14.372 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:14.372 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:31:14.372 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:3300", 2024-04-25T10:31:14.373 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:31:14.373 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:14.373 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:14.373 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:31:14.373 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:6789", 2024-04-25T10:31:14.373 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:31:14.373 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:31:14.373 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:31:14.373 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:14.373 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:6789/0", 2024-04-25T10:31:14.373 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.106:6789/0", 2024-04-25T10:31:14.373 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:31:14.373 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:31:14.373 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:31:14.373 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:14.374 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:14.374 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 1, 2024-04-25T10:31:14.374 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "b", 2024-04-25T10:31:14.374 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:31:14.374 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:31:14.374 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:14.374 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:31:14.374 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:3300", 2024-04-25T10:31:14.374 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:31:14.374 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:14.374 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:14.375 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:31:14.375 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6789", 2024-04-25T10:31:14.375 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:31:14.375 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:31:14.375 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:31:14.375 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:14.375 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6789/0", 2024-04-25T10:31:14.375 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.134:6789/0", 2024-04-25T10:31:14.375 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:31:14.375 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:31:14.375 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:31:14.375 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:14.376 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:14.376 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 2, 2024-04-25T10:31:14.376 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "c", 2024-04-25T10:31:14.376 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:31:14.376 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:31:14.376 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:14.376 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:31:14.376 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:3301", 2024-04-25T10:31:14.376 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:31:14.376 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:14.376 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:14.376 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:31:14.376 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6790", 2024-04-25T10:31:14.376 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:31:14.376 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:31:14.377 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:31:14.377 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:14.377 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6790/0", 2024-04-25T10:31:14.378 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.134:6790/0", 2024-04-25T10:31:14.378 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:31:14.378 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:31:14.378 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:31:14.378 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:31:14.378 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:31:14.378 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:14.379 INFO:teuthology.orchestra.run.smithi106.stdout: "feature_map": { 2024-04-25T10:31:14.379 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": [ 2024-04-25T10:31:14.379 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:14.379 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:31:14.379 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:31:14.379 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 1 2024-04-25T10:31:14.379 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:31:14.379 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:31:14.379 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": [ 2024-04-25T10:31:14.380 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:14.380 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:31:14.380 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:31:14.380 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 5 2024-04-25T10:31:14.380 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:31:14.380 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:31:14.380 INFO:teuthology.orchestra.run.smithi106.stdout: "client": [ 2024-04-25T10:31:14.380 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:14.381 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:31:14.381 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:31:14.381 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 1 2024-04-25T10:31:14.381 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:31:14.381 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:31:14.381 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": [ 2024-04-25T10:31:14.381 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:14.381 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:31:14.381 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:31:14.381 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 1 2024-04-25T10:31:14.381 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:31:14.382 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:31:14.382 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:14.382 INFO:teuthology.orchestra.run.smithi106.stdout: "stretch_mode": false 2024-04-25T10:31:14.382 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-25T10:31:14.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.364+0000 7f6c65f1f640 1 -- 172.21.15.106:0/1998311589 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f6c4c0744b0 msgr2=0x7f6c4c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:14.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.364+0000 7f6c65f1f640 1 --2- 172.21.15.106:0/1998311589 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f6c4c0744b0 0x7f6c4c0769c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6c58033000 tx=0x7f6c58009000 comp rx=0 tx=0).stop 2024-04-25T10:31:14.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.364+0000 7f6c65f1f640 1 -- 172.21.15.106:0/1998311589 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c600c7e30 msgr2=0x7f6c6014ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:14.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.364+0000 7f6c65f1f640 1 --2- 172.21.15.106:0/1998311589 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c600c7e30 0x7f6c6014ab80 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6c5400c3c0 tx=0x7f6c54001e70 comp rx=0 tx=0).stop 2024-04-25T10:31:14.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.364+0000 7f6c5ffff640 1 -- 172.21.15.106:0/1998311589 reap_dead start 2024-04-25T10:31:14.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.364+0000 7f6c65f1f640 1 -- 172.21.15.106:0/1998311589 shutdown_connections 2024-04-25T10:31:14.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.364+0000 7f6c65f1f640 1 --2- 172.21.15.106:0/1998311589 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c600c7e30 0x7f6c6014ab80 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:14.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.364+0000 7f6c65f1f640 1 -- 172.21.15.106:0/1998311589 >> 172.21.15.106:0/1998311589 conn(0x7f6c60068b10 msgr2=0x7f6c6015b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:14.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.364+0000 7f6c65f1f640 1 -- 172.21.15.106:0/1998311589 shutdown_connections 2024-04-25T10:31:14.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.364+0000 7f6c65f1f640 1 -- 172.21.15.106:0/1998311589 wait complete. 2024-04-25T10:31:14.384 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T10:31:14.385 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c'] 2024-04-25T10:31:14.385 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-04-25T10:31:14.385 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-04-25T10:31:14.385 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-04-25T10:31:14.385 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-25T10:31:14.385 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:31:14.401 INFO:tasks.ceph.mon.c:Stopped 2024-04-25T10:31:14.401 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-04-25T10:31:14.402 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T10:31:14.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.507+0000 7f1ba7d2d640 1 Processor -- start 2024-04-25T10:31:14.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.507+0000 7f1ba7d2d640 1 -- start start 2024-04-25T10:31:14.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.508+0000 7f1ba7d2d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1ba00cbff0 0x7f1ba00cb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:14.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.508+0000 7f1ba7d2d640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ba015dae0 con 0x7f1ba0156ec0 2024-04-25T10:31:14.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.508+0000 7f1ba7d2d640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ba01604b0 con 0x7f1ba00cce00 2024-04-25T10:31:14.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.508+0000 7f1ba7d2d640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f1ba0161270 con 0x7f1ba00cbff0 2024-04-25T10:31:14.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.508+0000 7f1ba5ac9640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1ba00cbff0 0x7f1ba00cb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:14.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.508+0000 7f1ba52c8640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f1ba0156ec0 0x7f1ba00c61a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:34508/0 (socket says 172.21.15.106:34508) 2024-04-25T10:31:14.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.508+0000 7f1ba52c8640 1 -- 172.21.15.106:0/2804160127 learned_addr learned my addr 172.21.15.106:0/2804160127 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:31:14.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.509+0000 7f1ba5ac9640 1 -- 172.21.15.106:0/2804160127 >> v1:172.21.15.134:6789/0 conn(0x7f1ba00cce00 legacy=0x7f1ba0167630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:14.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.509+0000 7f1ba5ac9640 1 -- 172.21.15.106:0/2804160127 >> v1:172.21.15.106:6789/0 conn(0x7f1ba0156ec0 legacy=0x7f1ba00c61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:14.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.509+0000 7f1ba5ac9640 1 -- 172.21.15.106:0/2804160127 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ba0160790 con 0x7f1ba00cbff0 2024-04-25T10:31:14.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.509+0000 7f1ba4ac7640 1 -- 172.21.15.106:0/2804160127 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1800168061 0 0) 0x7f1ba015dae0 con 0x7f1ba0156ec0 2024-04-25T10:31:14.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.510+0000 7f1ba5ac9640 1 --2- 172.21.15.106:0/2804160127 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1ba00cbff0 0x7f1ba00cb270 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c0097a0 tx=0x7f1b9c02a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32624f895f1f113a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:14.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.510+0000 7f1ba4ac7640 1 -- 172.21.15.106:0/2804160127 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c037070 con 0x7f1ba00cbff0 2024-04-25T10:31:14.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.510+0000 7f1ba4ac7640 1 -- 172.21.15.106:0/2804160127 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b9c032640 con 0x7f1ba00cbff0 2024-04-25T10:31:14.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.510+0000 7f1ba4ac7640 1 -- 172.21.15.106:0/2804160127 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c032a10 con 0x7f1ba00cbff0 2024-04-25T10:31:14.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.510+0000 7f1ba7d2d640 1 -- 172.21.15.106:0/2804160127 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1ba00cbff0 msgr2=0x7f1ba00cb270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:14.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.510+0000 7f1ba7d2d640 1 --2- 172.21.15.106:0/2804160127 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1ba00cbff0 0x7f1ba00cb270 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c0097a0 tx=0x7f1b9c02a2c0 comp rx=0 tx=0).stop 2024-04-25T10:31:14.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.510+0000 7f1ba7d2d640 1 -- 172.21.15.106:0/2804160127 shutdown_connections 2024-04-25T10:31:14.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.510+0000 7f1ba7d2d640 1 --2- 172.21.15.106:0/2804160127 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1ba00cbff0 0x7f1ba00cb270 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:14.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.510+0000 7f1ba7d2d640 1 -- 172.21.15.106:0/2804160127 >> 172.21.15.106:0/2804160127 conn(0x7f1ba00727d0 msgr2=0x7f1ba0072c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:14.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.511+0000 7f1ba7d2d640 1 -- 172.21.15.106:0/2804160127 shutdown_connections 2024-04-25T10:31:14.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.511+0000 7f1ba7d2d640 1 -- 172.21.15.106:0/2804160127 wait complete. 2024-04-25T10:31:14.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.512+0000 7f1ba7d2d640 1 Processor -- start 2024-04-25T10:31:14.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.512+0000 7f1ba7d2d640 1 -- start start 2024-04-25T10:31:14.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.513+0000 7f1ba7d2d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1ba00cce00 0x7f1ba01f29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:14.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.513+0000 7f1ba7d2d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1ba0156ec0 0x7f1ba015fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:14.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.513+0000 7f1ba7d2d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ba014c410 0x7f1ba0156120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:14.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.513+0000 7f1ba7d2d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba01572d0 con 0x7f1ba0156ec0 2024-04-25T10:31:14.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.513+0000 7f1ba7d2d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba0160d50 con 0x7f1ba014c410 2024-04-25T10:31:14.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.513+0000 7f1ba7d2d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f1ba015d730 con 0x7f1ba00cce00 2024-04-25T10:31:14.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.514+0000 7f1ba52c8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1ba0156ec0 0x7f1ba015fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:14.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.514+0000 7f1ba52c8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1ba0156ec0 0x7f1ba015fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35660/0 (socket says 172.21.15.106:35660) 2024-04-25T10:31:14.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.514+0000 7f1ba5ac9640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1ba00cce00 msgr2=0x7f1ba01f29e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-04-25T10:31:14.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.514+0000 7f1ba52c8640 1 -- 172.21.15.106:0/1438439916 learned_addr learned my addr 172.21.15.106:0/1438439916 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:31:14.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.514+0000 7f1ba5ac9640 1 --2- 172.21.15.106:0/1438439916 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1ba00cce00 0x7f1ba01f29e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:31:14.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.514+0000 7f1ba62ca640 1 --2- 172.21.15.106:0/1438439916 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ba014c410 0x7f1ba0156120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:14.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.514+0000 7f1ba52c8640 1 -- 172.21.15.106:0/1438439916 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1ba00cce00 msgr2=0x7f1ba01f29e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:31:14.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.514+0000 7f1ba52c8640 1 --2- 172.21.15.106:0/1438439916 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1ba00cce00 0x7f1ba01f29e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:14.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.514+0000 7f1ba52c8640 1 -- 172.21.15.106:0/1438439916 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ba014c410 msgr2=0x7f1ba0156120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:14.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.514+0000 7f1ba52c8640 1 --2- 172.21.15.106:0/1438439916 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ba014c410 0x7f1ba0156120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:14.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.514+0000 7f1ba52c8640 1 -- 172.21.15.106:0/1438439916 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ba01f2f20 con 0x7f1ba0156ec0 2024-04-25T10:31:14.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.514+0000 7f1ba62ca640 1 --2- 172.21.15.106:0/1438439916 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ba014c410 0x7f1ba0156120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:31:14.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.514+0000 7f1ba52c8640 1 --2- 172.21.15.106:0/1438439916 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1ba0156ec0 0x7f1ba015fbf0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1b9000c3f0 tx=0x7f1b9000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:14.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.514+0000 7f1b967fc640 1 -- 172.21.15.106:0/1438439916 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b90017070 con 0x7f1ba0156ec0 2024-04-25T10:31:14.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.514+0000 7f1b967fc640 1 -- 172.21.15.106:0/1438439916 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b90005ce0 con 0x7f1ba0156ec0 2024-04-25T10:31:14.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.514+0000 7f1b967fc640 1 -- 172.21.15.106:0/1438439916 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b90007310 con 0x7f1ba0156ec0 2024-04-25T10:31:14.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.514+0000 7f1ba7d2d640 1 -- 172.21.15.106:0/1438439916 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ba0154730 con 0x7f1ba0156ec0 2024-04-25T10:31:14.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.514+0000 7f1ba7d2d640 1 -- 172.21.15.106:0/1438439916 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ba01554d0 con 0x7f1ba0156ec0 2024-04-25T10:31:14.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.515+0000 7f1ba7d2d640 1 -- 172.21.15.106:0/1438439916 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ba0156af0 con 0x7f1ba0156ec0 2024-04-25T10:31:14.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.516+0000 7f1b967fc640 1 -- 172.21.15.106:0/1438439916 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1b9001a070 con 0x7f1ba0156ec0 2024-04-25T10:31:14.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.516+0000 7f1b967fc640 1 --2- 172.21.15.106:0/1438439916 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f1b84074460 0x7f1b84076970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:14.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.517+0000 7f1ba5ac9640 1 --2- 172.21.15.106:0/1438439916 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f1b84074460 0x7f1b84076970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:14.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.517+0000 7f1b967fc640 1 -- 172.21.15.106:0/1438439916 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f1b90094620 con 0x7f1ba0156ec0 2024-04-25T10:31:14.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.517+0000 7f1ba5ac9640 1 --2- 172.21.15.106:0/1438439916 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f1b84074460 0x7f1b84076970 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c034000 tx=0x7f1b9c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:14.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.521+0000 7f1b967fc640 1 -- 172.21.15.106:0/1438439916 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1b9005ed30 con 0x7f1ba0156ec0 2024-04-25T10:31:14.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.768+0000 7f1ba7d2d640 1 -- 172.21.15.106:0/1438439916 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f1ba006a3d0 con 0x7f1ba0156ec0 2024-04-25T10:31:14.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.769+0000 7f1b967fc640 1 -- 172.21.15.106:0/1438439916 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1568 (secure 0 0 0) 0x7f1b9005e630 con 0x7f1ba0156ec0 2024-04-25T10:31:14.772 INFO:teuthology.orchestra.run.smithi106.stdout:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":66,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"3ee77985-e8e4-41cf-81e9-7c774cf7891f","modified":"2024-04-25T10:29:33.446228Z","created":"2024-04-25T10:29:33.446228Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:3300","nonce":0},{"type":"v1","addr":"172.21.15.106:6789","nonce":0}]},"addr":"172.21.15.106:6789/0","public_addr":"172.21.15.106:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T10:31:14.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.771+0000 7f1ba7d2d640 1 -- 172.21.15.106:0/1438439916 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f1b84074460 msgr2=0x7f1b84076970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:14.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.771+0000 7f1ba7d2d640 1 --2- 172.21.15.106:0/1438439916 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f1b84074460 0x7f1b84076970 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c034000 tx=0x7f1b9c02e000 comp rx=0 tx=0).stop 2024-04-25T10:31:14.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.771+0000 7f1ba7d2d640 1 -- 172.21.15.106:0/1438439916 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1ba0156ec0 msgr2=0x7f1ba015fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:14.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.771+0000 7f1ba7d2d640 1 --2- 172.21.15.106:0/1438439916 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1ba0156ec0 0x7f1ba015fbf0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1b9000c3f0 tx=0x7f1b9000c8b0 comp rx=0 tx=0).stop 2024-04-25T10:31:14.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.771+0000 7f1ba7d2d640 1 -- 172.21.15.106:0/1438439916 shutdown_connections 2024-04-25T10:31:14.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.771+0000 7f1ba7d2d640 1 --2- 172.21.15.106:0/1438439916 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f1b84074460 0x7f1b84076970 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:14.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.771+0000 7f1ba7d2d640 1 --2- 172.21.15.106:0/1438439916 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ba014c410 0x7f1ba0156120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:14.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.771+0000 7f1ba7d2d640 1 --2- 172.21.15.106:0/1438439916 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1ba0156ec0 0x7f1ba015fbf0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:14.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.771+0000 7f1ba7d2d640 1 --2- 172.21.15.106:0/1438439916 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1ba00cce00 0x7f1ba01f29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:14.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.771+0000 7f1ba7d2d640 1 -- 172.21.15.106:0/1438439916 >> 172.21.15.106:0/1438439916 conn(0x7f1ba00727d0 msgr2=0x7f1ba00cfef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:14.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.771+0000 7f1ba7d2d640 1 -- 172.21.15.106:0/1438439916 shutdown_connections 2024-04-25T10:31:14.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:14.771+0000 7f1ba7d2d640 1 -- 172.21.15.106:0/1438439916 wait complete. 2024-04-25T10:31:17.785 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T10:31:17.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:17.911+0000 7f1104042640 1 Processor -- start 2024-04-25T10:31:17.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:17.912+0000 7f1104042640 1 -- start start 2024-04-25T10:31:17.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:17.913+0000 7f1104042640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f10fc0cb2b0 0x7f10fc0cce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:17.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:17.913+0000 7f1104042640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f10fc154ac0 0x7f10fc0cd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:17.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:17.913+0000 7f1104042640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10fc15b8c0 con 0x7f10fc0cc600 2024-04-25T10:31:17.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:17.913+0000 7f1104042640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f10fc15eea0 con 0x7f10fc0cb2b0 2024-04-25T10:31:17.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:17.913+0000 7f1104042640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f10fc15b5e0 con 0x7f10fc154ac0 2024-04-25T10:31:17.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:17.913+0000 7f11025df640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f10fc154ac0 msgr2=0x7f10fc0cd3d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-04-25T10:31:17.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:17.913+0000 7f11025df640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f10fc154ac0 0x7f10fc0cd3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:31:17.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:17.913+0000 7f1101dde640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f10fc0cc600 0x7f10fc0c5390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:34518/0 (socket says 172.21.15.106:34518) 2024-04-25T10:31:17.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:17.913+0000 7f1101dde640 1 -- 172.21.15.106:0/2959841959 learned_addr learned my addr 172.21.15.106:0/2959841959 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:31:17.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:17.913+0000 7f11015dd640 1 --2- 172.21.15.106:0/2959841959 >> v2:172.21.15.134:3300/0 conn(0x7f10fc0cb2b0 0x7f10fc0cce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:17.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:17.913+0000 7f1100ddc640 1 -- 172.21.15.106:0/2959841959 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 190641210 0 0) 0x7f10fc15b8c0 con 0x7f10fc0cc600 2024-04-25T10:31:17.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:17.913+0000 7f1100ddc640 1 -- 172.21.15.106:0/2959841959 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f10e4003520 con 0x7f10fc0cc600 2024-04-25T10:31:17.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:17.914+0000 7f1100ddc640 1 -- 172.21.15.106:0/2959841959 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4216526547 0 0) 0x7f10e4003520 con 0x7f10fc0cc600 2024-04-25T10:31:17.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:17.914+0000 7f11015dd640 1 -- 172.21.15.106:0/2959841959 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f10fc154ac0 msgr2=0x7f10fc0cd3d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:31:17.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:17.914+0000 7f11015dd640 1 --2- 172.21.15.106:0/2959841959 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f10fc154ac0 0x7f10fc0cd3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:17.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:17.914+0000 7f11015dd640 1 -- 172.21.15.106:0/2959841959 >> v1:172.21.15.106:6789/0 conn(0x7f10fc0cc600 legacy=0x7f10fc0c5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:17.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:17.914+0000 7f11015dd640 1 -- 172.21.15.106:0/2959841959 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10fc15e0e0 con 0x7f10fc0cb2b0 2024-04-25T10:31:17.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:17.915+0000 7f11015dd640 1 --2- 172.21.15.106:0/2959841959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10fc0cb2b0 0x7f10fc0cce90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f10f00097a0 tx=0x7f10f002a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d93666aba0472104 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:17.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:17.915+0000 7f1100ddc640 1 -- 172.21.15.106:0/2959841959 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f0037070 con 0x7f10fc0cb2b0 2024-04-25T10:31:17.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:17.915+0000 7f1100ddc640 1 -- 172.21.15.106:0/2959841959 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10f002abc0 con 0x7f10fc0cb2b0 2024-04-25T10:31:17.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:17.915+0000 7f1100ddc640 1 -- 172.21.15.106:0/2959841959 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f4001670 con 0x7f10fc0cb2b0 2024-04-25T10:31:17.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:17.916+0000 7f1104042640 1 -- 172.21.15.106:0/2959841959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10fc0cb2b0 msgr2=0x7f10fc0cce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:17.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:17.916+0000 7f1104042640 1 --2- 172.21.15.106:0/2959841959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10fc0cb2b0 0x7f10fc0cce90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f10f00097a0 tx=0x7f10f002a130 comp rx=0 tx=0).stop 2024-04-25T10:31:17.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:17.916+0000 7f1104042640 1 -- 172.21.15.106:0/2959841959 shutdown_connections 2024-04-25T10:31:17.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:17.916+0000 7f1104042640 1 --2- 172.21.15.106:0/2959841959 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f10fc154ac0 0x7f10fc0cd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:17.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:17.916+0000 7f1104042640 1 --2- 172.21.15.106:0/2959841959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10fc0cb2b0 0x7f10fc0cce90 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:17.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:17.916+0000 7f1104042640 1 -- 172.21.15.106:0/2959841959 >> 172.21.15.106:0/2959841959 conn(0x7f10fc0727d0 msgr2=0x7f10fc072c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:17.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:17.916+0000 7f1104042640 1 -- 172.21.15.106:0/2959841959 shutdown_connections 2024-04-25T10:31:17.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:17.916+0000 7f1104042640 1 -- 172.21.15.106:0/2959841959 wait complete. 2024-04-25T10:31:17.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:17.918+0000 7f1104042640 1 Processor -- start 2024-04-25T10:31:17.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:17.918+0000 7f1104042640 1 -- start start 2024-04-25T10:31:17.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:17.919+0000 7f1104042640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10fc0cb2b0 0x7f10fc1f36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:17.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:17.919+0000 7f1104042640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f10fc0cc600 0x7f10fc1f6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:17.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:17.919+0000 7f1104042640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10fc154ac0 0x7f10fc1e9790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:17.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:17.919+0000 7f1104042640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f10fc15e980 con 0x7f10fc154ac0 2024-04-25T10:31:17.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:17.919+0000 7f1104042640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f10fc15e6a0 con 0x7f10fc0cb2b0 2024-04-25T10:31:17.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:17.919+0000 7f1104042640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f10fc0cbff0 con 0x7f10fc0cc600 2024-04-25T10:31:17.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:17.919+0000 7f11025df640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10fc154ac0 0x7f10fc1e9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:17.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:17.919+0000 7f11015dd640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f10fc0cc600 msgr2=0x7f10fc1f6ad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-04-25T10:31:17.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:17.919+0000 7f11015dd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f10fc0cc600 0x7f10fc1f6ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:31:17.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:17.919+0000 7f1101dde640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10fc0cb2b0 0x7f10fc1f36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:17.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:17.919+0000 7f1101dde640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10fc0cb2b0 0x7f10fc1f36f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.106:49406/0 (socket says 172.21.15.106:49406) 2024-04-25T10:31:17.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:17.919+0000 7f1101dde640 1 -- 172.21.15.106:0/3886424348 learned_addr learned my addr 172.21.15.106:0/3886424348 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:31:17.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:17.920+0000 7f1101dde640 1 -- 172.21.15.106:0/3886424348 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f10fc0cc600 msgr2=0x7f10fc1f6ad0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:31:17.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:17.920+0000 7f1101dde640 1 --2- 172.21.15.106:0/3886424348 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f10fc0cc600 0x7f10fc1f6ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:17.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:17.920+0000 7f1101dde640 1 -- 172.21.15.106:0/3886424348 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10fc154ac0 msgr2=0x7f10fc1e9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:17.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:17.920+0000 7f1101dde640 1 --2- 172.21.15.106:0/3886424348 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10fc154ac0 0x7f10fc1e9790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:17.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:17.920+0000 7f1101dde640 1 -- 172.21.15.106:0/3886424348 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10fc0cdd60 con 0x7f10fc0cb2b0 2024-04-25T10:31:17.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:17.920+0000 7f1101dde640 1 --2- 172.21.15.106:0/3886424348 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10fc0cb2b0 0x7f10fc1f36f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f10f40011e0 tx=0x7f10f400cb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:17.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:17.920+0000 7f10fa7fc640 1 -- 172.21.15.106:0/3886424348 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f4017070 con 0x7f10fc0cb2b0 2024-04-25T10:31:17.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:17.920+0000 7f10fa7fc640 1 -- 172.21.15.106:0/3886424348 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10f4005ad0 con 0x7f10fc0cb2b0 2024-04-25T10:31:17.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:17.920+0000 7f1104042640 1 -- 172.21.15.106:0/3886424348 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10fc0d0d80 con 0x7f10fc0cb2b0 2024-04-25T10:31:17.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:17.920+0000 7f10fa7fc640 1 -- 172.21.15.106:0/3886424348 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f4007070 con 0x7f10fc0cb2b0 2024-04-25T10:31:17.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:17.920+0000 7f1104042640 1 -- 172.21.15.106:0/3886424348 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10fc1f6310 con 0x7f10fc0cb2b0 2024-04-25T10:31:17.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:17.922+0000 7f1104042640 1 -- 172.21.15.106:0/3886424348 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10fc0cafb0 con 0x7f10fc0cb2b0 2024-04-25T10:31:17.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:17.922+0000 7f10fa7fc640 1 -- 172.21.15.106:0/3886424348 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f10f401a070 con 0x7f10fc0cb2b0 2024-04-25T10:31:17.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:17.923+0000 7f10fa7fc640 1 --2- 172.21.15.106:0/3886424348 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f10e8074460 0x7f10e8076970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:17.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:17.923+0000 7f10fa7fc640 1 -- 172.21.15.106:0/3886424348 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f10f40944e0 con 0x7f10fc0cb2b0 2024-04-25T10:31:17.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:17.923+0000 7f11015dd640 1 --2- 172.21.15.106:0/3886424348 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f10e8074460 0x7f10e8076970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:17.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:17.923+0000 7f11015dd640 1 --2- 172.21.15.106:0/3886424348 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f10e8074460 0x7f10e8076970 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f10f0034000 tx=0x7f10f002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:17.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:17.928+0000 7f10fa7fc640 1 -- 172.21.15.106:0/3886424348 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f10fc0cafb0 con 0x7f10fc0cb2b0 2024-04-25T10:31:18.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:18.174+0000 7f1104042640 1 -- 172.21.15.106:0/3886424348 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f10fc06a3d0 con 0x7f10fc0cb2b0 2024-04-25T10:31:18.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:18.175+0000 7f10fa7fc640 1 -- 172.21.15.106:0/3886424348 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1568 (secure 0 0 0) 0x7f10f405e6c0 con 0x7f10fc0cb2b0 2024-04-25T10:31:18.177 INFO:teuthology.orchestra.run.smithi106.stdout:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":70,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"3ee77985-e8e4-41cf-81e9-7c774cf7891f","modified":"2024-04-25T10:29:33.446228Z","created":"2024-04-25T10:29:33.446228Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:3300","nonce":0},{"type":"v1","addr":"172.21.15.106:6789","nonce":0}]},"addr":"172.21.15.106:6789/0","public_addr":"172.21.15.106:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T10:31:18.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:18.177+0000 7f1104042640 1 -- 172.21.15.106:0/3886424348 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f10e8074460 msgr2=0x7f10e8076970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:18.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:18.177+0000 7f1104042640 1 --2- 172.21.15.106:0/3886424348 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f10e8074460 0x7f10e8076970 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f10f0034000 tx=0x7f10f002e000 comp rx=0 tx=0).stop 2024-04-25T10:31:18.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:18.177+0000 7f1104042640 1 -- 172.21.15.106:0/3886424348 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10fc0cb2b0 msgr2=0x7f10fc1f36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:18.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:18.177+0000 7f1104042640 1 --2- 172.21.15.106:0/3886424348 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10fc0cb2b0 0x7f10fc1f36f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f10f40011e0 tx=0x7f10f400cb90 comp rx=0 tx=0).stop 2024-04-25T10:31:18.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:18.177+0000 7f1104042640 1 -- 172.21.15.106:0/3886424348 shutdown_connections 2024-04-25T10:31:18.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:18.177+0000 7f1104042640 1 --2- 172.21.15.106:0/3886424348 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f10e8074460 0x7f10e8076970 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:18.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:18.177+0000 7f1104042640 1 --2- 172.21.15.106:0/3886424348 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10fc154ac0 0x7f10fc1e9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:18.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:18.177+0000 7f1104042640 1 --2- 172.21.15.106:0/3886424348 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f10fc0cc600 0x7f10fc1f6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:18.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:18.177+0000 7f1104042640 1 --2- 172.21.15.106:0/3886424348 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10fc0cb2b0 0x7f10fc1f36f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:18.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:18.177+0000 7f1104042640 1 -- 172.21.15.106:0/3886424348 >> 172.21.15.106:0/3886424348 conn(0x7f10fc0727d0 msgr2=0x7f10fc0c6980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:18.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:18.177+0000 7f1104042640 1 -- 172.21.15.106:0/3886424348 shutdown_connections 2024-04-25T10:31:18.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:18.177+0000 7f1104042640 1 -- 172.21.15.106:0/3886424348 wait complete. 2024-04-25T10:31:21.191 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T10:31:21.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:21.311+0000 7f82b0b38640 1 Processor -- start 2024-04-25T10:31:21.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:21.311+0000 7f82b0b38640 1 -- start start 2024-04-25T10:31:21.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:21.313+0000 7f82b0b38640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f82ac072870 0x7f82ac15a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:21.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:21.313+0000 7f82b0b38640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82ac0c5690 con 0x7f82ac15d9a0 2024-04-25T10:31:21.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:21.313+0000 7f82b0b38640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82ac0c7c40 con 0x7f82ac0c5280 2024-04-25T10:31:21.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:21.313+0000 7f82b0b38640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f82ac0cb590 con 0x7f82ac072870 2024-04-25T10:31:21.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:21.313+0000 7f82aad9d640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f82ac15d9a0 0x7f82ac153710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:34528/0 (socket says 172.21.15.106:34528) 2024-04-25T10:31:21.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:21.313+0000 7f82aad9d640 1 -- 172.21.15.106:0/2102745588 learned_addr learned my addr 172.21.15.106:0/2102745588 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:31:21.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:21.313+0000 7f82aa59c640 1 --2- 172.21.15.106:0/2102745588 >> v2:172.21.15.134:3300/0 conn(0x7f82ac072870 0x7f82ac15a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:21.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:21.313+0000 7f82aa59c640 1 -- 172.21.15.106:0/2102745588 >> v1:172.21.15.134:6789/0 conn(0x7f82ac0c5280 legacy=0x7f82ac0cae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:21.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:21.313+0000 7f82aa59c640 1 -- 172.21.15.106:0/2102745588 >> v1:172.21.15.106:6789/0 conn(0x7f82ac15d9a0 legacy=0x7f82ac153710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:21.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:21.314+0000 7f82aa59c640 1 -- 172.21.15.106:0/2102745588 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82ac0c7f20 con 0x7f82ac072870 2024-04-25T10:31:21.320 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:21.314+0000 7f82aa59c640 1 --2- 172.21.15.106:0/2102745588 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f82ac072870 0x7f82ac15a590 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f829c009d90 tx=0x7f829c02a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5fdb55eef2618d02 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:21.320 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:21.314+0000 7f82a959a640 1 -- 172.21.15.106:0/2102745588 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829c007740 con 0x7f82ac072870 2024-04-25T10:31:21.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:21.314+0000 7f82a959a640 1 -- 172.21.15.106:0/2102745588 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f829c0078e0 con 0x7f82ac072870 2024-04-25T10:31:21.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:21.314+0000 7f82a959a640 1 -- 172.21.15.106:0/2102745588 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829c007cb0 con 0x7f82ac072870 2024-04-25T10:31:21.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:21.315+0000 7f82b0b38640 1 -- 172.21.15.106:0/2102745588 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f82ac072870 msgr2=0x7f82ac15a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:21.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:21.315+0000 7f82b0b38640 1 --2- 172.21.15.106:0/2102745588 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f82ac072870 0x7f82ac15a590 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f829c009d90 tx=0x7f829c02a660 comp rx=0 tx=0).stop 2024-04-25T10:31:21.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:21.315+0000 7f82b0b38640 1 -- 172.21.15.106:0/2102745588 shutdown_connections 2024-04-25T10:31:21.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:21.315+0000 7f82b0b38640 1 --2- 172.21.15.106:0/2102745588 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f82ac072870 0x7f82ac15a590 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:21.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:21.315+0000 7f82b0b38640 1 -- 172.21.15.106:0/2102745588 >> 172.21.15.106:0/2102745588 conn(0x7f82ac06c9b0 msgr2=0x7f82ac06cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:21.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:21.315+0000 7f82b0b38640 1 -- 172.21.15.106:0/2102745588 shutdown_connections 2024-04-25T10:31:21.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:21.315+0000 7f82b0b38640 1 -- 172.21.15.106:0/2102745588 wait complete. 2024-04-25T10:31:21.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:21.317+0000 7f82b0b38640 1 Processor -- start 2024-04-25T10:31:21.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:21.317+0000 7f82b0b38640 1 -- start start 2024-04-25T10:31:21.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:21.318+0000 7f82b0b38640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f82ac072870 0x7f82ac1f1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:21.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:21.319+0000 7f82b0b38640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f82ac0c5280 0x7f82ac0cc950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:21.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:21.319+0000 7f82b0b38640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82ac15d9a0 0x7f82ac1f4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:21.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:21.319+0000 7f82b0b38640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f82ac159d90 con 0x7f82ac15d9a0 2024-04-25T10:31:21.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:21.319+0000 7f82b0b38640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f82ac159ab0 con 0x7f82ac072870 2024-04-25T10:31:21.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:21.319+0000 7f82b0b38640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f82ac15a070 con 0x7f82ac0c5280 2024-04-25T10:31:21.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:21.319+0000 7f82a9d9b640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f82ac0c5280 msgr2=0x7f82ac0cc950 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-04-25T10:31:21.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:21.319+0000 7f82a9d9b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f82ac0c5280 0x7f82ac0cc950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:31:21.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:21.319+0000 7f82aad9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82ac15d9a0 0x7f82ac1f4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:21.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:21.319+0000 7f82aad9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82ac15d9a0 0x7f82ac1f4db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35668/0 (socket says 172.21.15.106:35668) 2024-04-25T10:31:21.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:21.319+0000 7f82aad9d640 1 -- 172.21.15.106:0/532809222 learned_addr learned my addr 172.21.15.106:0/532809222 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:31:21.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:21.319+0000 7f82aa59c640 1 --2- 172.21.15.106:0/532809222 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f82ac072870 0x7f82ac1f1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:21.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:21.319+0000 7f82aad9d640 1 -- 172.21.15.106:0/532809222 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f82ac0c5280 msgr2=0x7f82ac0cc950 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:31:21.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:21.319+0000 7f82aad9d640 1 --2- 172.21.15.106:0/532809222 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f82ac0c5280 0x7f82ac0cc950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:21.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:21.319+0000 7f82aad9d640 1 -- 172.21.15.106:0/532809222 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f82ac072870 msgr2=0x7f82ac1f1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:21.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:21.319+0000 7f82aad9d640 1 --2- 172.21.15.106:0/532809222 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f82ac072870 0x7f82ac1f1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:21.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:21.319+0000 7f82aad9d640 1 -- 172.21.15.106:0/532809222 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82ac1f52f0 con 0x7f82ac15d9a0 2024-04-25T10:31:21.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:21.319+0000 7f82aad9d640 1 --2- 172.21.15.106:0/532809222 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82ac15d9a0 0x7f82ac1f4db0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f82a400c3a0 tx=0x7f82a400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:21.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:21.319+0000 7f829affd640 1 -- 172.21.15.106:0/532809222 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a4017070 con 0x7f82ac15d9a0 2024-04-25T10:31:21.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:21.319+0000 7f829affd640 1 -- 172.21.15.106:0/532809222 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82a4005ce0 con 0x7f82ac15d9a0 2024-04-25T10:31:21.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:21.319+0000 7f82b0b38640 1 -- 172.21.15.106:0/532809222 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82ac1f45f0 con 0x7f82ac15d9a0 2024-04-25T10:31:21.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:21.319+0000 7f829affd640 1 -- 172.21.15.106:0/532809222 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a40072d0 con 0x7f82ac15d9a0 2024-04-25T10:31:21.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:21.319+0000 7f82b0b38640 1 -- 172.21.15.106:0/532809222 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82ac0cbb80 con 0x7f82ac15d9a0 2024-04-25T10:31:21.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:21.320+0000 7f82b0b38640 1 -- 172.21.15.106:0/532809222 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82ac06a3d0 con 0x7f82ac15d9a0 2024-04-25T10:31:21.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:21.321+0000 7f829affd640 1 -- 172.21.15.106:0/532809222 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f82a401a070 con 0x7f82ac15d9a0 2024-04-25T10:31:21.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:21.321+0000 7f829affd640 1 --2- 172.21.15.106:0/532809222 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f828c074460 0x7f828c076970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:21.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:21.322+0000 7f829affd640 1 -- 172.21.15.106:0/532809222 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f82a4094460 con 0x7f82ac15d9a0 2024-04-25T10:31:21.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:21.322+0000 7f82aa59c640 1 --2- 172.21.15.106:0/532809222 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f828c074460 0x7f828c076970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:21.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:21.322+0000 7f82aa59c640 1 --2- 172.21.15.106:0/532809222 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f828c074460 0x7f828c076970 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f829c034000 tx=0x7f829c02f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:21.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:21.326+0000 7f829affd640 1 -- 172.21.15.106:0/532809222 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f82ac0c5690 con 0x7f82ac15d9a0 2024-04-25T10:31:21.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:21.569+0000 7f82b0b38640 1 -- 172.21.15.106:0/532809222 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f82ac0743c0 con 0x7f82ac15d9a0 2024-04-25T10:31:21.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:21.570+0000 7f829affd640 1 -- 172.21.15.106:0/532809222 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1568 (secure 0 0 0) 0x7f82a405e5b0 con 0x7f82ac15d9a0 2024-04-25T10:31:21.572 INFO:teuthology.orchestra.run.smithi106.stdout:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":73,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"3ee77985-e8e4-41cf-81e9-7c774cf7891f","modified":"2024-04-25T10:29:33.446228Z","created":"2024-04-25T10:29:33.446228Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:3300","nonce":0},{"type":"v1","addr":"172.21.15.106:6789","nonce":0}]},"addr":"172.21.15.106:6789/0","public_addr":"172.21.15.106:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T10:31:21.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:21.571+0000 7f82b0b38640 1 -- 172.21.15.106:0/532809222 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f828c074460 msgr2=0x7f828c076970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:21.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:21.571+0000 7f82b0b38640 1 --2- 172.21.15.106:0/532809222 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f828c074460 0x7f828c076970 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f829c034000 tx=0x7f829c02f000 comp rx=0 tx=0).stop 2024-04-25T10:31:21.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:21.571+0000 7f82b0b38640 1 -- 172.21.15.106:0/532809222 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82ac15d9a0 msgr2=0x7f82ac1f4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:21.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:21.571+0000 7f82b0b38640 1 --2- 172.21.15.106:0/532809222 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82ac15d9a0 0x7f82ac1f4db0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f82a400c3a0 tx=0x7f82a400c860 comp rx=0 tx=0).stop 2024-04-25T10:31:21.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:21.572+0000 7f82b0b38640 1 -- 172.21.15.106:0/532809222 shutdown_connections 2024-04-25T10:31:21.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:21.572+0000 7f82b0b38640 1 --2- 172.21.15.106:0/532809222 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f828c074460 0x7f828c076970 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:21.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:21.572+0000 7f82b0b38640 1 --2- 172.21.15.106:0/532809222 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82ac15d9a0 0x7f82ac1f4db0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:21.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:21.572+0000 7f82b0b38640 1 --2- 172.21.15.106:0/532809222 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f82ac0c5280 0x7f82ac0cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:21.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:21.572+0000 7f82b0b38640 1 --2- 172.21.15.106:0/532809222 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f82ac072870 0x7f82ac1f1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:21.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:21.572+0000 7f82b0b38640 1 -- 172.21.15.106:0/532809222 >> 172.21.15.106:0/532809222 conn(0x7f82ac06c9b0 msgr2=0x7f82ac150b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:21.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:21.572+0000 7f82b0b38640 1 -- 172.21.15.106:0/532809222 shutdown_connections 2024-04-25T10:31:21.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:21.572+0000 7f82b0b38640 1 -- 172.21.15.106:0/532809222 wait complete. 2024-04-25T10:31:24.586 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T10:31:24.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:24.706+0000 7f38772e0640 1 Processor -- start 2024-04-25T10:31:24.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:24.706+0000 7f38772e0640 1 -- start start 2024-04-25T10:31:24.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:24.708+0000 7f38772e0640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3870153180 0x7f387006ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:24.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:24.708+0000 7f38772e0640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f3870153590 0x7f3870070510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:24.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:24.708+0000 7f38772e0640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3870074790 con 0x7f38700c9c00 2024-04-25T10:31:24.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:24.708+0000 7f38772e0640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f38700741d0 con 0x7f3870153180 2024-04-25T10:31:24.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:24.708+0000 7f38772e0640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f38700744b0 con 0x7f3870153590 2024-04-25T10:31:24.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:24.708+0000 7f387487b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3870153180 0x7f387006ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:24.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:24.708+0000 7f387487b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3870153180 0x7f387006ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47832/0 (socket says 172.21.15.106:47832) 2024-04-25T10:31:24.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:24.708+0000 7f387487b640 1 -- 172.21.15.106:0/4206744445 learned_addr learned my addr 172.21.15.106:0/4206744445 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:31:24.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:24.708+0000 7f387487b640 1 -- 172.21.15.106:0/4206744445 >> v2:172.21.15.106:3300/0 conn(0x7f3870153180 msgr2=0x7f387006ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T10:31:24.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:24.708+0000 7f387487b640 1 -- 172.21.15.106:0/4206744445 >> v2:172.21.15.106:3300/0 conn(0x7f3870153180 msgr2=0x7f387006ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:31:24.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:24.708+0000 7f387487b640 1 --2- 172.21.15.106:0/4206744445 >> v2:172.21.15.106:3300/0 conn(0x7f3870153180 0x7f387006ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:31:24.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:24.708+0000 7f387487b640 1 --2- 172.21.15.106:0/4206744445 >> v2:172.21.15.106:3300/0 conn(0x7f3870153180 0x7f387006ffd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:31:24.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:24.708+0000 7f387587d640 1 --2- 172.21.15.106:0/4206744445 >> v2:172.21.15.134:3300/0 conn(0x7f3870153590 0x7f3870070510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:24.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:24.708+0000 7f387587d640 1 -- 172.21.15.106:0/4206744445 >> v2:172.21.15.134:3300/0 conn(0x7f3870153590 msgr2=0x7f3870070510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T10:31:24.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:24.708+0000 7f387587d640 1 -- 172.21.15.106:0/4206744445 >> v2:172.21.15.134:3300/0 conn(0x7f3870153590 msgr2=0x7f3870070510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:31:24.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:24.708+0000 7f387587d640 1 --2- 172.21.15.106:0/4206744445 >> v2:172.21.15.134:3300/0 conn(0x7f3870153590 0x7f3870070510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:31:24.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:24.708+0000 7f387587d640 1 --2- 172.21.15.106:0/4206744445 >> v2:172.21.15.134:3300/0 conn(0x7f3870153590 0x7f3870070510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:31:24.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:24.909+0000 7f387487b640 1 --2- 172.21.15.106:0/4206744445 >> v2:172.21.15.106:3300/0 conn(0x7f3870153180 0x7f387006ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:24.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:24.909+0000 7f387487b640 1 -- 172.21.15.106:0/4206744445 >> v2:172.21.15.106:3300/0 conn(0x7f3870153180 msgr2=0x7f387006ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T10:31:24.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:24.909+0000 7f387487b640 1 -- 172.21.15.106:0/4206744445 >> v2:172.21.15.106:3300/0 conn(0x7f3870153180 msgr2=0x7f387006ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:31:24.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:24.909+0000 7f387487b640 1 --2- 172.21.15.106:0/4206744445 >> v2:172.21.15.106:3300/0 conn(0x7f3870153180 0x7f387006ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:31:24.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:24.909+0000 7f387487b640 1 --2- 172.21.15.106:0/4206744445 >> v2:172.21.15.106:3300/0 conn(0x7f3870153180 0x7f387006ffd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T10:31:24.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:24.909+0000 7f387587d640 1 --2- 172.21.15.106:0/4206744445 >> v2:172.21.15.134:3300/0 conn(0x7f3870153590 0x7f3870070510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:24.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:24.909+0000 7f387587d640 1 -- 172.21.15.106:0/4206744445 >> v2:172.21.15.134:3300/0 conn(0x7f3870153590 msgr2=0x7f3870070510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T10:31:24.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:24.909+0000 7f387587d640 1 -- 172.21.15.106:0/4206744445 >> v2:172.21.15.134:3300/0 conn(0x7f3870153590 msgr2=0x7f3870070510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:31:24.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:24.909+0000 7f387587d640 1 --2- 172.21.15.106:0/4206744445 >> v2:172.21.15.134:3300/0 conn(0x7f3870153590 0x7f3870070510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:31:24.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:24.909+0000 7f387587d640 1 --2- 172.21.15.106:0/4206744445 >> v2:172.21.15.134:3300/0 conn(0x7f3870153590 0x7f3870070510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T10:31:25.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:25.310+0000 7f387487b640 1 --2- 172.21.15.106:0/4206744445 >> v2:172.21.15.106:3300/0 conn(0x7f3870153180 0x7f387006ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:25.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:25.310+0000 7f387487b640 1 -- 172.21.15.106:0/4206744445 >> v2:172.21.15.106:3300/0 conn(0x7f3870153180 msgr2=0x7f387006ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T10:31:25.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:25.310+0000 7f387487b640 1 -- 172.21.15.106:0/4206744445 >> v2:172.21.15.106:3300/0 conn(0x7f3870153180 msgr2=0x7f387006ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:31:25.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:25.310+0000 7f387487b640 1 --2- 172.21.15.106:0/4206744445 >> v2:172.21.15.106:3300/0 conn(0x7f3870153180 0x7f387006ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:31:25.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:25.310+0000 7f387487b640 1 --2- 172.21.15.106:0/4206744445 >> v2:172.21.15.106:3300/0 conn(0x7f3870153180 0x7f387006ffd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T10:31:25.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:25.310+0000 7f387587d640 1 --2- 172.21.15.106:0/4206744445 >> v2:172.21.15.134:3300/0 conn(0x7f3870153590 0x7f3870070510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:25.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:25.310+0000 7f387587d640 1 -- 172.21.15.106:0/4206744445 >> v2:172.21.15.134:3300/0 conn(0x7f3870153590 msgr2=0x7f3870070510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T10:31:25.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:25.310+0000 7f387587d640 1 -- 172.21.15.106:0/4206744445 >> v2:172.21.15.134:3300/0 conn(0x7f3870153590 msgr2=0x7f3870070510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:31:25.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:25.310+0000 7f387587d640 1 --2- 172.21.15.106:0/4206744445 >> v2:172.21.15.134:3300/0 conn(0x7f3870153590 0x7f3870070510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:31:25.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:25.310+0000 7f387587d640 1 --2- 172.21.15.106:0/4206744445 >> v2:172.21.15.134:3300/0 conn(0x7f3870153590 0x7f3870070510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T10:31:26.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:26.111+0000 7f387487b640 1 --2- 172.21.15.106:0/4206744445 >> v2:172.21.15.106:3300/0 conn(0x7f3870153180 0x7f387006ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:26.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:26.111+0000 7f387487b640 1 -- 172.21.15.106:0/4206744445 >> v2:172.21.15.106:3300/0 conn(0x7f3870153180 msgr2=0x7f387006ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T10:31:26.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:26.111+0000 7f387487b640 1 -- 172.21.15.106:0/4206744445 >> v2:172.21.15.106:3300/0 conn(0x7f3870153180 msgr2=0x7f387006ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:31:26.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:26.111+0000 7f387487b640 1 --2- 172.21.15.106:0/4206744445 >> v2:172.21.15.106:3300/0 conn(0x7f3870153180 0x7f387006ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:31:26.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:26.111+0000 7f387487b640 1 --2- 172.21.15.106:0/4206744445 >> v2:172.21.15.106:3300/0 conn(0x7f3870153180 0x7f387006ffd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T10:31:26.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:26.111+0000 7f387587d640 1 --2- 172.21.15.106:0/4206744445 >> v2:172.21.15.134:3300/0 conn(0x7f3870153590 0x7f3870070510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:26.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:26.111+0000 7f387587d640 1 -- 172.21.15.106:0/4206744445 >> v2:172.21.15.134:3300/0 conn(0x7f3870153590 msgr2=0x7f3870070510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T10:31:26.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:26.111+0000 7f387587d640 1 -- 172.21.15.106:0/4206744445 >> v2:172.21.15.134:3300/0 conn(0x7f3870153590 msgr2=0x7f3870070510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:31:26.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:26.111+0000 7f387587d640 1 --2- 172.21.15.106:0/4206744445 >> v2:172.21.15.134:3300/0 conn(0x7f3870153590 0x7f3870070510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:31:26.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:26.111+0000 7f387587d640 1 --2- 172.21.15.106:0/4206744445 >> v2:172.21.15.134:3300/0 conn(0x7f3870153590 0x7f3870070510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T10:31:27.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:27.708+0000 7f386effd640 1 -- 172.21.15.106:0/4206744445 >> v2:172.21.15.134:3300/0 conn(0x7f3870153590 msgr2=0x7f3870070510 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:31:27.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:27.708+0000 7f386effd640 1 --2- 172.21.15.106:0/4206744445 >> v2:172.21.15.134:3300/0 conn(0x7f3870153590 0x7f3870070510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:27.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:27.708+0000 7f386effd640 1 -- 172.21.15.106:0/4206744445 >> v2:172.21.15.106:3300/0 conn(0x7f3870153180 msgr2=0x7f387006ffd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:31:27.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:27.708+0000 7f386effd640 1 --2- 172.21.15.106:0/4206744445 >> v2:172.21.15.106:3300/0 conn(0x7f3870153180 0x7f387006ffd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:27.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:27.708+0000 7f386effd640 1 -- 172.21.15.106:0/4206744445 >> v1:172.21.15.134:6789/0 conn(0x7f38700c9c00 legacy=0x7f38700c9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:27.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:27.708+0000 7f386effd640 1 --2- 172.21.15.106:0/4206744445 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f385c0095b0 0x7f385c00baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:27.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:27.708+0000 7f386effd640 1 -- 172.21.15.106:0/4206744445 --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38700c5320 con 0x7f385c005880 2024-04-25T10:31:27.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:27.708+0000 7f386effd640 1 -- 172.21.15.106:0/4206744445 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f38700741d0 con 0x7f385c0095b0 2024-04-25T10:31:27.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:27.708+0000 7f387487b640 1 -- 172.21.15.106:0/4206744445 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f385c0095b0 msgr2=0x7f385c00baa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-04-25T10:31:27.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:27.708+0000 7f387487b640 1 --2- 172.21.15.106:0/4206744445 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f385c0095b0 0x7f385c00baa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:31:27.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:27.909+0000 7f387487b640 1 -- 172.21.15.106:0/4206744445 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f385c0095b0 msgr2=0x7f385c00baa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-04-25T10:31:27.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:27.909+0000 7f387487b640 1 --2- 172.21.15.106:0/4206744445 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f385c0095b0 0x7f385c00baa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T10:31:28.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.310+0000 7f387487b640 1 -- 172.21.15.106:0/4206744445 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f385c0095b0 msgr2=0x7f385c00baa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-04-25T10:31:28.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.310+0000 7f387487b640 1 --2- 172.21.15.106:0/4206744445 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f385c0095b0 0x7f385c00baa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T10:31:28.655 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.652+0000 7f386ffff640 1 -- 172.21.15.106:0/4206744445 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1471012811 0 0) 0x7f38700c5320 con 0x7f385c005880 2024-04-25T10:31:28.655 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.652+0000 7f386ffff640 1 -- 172.21.15.106:0/4206744445 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3858003520 con 0x7f385c005880 2024-04-25T10:31:28.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.654+0000 7f386ffff640 1 -- 172.21.15.106:0/4206744445 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4063812916 0 0) 0x7f3858003520 con 0x7f385c005880 2024-04-25T10:31:28.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.654+0000 7f386ffff640 1 -- 172.21.15.106:0/4206744445 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f385c0095b0 msgr2=0x7f385c00baa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:31:28.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.654+0000 7f386ffff640 1 --2- 172.21.15.106:0/4206744445 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f385c0095b0 0x7f385c00baa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:28.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.654+0000 7f386ffff640 1 -- 172.21.15.106:0/4206744445 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f385c00d2c0 con 0x7f385c005880 2024-04-25T10:31:28.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.654+0000 7f386ffff640 1 -- 172.21.15.106:0/4206744445 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3776459350 0 0) 0x7f38680016b0 con 0x7f385c005880 2024-04-25T10:31:28.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.654+0000 7f386ffff640 1 -- 172.21.15.106:0/4206744445 >> v1:172.21.15.106:6789/0 conn(0x7f385c005880 legacy=0x7f385c007d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:28.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.654+0000 7f387587d640 1 -- 172.21.15.106:0/4206744445 reap_dead start 2024-04-25T10:31:28.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.654+0000 7f386ffff640 1 --2- 172.21.15.106:0/4206744445 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38580039b0 0x7f3858003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:28.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.654+0000 7f386ffff640 1 --2- 172.21.15.106:0/4206744445 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3858004800 0x7f3858004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:28.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.654+0000 7f386ffff640 1 --2- 172.21.15.106:0/4206744445 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38580053a0 0x7f385800ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:28.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.654+0000 7f386ffff640 1 -- 172.21.15.106:0/4206744445 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f38700741d0 con 0x7f38580039b0 2024-04-25T10:31:28.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.654+0000 7f386ffff640 1 -- 172.21.15.106:0/4206744445 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f385800c300 con 0x7f38580053a0 2024-04-25T10:31:28.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.654+0000 7f386ffff640 1 -- 172.21.15.106:0/4206744445 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f385800c500 con 0x7f3858004800 2024-04-25T10:31:28.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.654+0000 7f387507c640 1 --2- 172.21.15.106:0/4206744445 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38580039b0 0x7f3858003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:28.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.654+0000 7f387487b640 1 -- 172.21.15.106:0/4206744445 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3858004800 msgr2=0x7f3858004c10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-04-25T10:31:28.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.654+0000 7f387487b640 1 --2- 172.21.15.106:0/4206744445 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3858004800 0x7f3858004c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:31:28.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.654+0000 7f387587d640 1 --2- 172.21.15.106:0/4206744445 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38580053a0 0x7f385800ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:28.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.654+0000 7f387507c640 1 -- 172.21.15.106:0/4206744445 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3858004800 msgr2=0x7f3858004c10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:31:28.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.654+0000 7f387507c640 1 --2- 172.21.15.106:0/4206744445 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3858004800 0x7f3858004c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:28.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.654+0000 7f387507c640 1 -- 172.21.15.106:0/4206744445 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38580053a0 msgr2=0x7f385800ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:28.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.654+0000 7f387507c640 1 --2- 172.21.15.106:0/4206744445 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38580053a0 0x7f385800ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:28.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.654+0000 7f387507c640 1 -- 172.21.15.106:0/4206744445 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3858003520 con 0x7f38580039b0 2024-04-25T10:31:28.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.654+0000 7f387587d640 1 --2- 172.21.15.106:0/4206744445 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38580053a0 0x7f385800ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:31:28.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.655+0000 7f387507c640 1 --2- 172.21.15.106:0/4206744445 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38580039b0 0x7f3858003e00 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3868007f90 tx=0x7f3858005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=106598e83a9d3d88 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:28.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.655+0000 7f386ffff640 1 -- 172.21.15.106:0/4206744445 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3868036070 con 0x7f38580039b0 2024-04-25T10:31:28.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.655+0000 7f386ffff640 1 -- 172.21.15.106:0/4206744445 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3868031620 con 0x7f38580039b0 2024-04-25T10:31:28.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.655+0000 7f386ffff640 1 -- 172.21.15.106:0/4206744445 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3868007bb0 con 0x7f38580039b0 2024-04-25T10:31:28.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.656+0000 7f38772e0640 1 -- 172.21.15.106:0/4206744445 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38580039b0 msgr2=0x7f3858003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:28.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.656+0000 7f38772e0640 1 --2- 172.21.15.106:0/4206744445 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38580039b0 0x7f3858003e00 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3868007f90 tx=0x7f3858005830 comp rx=0 tx=0).stop 2024-04-25T10:31:28.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.656+0000 7f38772e0640 1 -- 172.21.15.106:0/4206744445 shutdown_connections 2024-04-25T10:31:28.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.656+0000 7f38772e0640 1 --2- 172.21.15.106:0/4206744445 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38580053a0 0x7f385800ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:28.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.656+0000 7f38772e0640 1 --2- 172.21.15.106:0/4206744445 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3858004800 0x7f3858004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:28.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.656+0000 7f38772e0640 1 --2- 172.21.15.106:0/4206744445 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38580039b0 0x7f3858003e00 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:28.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.656+0000 7f38772e0640 1 -- 172.21.15.106:0/4206744445 >> 172.21.15.106:0/4206744445 conn(0x7f387006c9b0 msgr2=0x7f387006cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:28.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.657+0000 7f38772e0640 1 -- 172.21.15.106:0/4206744445 shutdown_connections 2024-04-25T10:31:28.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.657+0000 7f38772e0640 1 -- 172.21.15.106:0/4206744445 wait complete. 2024-04-25T10:31:28.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.659+0000 7f38772e0640 1 Processor -- start 2024-04-25T10:31:28.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.659+0000 7f38772e0640 1 -- start start 2024-04-25T10:31:28.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.661+0000 7f38772e0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38580039b0 0x7f3870148680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:28.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.661+0000 7f38772e0640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3858004800 0x7f38701578d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:28.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.661+0000 7f38772e0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38580053a0 0x7f38701425e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:28.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.661+0000 7f38772e0640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f38700ca5e0 con 0x7f38580039b0 2024-04-25T10:31:28.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.661+0000 7f38772e0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f38700c58e0 con 0x7f38580053a0 2024-04-25T10:31:28.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.661+0000 7f38772e0640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f38700c9190 con 0x7f3858004800 2024-04-25T10:31:28.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.661+0000 7f387507c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38580039b0 0x7f3870148680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:28.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.661+0000 7f387487b640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3858004800 msgr2=0x7f38701578d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-04-25T10:31:28.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.661+0000 7f387487b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3858004800 0x7f38701578d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:31:28.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.661+0000 7f387507c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38580039b0 0x7f3870148680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47886/0 (socket says 172.21.15.106:47886) 2024-04-25T10:31:28.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.661+0000 7f387507c640 1 -- 172.21.15.106:0/2012972248 learned_addr learned my addr 172.21.15.106:0/2012972248 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:31:28.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.661+0000 7f387587d640 1 --2- 172.21.15.106:0/2012972248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38580053a0 0x7f38701425e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:28.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.661+0000 7f387507c640 1 -- 172.21.15.106:0/2012972248 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3858004800 msgr2=0x7f38701578d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:31:28.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.661+0000 7f387507c640 1 --2- 172.21.15.106:0/2012972248 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3858004800 0x7f38701578d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:28.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.661+0000 7f387507c640 1 -- 172.21.15.106:0/2012972248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38580053a0 msgr2=0x7f38701425e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:28.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.661+0000 7f387507c640 1 --2- 172.21.15.106:0/2012972248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38580053a0 0x7f38701425e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:28.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.661+0000 7f387507c640 1 -- 172.21.15.106:0/2012972248 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3870148bc0 con 0x7f38580039b0 2024-04-25T10:31:28.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.661+0000 7f387507c640 1 --2- 172.21.15.106:0/2012972248 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38580039b0 0x7f3870148680 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f386800c1d0 tx=0x7f38680019a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:28.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.662+0000 7f386dffb640 1 -- 172.21.15.106:0/2012972248 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3868041020 con 0x7f38580039b0 2024-04-25T10:31:28.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.662+0000 7f386dffb640 1 -- 172.21.15.106:0/2012972248 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3868000ed0 con 0x7f38580039b0 2024-04-25T10:31:28.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.662+0000 7f38772e0640 1 -- 172.21.15.106:0/2012972248 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38701f1de0 con 0x7f38580039b0 2024-04-25T10:31:28.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.662+0000 7f386dffb640 1 -- 172.21.15.106:0/2012972248 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3868036040 con 0x7f38580039b0 2024-04-25T10:31:28.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.662+0000 7f38772e0640 1 -- 172.21.15.106:0/2012972248 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38701f23a0 con 0x7f38580039b0 2024-04-25T10:31:28.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.663+0000 7f38772e0640 1 -- 172.21.15.106:0/2012972248 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f387006a3d0 con 0x7f38580039b0 2024-04-25T10:31:28.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.664+0000 7f386dffb640 1 -- 172.21.15.106:0/2012972248 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3868044020 con 0x7f38580039b0 2024-04-25T10:31:28.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.664+0000 7f386dffb640 1 --2- 172.21.15.106:0/2012972248 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f385c076ff0 0x7f385c079500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:28.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.664+0000 7f386dffb640 1 -- 172.21.15.106:0/2012972248 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f38680b5550 con 0x7f38580039b0 2024-04-25T10:31:28.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.665+0000 7f387487b640 1 --2- 172.21.15.106:0/2012972248 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f385c076ff0 0x7f385c079500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:28.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.665+0000 7f387487b640 1 --2- 172.21.15.106:0/2012972248 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f385c076ff0 0x7f385c079500 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3870070ef0 tx=0x7f3864008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:28.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.669+0000 7f386dffb640 1 -- 172.21.15.106:0/2012972248 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f386804b030 con 0x7f38580039b0 2024-04-25T10:31:28.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.920+0000 7f38772e0640 1 -- 172.21.15.106:0/2012972248 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f3870072e40 con 0x7f38580039b0 2024-04-25T10:31:28.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.920+0000 7f386dffb640 1 -- 172.21.15.106:0/2012972248 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1561 (secure 0 0 0) 0x7f38680b3e20 con 0x7f38580039b0 2024-04-25T10:31:28.922 INFO:teuthology.orchestra.run.smithi106.stdout:{"election_epoch":8,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"3ee77985-e8e4-41cf-81e9-7c774cf7891f","modified":"2024-04-25T10:29:33.446228Z","created":"2024-04-25T10:29:33.446228Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:3300","nonce":0},{"type":"v1","addr":"172.21.15.106:6789","nonce":0}]},"addr":"172.21.15.106:6789/0","public_addr":"172.21.15.106:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T10:31:28.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.922+0000 7f38772e0640 1 -- 172.21.15.106:0/2012972248 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f385c076ff0 msgr2=0x7f385c079500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:28.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.922+0000 7f38772e0640 1 --2- 172.21.15.106:0/2012972248 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f385c076ff0 0x7f385c079500 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3870070ef0 tx=0x7f3864008040 comp rx=0 tx=0).stop 2024-04-25T10:31:28.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.922+0000 7f38772e0640 1 -- 172.21.15.106:0/2012972248 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38580039b0 msgr2=0x7f3870148680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:28.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.922+0000 7f38772e0640 1 --2- 172.21.15.106:0/2012972248 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38580039b0 0x7f3870148680 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f386800c1d0 tx=0x7f38680019a0 comp rx=0 tx=0).stop 2024-04-25T10:31:28.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.922+0000 7f38772e0640 1 -- 172.21.15.106:0/2012972248 shutdown_connections 2024-04-25T10:31:28.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.922+0000 7f38772e0640 1 --2- 172.21.15.106:0/2012972248 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f385c076ff0 0x7f385c079500 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:28.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.922+0000 7f38772e0640 1 --2- 172.21.15.106:0/2012972248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38580053a0 0x7f38701425e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:28.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.922+0000 7f38772e0640 1 --2- 172.21.15.106:0/2012972248 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3858004800 0x7f38701578d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:28.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.922+0000 7f38772e0640 1 --2- 172.21.15.106:0/2012972248 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38580039b0 0x7f3870148680 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:28.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.922+0000 7f38772e0640 1 -- 172.21.15.106:0/2012972248 >> 172.21.15.106:0/2012972248 conn(0x7f387006c9b0 msgr2=0x7f3870073570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:28.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.922+0000 7f38772e0640 1 -- 172.21.15.106:0/2012972248 shutdown_connections 2024-04-25T10:31:28.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:28.922+0000 7f38772e0640 1 -- 172.21.15.106:0/2012972248 wait complete. 2024-04-25T10:31:28.936 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-04-25T10:31:28.936 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-04-25T10:31:28.936 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-25T10:31:29.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.042+0000 7f348df4d640 1 Processor -- start 2024-04-25T10:31:29.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.042+0000 7f348df4d640 1 -- start start 2024-04-25T10:31:29.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.043+0000 7f348df4d640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f34880c8630 0x7f34880c8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:29.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.043+0000 7f348df4d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f348815e5a0 0x7f348815e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:29.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.043+0000 7f348df4d640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34880ca6e0 con 0x7f3488073980 2024-04-25T10:31:29.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.043+0000 7f348df4d640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f348815e110 con 0x7f34880c8630 2024-04-25T10:31:29.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.043+0000 7f348df4d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f3488071310 con 0x7f348815e5a0 2024-04-25T10:31:29.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.043+0000 7f34877fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f3488073980 0x7f3488154c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:39188/0 (socket says 172.21.15.106:39188) 2024-04-25T10:31:29.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.043+0000 7f34877fe640 1 -- 172.21.15.106:0/2005340543 learned_addr learned my addr 172.21.15.106:0/2005340543 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:31:29.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.043+0000 7f3487fff640 1 -- 172.21.15.106:0/2005340543 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f348815e5a0 msgr2=0x7f348815e9b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-04-25T10:31:29.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.043+0000 7f3487fff640 1 --2- 172.21.15.106:0/2005340543 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f348815e5a0 0x7f348815e9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:31:29.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.044+0000 7f3486ffd640 1 --2- 172.21.15.106:0/2005340543 >> v2:172.21.15.134:3300/0 conn(0x7f34880c8630 0x7f34880c8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:29.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.044+0000 7f34867fc640 1 -- 172.21.15.106:0/2005340543 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3029465661 0 0) 0x7f34880ca6e0 con 0x7f3488073980 2024-04-25T10:31:29.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.044+0000 7f34867fc640 1 -- 172.21.15.106:0/2005340543 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3470003520 con 0x7f3488073980 2024-04-25T10:31:29.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.044+0000 7f34867fc640 1 -- 172.21.15.106:0/2005340543 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1569445 0 0) 0x7f3470003520 con 0x7f3488073980 2024-04-25T10:31:29.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.044+0000 7f34867fc640 1 -- 172.21.15.106:0/2005340543 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f348815e5a0 msgr2=0x7f348815e9b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:31:29.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.044+0000 7f34867fc640 1 --2- 172.21.15.106:0/2005340543 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f348815e5a0 0x7f348815e9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:29.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.044+0000 7f34867fc640 1 -- 172.21.15.106:0/2005340543 >> v2:172.21.15.134:3300/0 conn(0x7f34880c8630 msgr2=0x7f34880c8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:29.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.044+0000 7f34867fc640 1 --2- 172.21.15.106:0/2005340543 >> v2:172.21.15.134:3300/0 conn(0x7f34880c8630 0x7f34880c8a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:29.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.044+0000 7f34867fc640 1 -- 172.21.15.106:0/2005340543 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34880715f0 con 0x7f3488073980 2024-04-25T10:31:29.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.044+0000 7f34867fc640 1 -- 172.21.15.106:0/2005340543 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3776459350 0 0) 0x7f3478001670 con 0x7f3488073980 2024-04-25T10:31:29.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.044+0000 7f34867fc640 1 -- 172.21.15.106:0/2005340543 >> v1:172.21.15.106:6789/0 conn(0x7f3488073980 legacy=0x7f3488154c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:29.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.044+0000 7f34867fc640 1 --2- 172.21.15.106:0/2005340543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34700039b0 0x7f3470003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:29.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.044+0000 7f34867fc640 1 --2- 172.21.15.106:0/2005340543 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3470004800 0x7f3470004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:29.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.044+0000 7f34867fc640 1 --2- 172.21.15.106:0/2005340543 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f348815e5a0 0x7f3470005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:29.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.044+0000 7f34867fc640 1 -- 172.21.15.106:0/2005340543 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f348815e110 con 0x7f3470004800 2024-04-25T10:31:29.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.044+0000 7f34867fc640 1 -- 172.21.15.106:0/2005340543 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3488071310 con 0x7f34700039b0 2024-04-25T10:31:29.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.044+0000 7f34867fc640 1 -- 172.21.15.106:0/2005340543 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f347000bbe0 con 0x7f348815e5a0 2024-04-25T10:31:29.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.044+0000 7f3487fff640 1 -- 172.21.15.106:0/2005340543 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f348815e5a0 msgr2=0x7f3470005190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-04-25T10:31:29.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.044+0000 7f3486ffd640 1 --2- 172.21.15.106:0/2005340543 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3470004800 0x7f3470004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:29.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.044+0000 7f3487fff640 1 --2- 172.21.15.106:0/2005340543 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f348815e5a0 0x7f3470005190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:31:29.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.044+0000 7f34877fe640 1 --2- 172.21.15.106:0/2005340543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34700039b0 0x7f3470003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:29.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.044+0000 7f3486ffd640 1 -- 172.21.15.106:0/2005340543 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f348815e5a0 msgr2=0x7f3470005190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:31:29.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.044+0000 7f3486ffd640 1 --2- 172.21.15.106:0/2005340543 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f348815e5a0 0x7f3470005190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:29.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.044+0000 7f3486ffd640 1 -- 172.21.15.106:0/2005340543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34700039b0 msgr2=0x7f3470003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:29.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.044+0000 7f3486ffd640 1 --2- 172.21.15.106:0/2005340543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34700039b0 0x7f3470003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:29.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.044+0000 7f3486ffd640 1 -- 172.21.15.106:0/2005340543 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3470003520 con 0x7f3470004800 2024-04-25T10:31:29.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.045+0000 7f3486ffd640 1 --2- 172.21.15.106:0/2005340543 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3470004800 0x7f3470004c10 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3488069800 tx=0x7f3470005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef287c83596f0ff7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:29.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.045+0000 7f34867fc640 1 -- 172.21.15.106:0/2005340543 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f347c036070 con 0x7f3470004800 2024-04-25T10:31:29.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.045+0000 7f34867fc640 1 -- 172.21.15.106:0/2005340543 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f347c031400 con 0x7f3470004800 2024-04-25T10:31:29.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.045+0000 7f34867fc640 1 -- 172.21.15.106:0/2005340543 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f347c009690 con 0x7f3470004800 2024-04-25T10:31:29.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.046+0000 7f348df4d640 1 -- 172.21.15.106:0/2005340543 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3470004800 msgr2=0x7f3470004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:29.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.046+0000 7f348df4d640 1 --2- 172.21.15.106:0/2005340543 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3470004800 0x7f3470004c10 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3488069800 tx=0x7f3470005830 comp rx=0 tx=0).stop 2024-04-25T10:31:29.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.046+0000 7f3487fff640 1 -- 172.21.15.106:0/2005340543 reap_dead start 2024-04-25T10:31:29.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.046+0000 7f348df4d640 1 -- 172.21.15.106:0/2005340543 shutdown_connections 2024-04-25T10:31:29.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.046+0000 7f348df4d640 1 -- 172.21.15.106:0/2005340543 >> 172.21.15.106:0/2005340543 conn(0x7f34880cef20 msgr2=0x7f34880cf330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:29.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.046+0000 7f348df4d640 1 -- 172.21.15.106:0/2005340543 shutdown_connections 2024-04-25T10:31:29.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.046+0000 7f348df4d640 1 -- 172.21.15.106:0/2005340543 wait complete. 2024-04-25T10:31:29.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.048+0000 7f348df4d640 1 Processor -- start 2024-04-25T10:31:29.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.048+0000 7f348df4d640 1 -- start start 2024-04-25T10:31:29.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.049+0000 7f348df4d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3470004800 0x7f34881f4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:29.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.049+0000 7f348df4d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34881510f0 0x7f348814a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:29.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.049+0000 7f348df4d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3488151500 0x7f348806ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:29.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.049+0000 7f348df4d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3488074ae0 con 0x7f3470004800 2024-04-25T10:31:29.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.049+0000 7f348df4d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3488072220 con 0x7f34881510f0 2024-04-25T10:31:29.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.049+0000 7f348df4d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f3488071cf0 con 0x7f3488151500 2024-04-25T10:31:29.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.049+0000 7f34877fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3470004800 0x7f34881f4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:29.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.049+0000 7f34877fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3470004800 0x7f34881f4820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47892/0 (socket says 172.21.15.106:47892) 2024-04-25T10:31:29.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.049+0000 7f34877fe640 1 -- 172.21.15.106:0/1516097728 learned_addr learned my addr 172.21.15.106:0/1516097728 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:31:29.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.049+0000 7f3487fff640 1 -- 172.21.15.106:0/1516097728 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3488151500 msgr2=0x7f348806ca80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-04-25T10:31:29.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.049+0000 7f3487fff640 1 --2- 172.21.15.106:0/1516097728 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3488151500 0x7f348806ca80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:31:29.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.049+0000 7f3486ffd640 1 --2- 172.21.15.106:0/1516097728 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34881510f0 0x7f348814a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:29.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.049+0000 7f34877fe640 1 -- 172.21.15.106:0/1516097728 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3488151500 msgr2=0x7f348806ca80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:31:29.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.049+0000 7f34877fe640 1 --2- 172.21.15.106:0/1516097728 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3488151500 0x7f348806ca80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:29.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.049+0000 7f34877fe640 1 -- 172.21.15.106:0/1516097728 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34881510f0 msgr2=0x7f348814a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:29.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.049+0000 7f34877fe640 1 --2- 172.21.15.106:0/1516097728 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34881510f0 0x7f348814a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:29.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.049+0000 7f34877fe640 1 -- 172.21.15.106:0/1516097728 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34881f4d60 con 0x7f3470004800 2024-04-25T10:31:29.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.049+0000 7f3486ffd640 1 --2- 172.21.15.106:0/1516097728 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34881510f0 0x7f348814a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:31:29.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.049+0000 7f34877fe640 1 --2- 172.21.15.106:0/1516097728 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3470004800 0x7f34881f4820 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3478007340 tx=0x7f3478007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:29.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.050+0000 7f346ffff640 1 -- 172.21.15.106:0/1516097728 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f347800f940 con 0x7f3470004800 2024-04-25T10:31:29.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.050+0000 7f346ffff640 1 -- 172.21.15.106:0/1516097728 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3478018070 con 0x7f3470004800 2024-04-25T10:31:29.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.050+0000 7f348df4d640 1 -- 172.21.15.106:0/1516097728 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f348814f190 con 0x7f3470004800 2024-04-25T10:31:29.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.050+0000 7f346ffff640 1 -- 172.21.15.106:0/1516097728 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f347800fd00 con 0x7f3470004800 2024-04-25T10:31:29.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.050+0000 7f348df4d640 1 -- 172.21.15.106:0/1516097728 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f348814f750 con 0x7f3470004800 2024-04-25T10:31:29.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.051+0000 7f348df4d640 1 --2- 172.21.15.106:0/1516097728 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f34881f4f30 0x7f348806cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:29.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.051+0000 7f348df4d640 1 -- 172.21.15.106:0/1516097728 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f348814ef20 con 0x7f34881f4f30 2024-04-25T10:31:29.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.051+0000 7f3486ffd640 1 --2- 172.21.15.106:0/1516097728 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f34881f4f30 0x7f348806cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:29.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.051+0000 7f3486ffd640 1 -- 172.21.15.106:0/1516097728 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f348806a3d0 con 0x7f34881f4f30 2024-04-25T10:31:29.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.051+0000 7f3486ffd640 1 --2- 172.21.15.106:0/1516097728 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f34881f4f30 0x7f348806cfc0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f34880ceed0 tx=0x7f347c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:29.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.051+0000 7f346ffff640 1 -- 172.21.15.106:0/1516097728 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f347c036040 con 0x7f34881f4f30 2024-04-25T10:31:29.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.051+0000 7f346ffff640 1 -- 172.21.15.106:0/1516097728 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f347801d460 con 0x7f3470004800 2024-04-25T10:31:29.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.052+0000 7f346ffff640 1 --2- 172.21.15.106:0/1516097728 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f34740743e0 0x7f34740768f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:29.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.052+0000 7f346ffff640 1 -- 172.21.15.106:0/1516097728 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f3478095370 con 0x7f3470004800 2024-04-25T10:31:29.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.052+0000 7f3487fff640 1 --2- 172.21.15.106:0/1516097728 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f34740743e0 0x7f34740768f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:29.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.052+0000 7f346ffff640 1 -- 172.21.15.106:0/1516097728 <== mon.0 v2:172.21.15.106:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f347c03b440 con 0x7f34881f4f30 2024-04-25T10:31:29.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.052+0000 7f346ffff640 1 -- 172.21.15.106:0/1516097728 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f34881f4f30 msgr2=0x7f348806cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:29.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.052+0000 7f346ffff640 1 --2- 172.21.15.106:0/1516097728 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f34881f4f30 0x7f348806cfc0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f34880ceed0 tx=0x7f347c004730 comp rx=0 tx=0).stop 2024-04-25T10:31:29.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.052+0000 7f346ffff640 1 -- 172.21.15.106:0/1516097728 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f34881f4f30 msgr2=0x7f348806cfc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T10:31:29.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.052+0000 7f346ffff640 1 --2- 172.21.15.106:0/1516097728 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f34881f4f30 0x7f348806cfc0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:29.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.052+0000 7f3487fff640 1 --2- 172.21.15.106:0/1516097728 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f34740743e0 0x7f34740768f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3488073750 tx=0x7f34800091b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:29.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.058+0000 7f348df4d640 1 --2- 172.21.15.106:0/1516097728 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f348814f920 0x7f348815bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:29.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.058+0000 7f348df4d640 1 -- 172.21.15.106:0/1516097728 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f34880c9c50 con 0x7f348814f920 2024-04-25T10:31:29.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.059+0000 7f3486ffd640 1 --2- 172.21.15.106:0/1516097728 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f348814f920 0x7f348815bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:29.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.059+0000 7f3486ffd640 1 -- 172.21.15.106:0/1516097728 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f3488158540 con 0x7f348814f920 2024-04-25T10:31:29.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.059+0000 7f3486ffd640 1 --2- 172.21.15.106:0/1516097728 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f348814f920 0x7f348815bc90 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f347c000c00 tx=0x7f347c02f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:29.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.059+0000 7f346ffff640 1 -- 172.21.15.106:0/1516097728 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f347c04d070 con 0x7f348814f920 2024-04-25T10:31:29.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.059+0000 7f346ffff640 1 -- 172.21.15.106:0/1516097728 <== mon.0 v2:172.21.15.106:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5058 (secure 0 0 0) 0x7f347c046710 con 0x7f348814f920 2024-04-25T10:31:29.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.059+0000 7f346ffff640 1 -- 172.21.15.106:0/1516097728 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f348814f920 msgr2=0x7f348815bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:29.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.059+0000 7f346ffff640 1 --2- 172.21.15.106:0/1516097728 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f348814f920 0x7f348815bc90 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f347c000c00 tx=0x7f347c02f000 comp rx=0 tx=0).stop 2024-04-25T10:31:29.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.059+0000 7f346ffff640 1 -- 172.21.15.106:0/1516097728 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f348814f920 msgr2=0x7f348815bc90 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T10:31:29.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.059+0000 7f346ffff640 1 --2- 172.21.15.106:0/1516097728 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f348814f920 0x7f348815bc90 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:29.063 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-25T10:31:29.063 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "a", 2024-04-25T10:31:29.063 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 0, 2024-04-25T10:31:29.063 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "leader", 2024-04-25T10:31:29.063 INFO:teuthology.orchestra.run.smithi106.stdout: "election_epoch": 8, 2024-04-25T10:31:29.063 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 81298, 2024-04-25T10:31:29.063 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum": [ 2024-04-25T10:31:29.063 INFO:teuthology.orchestra.run.smithi106.stdout: 0, 2024-04-25T10:31:29.063 INFO:teuthology.orchestra.run.smithi106.stdout: 1 2024-04-25T10:31:29.063 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:31:29.063 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_age": 0, 2024-04-25T10:31:29.063 INFO:teuthology.orchestra.run.smithi106.stdout: "features": { 2024-04-25T10:31:29.063 INFO:teuthology.orchestra.run.smithi106.stdout: "required_con": "2450521708007866372", 2024-04-25T10:31:29.064 INFO:teuthology.orchestra.run.smithi106.stdout: "required_mon": [ 2024-04-25T10:31:29.064 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:31:29.064 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:31:29.064 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:31:29.064 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:31:29.064 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:31:29.064 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:31:29.064 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:31:29.064 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:31:29.064 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:31:29.064 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:31:29.064 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:31:29.064 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:31:29.064 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_con": "4540701547738038271", 2024-04-25T10:31:29.065 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_mon": [ 2024-04-25T10:31:29.065 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:31:29.065 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:31:29.065 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:31:29.065 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:31:29.065 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:31:29.065 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:31:29.065 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:31:29.065 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:31:29.065 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:31:29.065 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:31:29.065 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:31:29.065 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:31:29.065 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:29.066 INFO:teuthology.orchestra.run.smithi106.stdout: "outside_quorum": [], 2024-04-25T10:31:29.066 INFO:teuthology.orchestra.run.smithi106.stdout: "extra_probe_peers": [], 2024-04-25T10:31:29.066 INFO:teuthology.orchestra.run.smithi106.stdout: "sync_provider": [], 2024-04-25T10:31:29.066 INFO:teuthology.orchestra.run.smithi106.stdout: "monmap": { 2024-04-25T10:31:29.066 INFO:teuthology.orchestra.run.smithi106.stdout: "epoch": 1, 2024-04-25T10:31:29.066 INFO:teuthology.orchestra.run.smithi106.stdout: "fsid": "3ee77985-e8e4-41cf-81e9-7c774cf7891f", 2024-04-25T10:31:29.066 INFO:teuthology.orchestra.run.smithi106.stdout: "modified": "2024-04-25T10:29:33.446228Z", 2024-04-25T10:31:29.066 INFO:teuthology.orchestra.run.smithi106.stdout: "created": "2024-04-25T10:29:33.446228Z", 2024-04-25T10:31:29.066 INFO:teuthology.orchestra.run.smithi106.stdout: "min_mon_release": 19, 2024-04-25T10:31:29.066 INFO:teuthology.orchestra.run.smithi106.stdout: "min_mon_release_name": "squid", 2024-04-25T10:31:29.066 INFO:teuthology.orchestra.run.smithi106.stdout: "election_strategy": 1, 2024-04-25T10:31:29.066 INFO:teuthology.orchestra.run.smithi106.stdout: "disallowed_leaders: ": "", 2024-04-25T10:31:29.066 INFO:teuthology.orchestra.run.smithi106.stdout: "stretch_mode": false, 2024-04-25T10:31:29.067 INFO:teuthology.orchestra.run.smithi106.stdout: "tiebreaker_mon": "", 2024-04-25T10:31:29.067 INFO:teuthology.orchestra.run.smithi106.stdout: "removed_ranks: ": "", 2024-04-25T10:31:29.067 INFO:teuthology.orchestra.run.smithi106.stdout: "features": { 2024-04-25T10:31:29.067 INFO:teuthology.orchestra.run.smithi106.stdout: "persistent": [ 2024-04-25T10:31:29.067 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:31:29.067 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:31:29.067 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:31:29.067 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:31:29.067 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:31:29.067 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:31:29.067 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:31:29.067 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:31:29.067 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:31:29.068 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:31:29.068 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:31:29.068 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:31:29.068 INFO:teuthology.orchestra.run.smithi106.stdout: "optional": [] 2024-04-25T10:31:29.068 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:29.068 INFO:teuthology.orchestra.run.smithi106.stdout: "mons": [ 2024-04-25T10:31:29.068 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:29.068 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 0, 2024-04-25T10:31:29.068 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "a", 2024-04-25T10:31:29.068 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:31:29.068 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:31:29.068 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:29.068 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:31:29.068 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:3300", 2024-04-25T10:31:29.069 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:31:29.069 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:29.069 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:29.069 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:31:29.069 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:6789", 2024-04-25T10:31:29.069 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:31:29.069 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:31:29.069 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:31:29.069 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:29.069 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:6789/0", 2024-04-25T10:31:29.069 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.106:6789/0", 2024-04-25T10:31:29.069 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:31:29.069 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:31:29.069 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:31:29.070 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:29.070 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:29.070 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 1, 2024-04-25T10:31:29.070 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "b", 2024-04-25T10:31:29.070 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:31:29.070 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:31:29.070 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:29.070 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:31:29.070 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:3300", 2024-04-25T10:31:29.070 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:31:29.070 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:29.070 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:29.070 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:31:29.071 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6789", 2024-04-25T10:31:29.071 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:31:29.071 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:31:29.071 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:31:29.071 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:29.071 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6789/0", 2024-04-25T10:31:29.071 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.134:6789/0", 2024-04-25T10:31:29.071 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:31:29.071 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:31:29.071 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:31:29.071 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:29.071 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:29.071 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 2, 2024-04-25T10:31:29.071 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "c", 2024-04-25T10:31:29.071 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:31:29.072 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:31:29.072 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:29.072 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:31:29.072 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:3301", 2024-04-25T10:31:29.072 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:31:29.072 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:29.072 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:29.072 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:31:29.072 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6790", 2024-04-25T10:31:29.072 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:31:29.072 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:31:29.072 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:31:29.072 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:29.072 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6790/0", 2024-04-25T10:31:29.072 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.134:6790/0", 2024-04-25T10:31:29.073 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:31:29.073 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:31:29.073 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:31:29.073 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:31:29.073 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:31:29.073 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:29.073 INFO:teuthology.orchestra.run.smithi106.stdout: "feature_map": { 2024-04-25T10:31:29.073 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": [ 2024-04-25T10:31:29.073 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:29.073 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:31:29.073 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:31:29.073 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 1 2024-04-25T10:31:29.073 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:31:29.073 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:31:29.073 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": [ 2024-04-25T10:31:29.074 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:29.074 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:31:29.074 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:31:29.074 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 6 2024-04-25T10:31:29.074 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:31:29.074 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:31:29.074 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": [ 2024-04-25T10:31:29.074 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:29.074 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:31:29.074 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:31:29.074 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 4 2024-04-25T10:31:29.074 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:31:29.074 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:31:29.074 INFO:teuthology.orchestra.run.smithi106.stdout: "client": [ 2024-04-25T10:31:29.075 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:29.075 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:31:29.075 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:31:29.075 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 7 2024-04-25T10:31:29.075 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:31:29.075 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:31:29.075 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": [ 2024-04-25T10:31:29.075 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:29.075 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:31:29.075 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:31:29.075 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 2 2024-04-25T10:31:29.075 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:31:29.075 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:31:29.075 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:29.076 INFO:teuthology.orchestra.run.smithi106.stdout: "stretch_mode": false 2024-04-25T10:31:29.076 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-25T10:31:29.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.060+0000 7f348df4d640 1 -- 172.21.15.106:0/1516097728 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f34740743e0 msgr2=0x7f34740768f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:29.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.060+0000 7f348df4d640 1 --2- 172.21.15.106:0/1516097728 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f34740743e0 0x7f34740768f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3488073750 tx=0x7f34800091b0 comp rx=0 tx=0).stop 2024-04-25T10:31:29.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.060+0000 7f348df4d640 1 -- 172.21.15.106:0/1516097728 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3470004800 msgr2=0x7f34881f4820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:29.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.060+0000 7f348df4d640 1 --2- 172.21.15.106:0/1516097728 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3470004800 0x7f34881f4820 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3478007340 tx=0x7f3478007800 comp rx=0 tx=0).stop 2024-04-25T10:31:29.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.060+0000 7f3487fff640 1 -- 172.21.15.106:0/1516097728 reap_dead start 2024-04-25T10:31:29.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.060+0000 7f3487fff640 1 -- 172.21.15.106:0/1516097728 reap_dead start 2024-04-25T10:31:29.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.060+0000 7f348df4d640 1 -- 172.21.15.106:0/1516097728 shutdown_connections 2024-04-25T10:31:29.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.060+0000 7f348df4d640 1 -- 172.21.15.106:0/1516097728 >> 172.21.15.106:0/1516097728 conn(0x7f34880cef20 msgr2=0x7f34880740d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:29.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.060+0000 7f348df4d640 1 -- 172.21.15.106:0/1516097728 shutdown_connections 2024-04-25T10:31:29.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.060+0000 7f348df4d640 1 -- 172.21.15.106:0/1516097728 wait complete. 2024-04-25T10:31:29.078 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-04-25T10:31:29.078 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-04-25T10:31:29.078 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-25T10:31:29.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.184+0000 7f6e58b1f640 1 Processor -- start 2024-04-25T10:31:29.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.184+0000 7f6e58b1f640 1 -- start start 2024-04-25T10:31:29.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.186+0000 7f6e58b1f640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6e540c7e30 0x7f6e540c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:29.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.186+0000 7f6e58b1f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6e540b4ce0 0x7f6e5415ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:29.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.186+0000 7f6e58b1f640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e540ce630 con 0x7f6e54156ef0 2024-04-25T10:31:29.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.186+0000 7f6e58b1f640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f6e540b50f0 con 0x7f6e540c7e30 2024-04-25T10:31:29.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.186+0000 7f6e58b1f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6e540c7c60 con 0x7f6e540b4ce0 2024-04-25T10:31:29.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.186+0000 7f6e52d9d640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6e540b4ce0 msgr2=0x7f6e5415ce30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-04-25T10:31:29.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.186+0000 7f6e5259c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6e540c7e30 0x7f6e540c8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:29.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.186+0000 7f6e52d9d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6e540b4ce0 0x7f6e5415ce30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:31:29.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.186+0000 7f6e5259c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6e540c7e30 0x7f6e540c8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47926/0 (socket says 172.21.15.106:47926) 2024-04-25T10:31:29.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.186+0000 7f6e5259c640 1 -- 172.21.15.106:0/3635270335 learned_addr learned my addr 172.21.15.106:0/3635270335 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:31:29.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.186+0000 7f6e5259c640 1 -- 172.21.15.106:0/3635270335 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6e540b4ce0 msgr2=0x7f6e5415ce30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:31:29.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.186+0000 7f6e5259c640 1 --2- 172.21.15.106:0/3635270335 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6e540b4ce0 0x7f6e5415ce30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:29.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.186+0000 7f6e5259c640 1 -- 172.21.15.106:0/3635270335 >> v1:172.21.15.134:6789/0 conn(0x7f6e54156ef0 legacy=0x7f6e5406d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:29.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.186+0000 7f6e5259c640 1 -- 172.21.15.106:0/3635270335 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e540c7980 con 0x7f6e540c7e30 2024-04-25T10:31:29.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.187+0000 7f6e5259c640 1 --2- 172.21.15.106:0/3635270335 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e540c7e30 0x7f6e540c8d90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6e540c7de0 tx=0x7f6e4402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=426dc7cde0fb34a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:29.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.187+0000 7f6e5159a640 1 -- 172.21.15.106:0/3635270335 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e44036070 con 0x7f6e540c7e30 2024-04-25T10:31:29.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.187+0000 7f6e5159a640 1 -- 172.21.15.106:0/3635270335 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e44031600 con 0x7f6e540c7e30 2024-04-25T10:31:29.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.187+0000 7f6e5159a640 1 -- 172.21.15.106:0/3635270335 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e440319d0 con 0x7f6e540c7e30 2024-04-25T10:31:29.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.188+0000 7f6e58b1f640 1 -- 172.21.15.106:0/3635270335 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e540c7e30 msgr2=0x7f6e540c8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:29.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.188+0000 7f6e58b1f640 1 --2- 172.21.15.106:0/3635270335 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e540c7e30 0x7f6e540c8d90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6e540c7de0 tx=0x7f6e4402a6a0 comp rx=0 tx=0).stop 2024-04-25T10:31:29.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.188+0000 7f6e58b1f640 1 -- 172.21.15.106:0/3635270335 shutdown_connections 2024-04-25T10:31:29.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.188+0000 7f6e58b1f640 1 --2- 172.21.15.106:0/3635270335 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6e540b4ce0 0x7f6e5415ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:29.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.188+0000 7f6e58b1f640 1 --2- 172.21.15.106:0/3635270335 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e540c7e30 0x7f6e540c8d90 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:29.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.188+0000 7f6e58b1f640 1 -- 172.21.15.106:0/3635270335 >> 172.21.15.106:0/3635270335 conn(0x7f6e54068b10 msgr2=0x7f6e540d1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:29.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.188+0000 7f6e58b1f640 1 -- 172.21.15.106:0/3635270335 shutdown_connections 2024-04-25T10:31:29.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.188+0000 7f6e58b1f640 1 -- 172.21.15.106:0/3635270335 wait complete. 2024-04-25T10:31:29.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.190+0000 7f6e58b1f640 1 Processor -- start 2024-04-25T10:31:29.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.190+0000 7f6e58b1f640 1 -- start start 2024-04-25T10:31:29.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.191+0000 7f6e58b1f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e540b4ce0 0x7f6e54149e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:29.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.191+0000 7f6e58b1f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e540c7e30 0x7f6e541e7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:29.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.191+0000 7f6e58b1f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6e54156ef0 0x7f6e541e8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:29.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.191+0000 7f6e58b1f640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6e5406d710 con 0x7f6e540b4ce0 2024-04-25T10:31:29.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.191+0000 7f6e58b1f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6e540c72f0 con 0x7f6e540c7e30 2024-04-25T10:31:29.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.191+0000 7f6e58b1f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6e5406d890 con 0x7f6e54156ef0 2024-04-25T10:31:29.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.191+0000 7f6e5259c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e540b4ce0 0x7f6e54149e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:29.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.191+0000 7f6e52d9d640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6e54156ef0 msgr2=0x7f6e541e8510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-04-25T10:31:29.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.191+0000 7f6e5259c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e540b4ce0 0x7f6e54149e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47934/0 (socket says 172.21.15.106:47934) 2024-04-25T10:31:29.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.191+0000 7f6e5259c640 1 -- 172.21.15.106:0/354864948 learned_addr learned my addr 172.21.15.106:0/354864948 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:31:29.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.191+0000 7f6e52d9d640 1 --2- 172.21.15.106:0/354864948 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6e54156ef0 0x7f6e541e8510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:31:29.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.191+0000 7f6e51d9b640 1 --2- 172.21.15.106:0/354864948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e540c7e30 0x7f6e541e7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:29.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.191+0000 7f6e5259c640 1 -- 172.21.15.106:0/354864948 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6e54156ef0 msgr2=0x7f6e541e8510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:31:29.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.191+0000 7f6e5259c640 1 --2- 172.21.15.106:0/354864948 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6e54156ef0 0x7f6e541e8510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:29.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.191+0000 7f6e5259c640 1 -- 172.21.15.106:0/354864948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e540c7e30 msgr2=0x7f6e541e7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:29.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.191+0000 7f6e5259c640 1 --2- 172.21.15.106:0/354864948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e540c7e30 0x7f6e541e7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:29.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.191+0000 7f6e5259c640 1 -- 172.21.15.106:0/354864948 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e5414a3c0 con 0x7f6e540b4ce0 2024-04-25T10:31:29.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.191+0000 7f6e51d9b640 1 --2- 172.21.15.106:0/354864948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e540c7e30 0x7f6e541e7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:31:29.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.192+0000 7f6e5259c640 1 --2- 172.21.15.106:0/354864948 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e540b4ce0 0x7f6e54149e80 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6e44009f40 tx=0x7f6e44004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:29.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.192+0000 7f6e42ffd640 1 -- 172.21.15.106:0/354864948 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e44041020 con 0x7f6e540b4ce0 2024-04-25T10:31:29.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.192+0000 7f6e42ffd640 1 -- 172.21.15.106:0/354864948 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e44007820 con 0x7f6e540b4ce0 2024-04-25T10:31:29.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.192+0000 7f6e58b1f640 1 -- 172.21.15.106:0/354864948 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e541f2fd0 con 0x7f6e540b4ce0 2024-04-25T10:31:29.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.192+0000 7f6e42ffd640 1 -- 172.21.15.106:0/354864948 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e44036040 con 0x7f6e540b4ce0 2024-04-25T10:31:29.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.192+0000 7f6e58b1f640 1 -- 172.21.15.106:0/354864948 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e541f3590 con 0x7f6e540b4ce0 2024-04-25T10:31:29.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.193+0000 7f6e58b1f640 1 --2- 172.21.15.106:0/354864948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e541f3760 0x7f6e54154550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:29.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.193+0000 7f6e58b1f640 1 -- 172.21.15.106:0/354864948 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6e5406a3d0 con 0x7f6e541f3760 2024-04-25T10:31:29.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.193+0000 7f6e51d9b640 1 --2- 172.21.15.106:0/354864948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e541f3760 0x7f6e54154550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:29.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.193+0000 7f6e51d9b640 1 -- 172.21.15.106:0/354864948 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6e5415a5a0 con 0x7f6e541f3760 2024-04-25T10:31:29.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.193+0000 7f6e42ffd640 1 -- 172.21.15.106:0/354864948 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6e44044020 con 0x7f6e540b4ce0 2024-04-25T10:31:29.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.193+0000 7f6e51d9b640 1 --2- 172.21.15.106:0/354864948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e541f3760 0x7f6e54154550 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6e540d03c0 tx=0x7f6e4800e890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:29.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.194+0000 7f6e42ffd640 1 --2- 172.21.15.106:0/354864948 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f6e34074490 0x7f6e340769a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:29.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.194+0000 7f6e52d9d640 1 --2- 172.21.15.106:0/354864948 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f6e34074490 0x7f6e340769a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:29.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.194+0000 7f6e42ffd640 1 -- 172.21.15.106:0/354864948 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f6e440b5dc0 con 0x7f6e540b4ce0 2024-04-25T10:31:29.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.194+0000 7f6e42ffd640 1 -- 172.21.15.106:0/354864948 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e480052e0 con 0x7f6e541f3760 2024-04-25T10:31:29.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.194+0000 7f6e42ffd640 1 -- 172.21.15.106:0/354864948 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f6e48005480 con 0x7f6e541f3760 2024-04-25T10:31:29.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.194+0000 7f6e42ffd640 1 -- 172.21.15.106:0/354864948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e541f3760 msgr2=0x7f6e54154550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:29.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.194+0000 7f6e42ffd640 1 --2- 172.21.15.106:0/354864948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e541f3760 0x7f6e54154550 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6e540d03c0 tx=0x7f6e4800e890 comp rx=0 tx=0).stop 2024-04-25T10:31:29.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.194+0000 7f6e42ffd640 1 -- 172.21.15.106:0/354864948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e541f3760 msgr2=0x7f6e54154550 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T10:31:29.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.194+0000 7f6e42ffd640 1 --2- 172.21.15.106:0/354864948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e541f3760 0x7f6e54154550 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:29.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.194+0000 7f6e52d9d640 1 --2- 172.21.15.106:0/354864948 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f6e34074490 0x7f6e340769a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6e540b4840 tx=0x7f6e4c006c60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:29.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.200+0000 7f6e58b1f640 1 --2- 172.21.15.106:0/354864948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e5414a590 0x7f6e54154a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:29.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.200+0000 7f6e58b1f640 1 -- 172.21.15.106:0/354864948 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6e540ce2e0 con 0x7f6e5414a590 2024-04-25T10:31:29.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.201+0000 7f6e51d9b640 1 --2- 172.21.15.106:0/354864948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e5414a590 0x7f6e54154a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:29.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.201+0000 7f6e51d9b640 1 -- 172.21.15.106:0/354864948 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f6e540d2870 con 0x7f6e5414a590 2024-04-25T10:31:29.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.202+0000 7f6e51d9b640 1 --2- 172.21.15.106:0/354864948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e5414a590 0x7f6e54154a90 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6e540d03c0 tx=0x7f6e48000f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:29.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.202+0000 7f6e42ffd640 1 -- 172.21.15.106:0/354864948 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e48006e00 con 0x7f6e5414a590 2024-04-25T10:31:29.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.202+0000 7f6e42ffd640 1 -- 172.21.15.106:0/354864948 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4716 (secure 0 0 0) 0x7f6e48006e00 con 0x7f6e5414a590 2024-04-25T10:31:29.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.202+0000 7f6e42ffd640 1 -- 172.21.15.106:0/354864948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e5414a590 msgr2=0x7f6e54154a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:29.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.202+0000 7f6e42ffd640 1 --2- 172.21.15.106:0/354864948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e5414a590 0x7f6e54154a90 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6e540d03c0 tx=0x7f6e48000f20 comp rx=0 tx=0).stop 2024-04-25T10:31:29.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.202+0000 7f6e42ffd640 1 -- 172.21.15.106:0/354864948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e5414a590 msgr2=0x7f6e54154a90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T10:31:29.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.202+0000 7f6e42ffd640 1 --2- 172.21.15.106:0/354864948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e5414a590 0x7f6e54154a90 secure :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6e540d03c0 tx=0x7f6e48000f20 comp rx=0 tx=0).stop 2024-04-25T10:31:29.205 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-25T10:31:29.205 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "b", 2024-04-25T10:31:29.205 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 1, 2024-04-25T10:31:29.205 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "peon", 2024-04-25T10:31:29.205 INFO:teuthology.orchestra.run.smithi106.stdout: "election_epoch": 8, 2024-04-25T10:31:29.205 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 81398, 2024-04-25T10:31:29.206 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum": [ 2024-04-25T10:31:29.206 INFO:teuthology.orchestra.run.smithi106.stdout: 0, 2024-04-25T10:31:29.206 INFO:teuthology.orchestra.run.smithi106.stdout: 1 2024-04-25T10:31:29.206 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:31:29.206 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_age": 0, 2024-04-25T10:31:29.206 INFO:teuthology.orchestra.run.smithi106.stdout: "features": { 2024-04-25T10:31:29.206 INFO:teuthology.orchestra.run.smithi106.stdout: "required_con": "2450521708007866372", 2024-04-25T10:31:29.206 INFO:teuthology.orchestra.run.smithi106.stdout: "required_mon": [ 2024-04-25T10:31:29.206 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:31:29.206 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:31:29.206 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:31:29.206 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:31:29.206 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:31:29.207 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:31:29.207 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:31:29.207 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:31:29.207 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:31:29.207 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:31:29.207 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:31:29.207 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:31:29.207 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_con": "4540701547738038271", 2024-04-25T10:31:29.207 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_mon": [ 2024-04-25T10:31:29.207 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:31:29.207 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:31:29.207 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:31:29.207 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:31:29.207 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:31:29.208 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:31:29.208 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:31:29.208 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:31:29.208 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:31:29.208 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:31:29.208 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:31:29.208 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:31:29.208 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:29.208 INFO:teuthology.orchestra.run.smithi106.stdout: "outside_quorum": [], 2024-04-25T10:31:29.208 INFO:teuthology.orchestra.run.smithi106.stdout: "extra_probe_peers": [], 2024-04-25T10:31:29.208 INFO:teuthology.orchestra.run.smithi106.stdout: "sync_provider": [], 2024-04-25T10:31:29.208 INFO:teuthology.orchestra.run.smithi106.stdout: "monmap": { 2024-04-25T10:31:29.208 INFO:teuthology.orchestra.run.smithi106.stdout: "epoch": 1, 2024-04-25T10:31:29.208 INFO:teuthology.orchestra.run.smithi106.stdout: "fsid": "3ee77985-e8e4-41cf-81e9-7c774cf7891f", 2024-04-25T10:31:29.209 INFO:teuthology.orchestra.run.smithi106.stdout: "modified": "2024-04-25T10:29:33.446228Z", 2024-04-25T10:31:29.209 INFO:teuthology.orchestra.run.smithi106.stdout: "created": "2024-04-25T10:29:33.446228Z", 2024-04-25T10:31:29.209 INFO:teuthology.orchestra.run.smithi106.stdout: "min_mon_release": 19, 2024-04-25T10:31:29.209 INFO:teuthology.orchestra.run.smithi106.stdout: "min_mon_release_name": "squid", 2024-04-25T10:31:29.209 INFO:teuthology.orchestra.run.smithi106.stdout: "election_strategy": 1, 2024-04-25T10:31:29.209 INFO:teuthology.orchestra.run.smithi106.stdout: "disallowed_leaders: ": "", 2024-04-25T10:31:29.209 INFO:teuthology.orchestra.run.smithi106.stdout: "stretch_mode": false, 2024-04-25T10:31:29.209 INFO:teuthology.orchestra.run.smithi106.stdout: "tiebreaker_mon": "", 2024-04-25T10:31:29.209 INFO:teuthology.orchestra.run.smithi106.stdout: "removed_ranks: ": "", 2024-04-25T10:31:29.209 INFO:teuthology.orchestra.run.smithi106.stdout: "features": { 2024-04-25T10:31:29.209 INFO:teuthology.orchestra.run.smithi106.stdout: "persistent": [ 2024-04-25T10:31:29.209 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:31:29.209 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:31:29.210 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:31:29.210 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:31:29.210 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:31:29.210 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:31:29.210 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:31:29.210 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:31:29.210 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:31:29.210 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:31:29.210 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:31:29.210 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:31:29.210 INFO:teuthology.orchestra.run.smithi106.stdout: "optional": [] 2024-04-25T10:31:29.210 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:29.210 INFO:teuthology.orchestra.run.smithi106.stdout: "mons": [ 2024-04-25T10:31:29.211 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:29.211 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 0, 2024-04-25T10:31:29.211 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "a", 2024-04-25T10:31:29.211 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:31:29.211 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:31:29.211 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:29.211 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:31:29.211 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:3300", 2024-04-25T10:31:29.211 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:31:29.211 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:29.211 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:29.211 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:31:29.211 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:6789", 2024-04-25T10:31:29.211 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:31:29.212 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:31:29.212 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:31:29.212 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:29.212 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:6789/0", 2024-04-25T10:31:29.212 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.106:6789/0", 2024-04-25T10:31:29.212 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:31:29.212 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:31:29.212 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:31:29.212 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:29.212 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:29.212 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 1, 2024-04-25T10:31:29.212 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "b", 2024-04-25T10:31:29.212 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:31:29.212 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:31:29.213 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:29.213 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:31:29.213 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:3300", 2024-04-25T10:31:29.213 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:31:29.213 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:29.213 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:29.213 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:31:29.213 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6789", 2024-04-25T10:31:29.213 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:31:29.213 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:31:29.213 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:31:29.213 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:29.213 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6789/0", 2024-04-25T10:31:29.213 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.134:6789/0", 2024-04-25T10:31:29.214 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:31:29.214 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:31:29.214 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:31:29.214 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:29.214 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:29.214 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 2, 2024-04-25T10:31:29.214 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "c", 2024-04-25T10:31:29.214 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:31:29.214 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:31:29.214 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:29.214 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:31:29.214 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:3301", 2024-04-25T10:31:29.214 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:31:29.215 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:29.215 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:29.215 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:31:29.215 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6790", 2024-04-25T10:31:29.215 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:31:29.215 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:31:29.215 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:31:29.215 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:29.215 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6790/0", 2024-04-25T10:31:29.215 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.134:6790/0", 2024-04-25T10:31:29.215 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:31:29.215 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:31:29.215 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:31:29.216 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:31:29.216 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:31:29.216 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:29.216 INFO:teuthology.orchestra.run.smithi106.stdout: "feature_map": { 2024-04-25T10:31:29.216 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": [ 2024-04-25T10:31:29.216 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:29.216 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:31:29.216 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:31:29.216 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 1 2024-04-25T10:31:29.216 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:31:29.216 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:31:29.216 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": [ 2024-04-25T10:31:29.216 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:29.216 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:31:29.217 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:31:29.217 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 4 2024-04-25T10:31:29.217 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:31:29.217 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:31:29.217 INFO:teuthology.orchestra.run.smithi106.stdout: "client": [ 2024-04-25T10:31:29.217 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:29.217 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:31:29.217 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:31:29.217 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 1 2024-04-25T10:31:29.217 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:31:29.217 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:31:29.217 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:29.217 INFO:teuthology.orchestra.run.smithi106.stdout: "stretch_mode": false 2024-04-25T10:31:29.218 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-25T10:31:29.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.202+0000 7f6e58b1f640 1 -- 172.21.15.106:0/354864948 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f6e34074490 msgr2=0x7f6e340769a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:29.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.202+0000 7f6e58b1f640 1 --2- 172.21.15.106:0/354864948 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f6e34074490 0x7f6e340769a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6e540b4840 tx=0x7f6e4c006c60 comp rx=0 tx=0).stop 2024-04-25T10:31:29.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.202+0000 7f6e58b1f640 1 -- 172.21.15.106:0/354864948 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e540b4ce0 msgr2=0x7f6e54149e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:29.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.202+0000 7f6e58b1f640 1 --2- 172.21.15.106:0/354864948 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e540b4ce0 0x7f6e54149e80 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6e44009f40 tx=0x7f6e44004a40 comp rx=0 tx=0).stop 2024-04-25T10:31:29.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.202+0000 7f6e52d9d640 1 -- 172.21.15.106:0/354864948 reap_dead start 2024-04-25T10:31:29.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.202+0000 7f6e52d9d640 1 -- 172.21.15.106:0/354864948 reap_dead start 2024-04-25T10:31:29.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.202+0000 7f6e58b1f640 1 -- 172.21.15.106:0/354864948 shutdown_connections 2024-04-25T10:31:29.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.202+0000 7f6e58b1f640 1 -- 172.21.15.106:0/354864948 >> 172.21.15.106:0/354864948 conn(0x7f6e54068b10 msgr2=0x7f6e5414ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:29.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.203+0000 7f6e58b1f640 1 -- 172.21.15.106:0/354864948 shutdown_connections 2024-04-25T10:31:29.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:29.203+0000 7f6e58b1f640 1 -- 172.21.15.106:0/354864948 wait complete. 2024-04-25T10:31:29.220 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-04-25T10:31:29.220 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-04-25T10:31:31.890 DEBUG:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:31:31.893 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:31:49.221 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-04-25T10:31:49.221 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-04-25T10:31:49.221 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-25T10:31:49.221 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-25T10:31:49.225 INFO:tasks.ceph.mon.c:Started 2024-04-25T10:31:49.225 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-04-25T10:31:49.225 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T10:31:49.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.346+0000 7f0602a92640 1 Processor -- start 2024-04-25T10:31:49.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.346+0000 7f0602a92640 1 -- start start 2024-04-25T10:31:49.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.347+0000 7f0602a92640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f05fc0c8d90 0x7f05fc156ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:49.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.347+0000 7f0602a92640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05fc0ce630 con 0x7f05fc0c7e30 2024-04-25T10:31:49.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.347+0000 7f0602a92640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05fc0c7980 con 0x7f05fc0b4ce0 2024-04-25T10:31:49.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.347+0000 7f0602a92640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f05fc0b50f0 con 0x7f05fc0c8d90 2024-04-25T10:31:49.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.347+0000 7f05fbfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f05fc0c8d90 0x7f05fc156ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:49.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.347+0000 7f060082e640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f05fc0c7e30 0x7f05fc06d000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:48536/0 (socket says 172.21.15.106:48536) 2024-04-25T10:31:49.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.347+0000 7f060082e640 1 -- 172.21.15.106:0/4238859828 learned_addr learned my addr 172.21.15.106:0/4238859828 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:31:49.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.348+0000 7f05fbfff640 1 -- 172.21.15.106:0/4238859828 >> v1:172.21.15.134:6789/0 conn(0x7f05fc0b4ce0 legacy=0x7f05fc15ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:49.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.348+0000 7f05fbfff640 1 -- 172.21.15.106:0/4238859828 >> v1:172.21.15.106:6789/0 conn(0x7f05fc0c7e30 legacy=0x7f05fc06d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:49.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.348+0000 7f05fbfff640 1 -- 172.21.15.106:0/4238859828 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05fc0c7010 con 0x7f05fc0c8d90 2024-04-25T10:31:49.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.348+0000 7f05fb7fe640 1 -- 172.21.15.106:0/4238859828 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 733577436 0 0) 0x7f05fc0ce630 con 0x7f05fc0c7e30 2024-04-25T10:31:49.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.348+0000 7f05fbfff640 1 --2- 172.21.15.106:0/4238859828 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05fc0c8d90 0x7f05fc156ef0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f05e8009790 tx=0x7f05e802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21cfd0ca9deaa004 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:49.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.348+0000 7f05fb7fe640 1 -- 172.21.15.106:0/4238859828 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e8037070 con 0x7f05fc0c8d90 2024-04-25T10:31:49.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.349+0000 7f05fb7fe640 1 -- 172.21.15.106:0/4238859828 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05e8032440 con 0x7f05fc0c8d90 2024-04-25T10:31:49.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.349+0000 7f05fb7fe640 1 -- 172.21.15.106:0/4238859828 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e8032810 con 0x7f05fc0c8d90 2024-04-25T10:31:49.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.349+0000 7f0602a92640 1 -- 172.21.15.106:0/4238859828 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05fc0c8d90 msgr2=0x7f05fc156ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:49.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.349+0000 7f0602a92640 1 --2- 172.21.15.106:0/4238859828 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05fc0c8d90 0x7f05fc156ef0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f05e8009790 tx=0x7f05e802a180 comp rx=0 tx=0).stop 2024-04-25T10:31:49.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.349+0000 7f0602a92640 1 -- 172.21.15.106:0/4238859828 shutdown_connections 2024-04-25T10:31:49.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.349+0000 7f0602a92640 1 --2- 172.21.15.106:0/4238859828 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05fc0c8d90 0x7f05fc156ef0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:49.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.349+0000 7f0602a92640 1 -- 172.21.15.106:0/4238859828 >> 172.21.15.106:0/4238859828 conn(0x7f05fc068b10 msgr2=0x7f05fc0d1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:49.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.349+0000 7f0602a92640 1 -- 172.21.15.106:0/4238859828 shutdown_connections 2024-04-25T10:31:49.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.349+0000 7f0602a92640 1 -- 172.21.15.106:0/4238859828 wait complete. 2024-04-25T10:31:49.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.351+0000 7f0602a92640 1 Processor -- start 2024-04-25T10:31:49.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.351+0000 7f0602a92640 1 -- start start 2024-04-25T10:31:49.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.352+0000 7f0602a92640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05fc0b4ce0 0x7f05fc1e6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:49.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.352+0000 7f0602a92640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05fc0c7e30 0x7f05fc1f6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:49.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.352+0000 7f0602a92640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f05fc0c8d90 0x7f05fc1e80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:49.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.352+0000 7f0602a92640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f05fc06d710 con 0x7f05fc0c7e30 2024-04-25T10:31:49.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.352+0000 7f0602a92640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f05fc0c7c60 con 0x7f05fc0b4ce0 2024-04-25T10:31:49.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.352+0000 7f0602a92640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f05fc06d890 con 0x7f05fc0c8d90 2024-04-25T10:31:49.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.352+0000 7f05fbfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05fc0c7e30 0x7f05fc1f6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:49.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.352+0000 7f05fbfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05fc0c7e30 0x7f05fc1f6c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42952/0 (socket says 172.21.15.106:42952) 2024-04-25T10:31:49.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.352+0000 7f05fbfff640 1 -- 172.21.15.106:0/3901348565 learned_addr learned my addr 172.21.15.106:0/3901348565 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:31:49.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.353+0000 7f060082e640 1 --2- 172.21.15.106:0/3901348565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05fc0b4ce0 0x7f05fc1e6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:49.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.353+0000 7f060102f640 1 --2- 172.21.15.106:0/3901348565 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f05fc0c8d90 0x7f05fc1e80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:49.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.353+0000 7f060102f640 1 -- 172.21.15.106:0/3901348565 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f05fc0c8d90 msgr2=0x7f05fc1e80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-04-25T10:31:49.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.353+0000 7f060102f640 1 -- 172.21.15.106:0/3901348565 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f05fc0c8d90 msgr2=0x7f05fc1e80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T10:31:49.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.353+0000 7f060102f640 1 --2- 172.21.15.106:0/3901348565 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f05fc0c8d90 0x7f05fc1e80d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:31:49.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.353+0000 7f060102f640 1 --2- 172.21.15.106:0/3901348565 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f05fc0c8d90 0x7f05fc1e80d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:31:49.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.353+0000 7f05fbfff640 1 -- 172.21.15.106:0/3901348565 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f05fc0c8d90 msgr2=0x7f05fc1e80d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:31:49.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.353+0000 7f05fbfff640 1 --2- 172.21.15.106:0/3901348565 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f05fc0c8d90 0x7f05fc1e80d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:49.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.353+0000 7f05fbfff640 1 -- 172.21.15.106:0/3901348565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05fc0b4ce0 msgr2=0x7f05fc1e6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:49.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.353+0000 7f05fbfff640 1 --2- 172.21.15.106:0/3901348565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05fc0b4ce0 0x7f05fc1e6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:49.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.353+0000 7f05fbfff640 1 -- 172.21.15.106:0/3901348565 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05fc1f71b0 con 0x7f05fc0c7e30 2024-04-25T10:31:49.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.353+0000 7f05fbfff640 1 --2- 172.21.15.106:0/3901348565 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05fc0c7e30 0x7f05fc1f6c70 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f05e8035040 tx=0x7f05e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:49.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.353+0000 7f05f97fa640 1 -- 172.21.15.106:0/3901348565 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e8037050 con 0x7f05fc0c7e30 2024-04-25T10:31:49.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.353+0000 7f05f97fa640 1 -- 172.21.15.106:0/3901348565 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05e8038040 con 0x7f05fc0c7e30 2024-04-25T10:31:49.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.353+0000 7f0602a92640 1 -- 172.21.15.106:0/3901348565 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05fc1f2d20 con 0x7f05fc0c7e30 2024-04-25T10:31:49.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.353+0000 7f05f97fa640 1 -- 172.21.15.106:0/3901348565 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e8005dd0 con 0x7f05fc0c7e30 2024-04-25T10:31:49.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.353+0000 7f0602a92640 1 -- 172.21.15.106:0/3901348565 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05fc1f32c0 con 0x7f05fc0c7e30 2024-04-25T10:31:49.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.354+0000 7f0602a92640 1 -- 172.21.15.106:0/3901348565 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05fc06a3d0 con 0x7f05fc0c7e30 2024-04-25T10:31:49.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.355+0000 7f05f97fa640 1 -- 172.21.15.106:0/3901348565 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f05e8030900 con 0x7f05fc0c7e30 2024-04-25T10:31:49.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.355+0000 7f05f97fa640 1 --2- 172.21.15.106:0/3901348565 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f05e00744b0 0x7f05e00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:49.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.355+0000 7f060082e640 1 --2- 172.21.15.106:0/3901348565 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f05e00744b0 0x7f05e00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:49.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.355+0000 7f05f97fa640 1 -- 172.21.15.106:0/3901348565 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f05e80b5de0 con 0x7f05fc0c7e30 2024-04-25T10:31:49.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.356+0000 7f060082e640 1 --2- 172.21.15.106:0/3901348565 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f05e00744b0 0x7f05e00769c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f05f0007c30 tx=0x7f05f0007b80 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:49.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.360+0000 7f05f97fa640 1 -- 172.21.15.106:0/3901348565 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f05e8080580 con 0x7f05fc0c7e30 2024-04-25T10:31:49.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.604+0000 7f0602a92640 1 -- 172.21.15.106:0/3901348565 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f05fc15a5a0 con 0x7f05fc0c7e30 2024-04-25T10:31:49.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.604+0000 7f05f97fa640 1 -- 172.21.15.106:0/3901348565 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1568 (secure 0 0 0) 0x7f05e807fe80 con 0x7f05fc0c7e30 2024-04-25T10:31:49.606 INFO:teuthology.orchestra.run.smithi106.stdout:{"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"3ee77985-e8e4-41cf-81e9-7c774cf7891f","modified":"2024-04-25T10:29:33.446228Z","created":"2024-04-25T10:29:33.446228Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:3300","nonce":0},{"type":"v1","addr":"172.21.15.106:6789","nonce":0}]},"addr":"172.21.15.106:6789/0","public_addr":"172.21.15.106:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T10:31:49.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.606+0000 7f0602a92640 1 -- 172.21.15.106:0/3901348565 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f05e00744b0 msgr2=0x7f05e00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:49.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.606+0000 7f0602a92640 1 --2- 172.21.15.106:0/3901348565 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f05e00744b0 0x7f05e00769c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f05f0007c30 tx=0x7f05f0007b80 comp rx=0 tx=0).stop 2024-04-25T10:31:49.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.606+0000 7f0602a92640 1 -- 172.21.15.106:0/3901348565 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05fc0c7e30 msgr2=0x7f05fc1f6c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:49.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.606+0000 7f0602a92640 1 --2- 172.21.15.106:0/3901348565 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05fc0c7e30 0x7f05fc1f6c70 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f05e8035040 tx=0x7f05e80046c0 comp rx=0 tx=0).stop 2024-04-25T10:31:49.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.606+0000 7f0602a92640 1 -- 172.21.15.106:0/3901348565 shutdown_connections 2024-04-25T10:31:49.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.606+0000 7f0602a92640 1 --2- 172.21.15.106:0/3901348565 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f05e00744b0 0x7f05e00769c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:49.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.606+0000 7f0602a92640 1 --2- 172.21.15.106:0/3901348565 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f05fc0c8d90 0x7f05fc1e80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:49.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.606+0000 7f0602a92640 1 --2- 172.21.15.106:0/3901348565 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05fc0c7e30 0x7f05fc1f6c70 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:49.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.606+0000 7f0602a92640 1 --2- 172.21.15.106:0/3901348565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05fc0b4ce0 0x7f05fc1e6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:49.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.606+0000 7f0602a92640 1 -- 172.21.15.106:0/3901348565 >> 172.21.15.106:0/3901348565 conn(0x7f05fc068b10 msgr2=0x7f05fc156280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:49.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.606+0000 7f0602a92640 1 -- 172.21.15.106:0/3901348565 shutdown_connections 2024-04-25T10:31:49.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.606+0000 7f0602a92640 1 -- 172.21.15.106:0/3901348565 wait complete. 2024-04-25T10:31:49.619 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-04-25T10:31:49.619 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-04-25T10:31:49.619 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-25T10:31:49.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.726+0000 7f56b389a640 1 Processor -- start 2024-04-25T10:31:49.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.726+0000 7f56b389a640 1 -- start start 2024-04-25T10:31:49.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.728+0000 7f56b389a640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f56ac0744c0 0x7f56ac06f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:49.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.728+0000 7f56b389a640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56ac068ad0 con 0x7f56ac073020 2024-04-25T10:31:49.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.728+0000 7f56b389a640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56ac0c9010 con 0x7f56ac068600 2024-04-25T10:31:49.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.728+0000 7f56b389a640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f56ac071190 con 0x7f56ac0744c0 2024-04-25T10:31:49.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.728+0000 7f56b1e37640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f56ac0744c0 0x7f56ac06f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:49.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.728+0000 7f56b1e37640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f56ac0744c0 0x7f56ac06f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42954/0 (socket says 172.21.15.106:42954) 2024-04-25T10:31:49.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.728+0000 7f56b1e37640 1 -- 172.21.15.106:0/3229699716 learned_addr learned my addr 172.21.15.106:0/3229699716 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:31:49.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.728+0000 7f56b0e35640 1 --1- 172.21.15.106:0/3229699716 >> v1:172.21.15.106:6789/0 conn(0x7f56ac073020 0x7f56ac073db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:48546/0 (socket says 172.21.15.106:48546) 2024-04-25T10:31:49.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.728+0000 7f56b1e37640 1 -- 172.21.15.106:0/3229699716 >> v1:172.21.15.134:6789/0 conn(0x7f56ac068600 legacy=0x7f56ac070a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:49.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.728+0000 7f56b1e37640 1 -- 172.21.15.106:0/3229699716 >> v1:172.21.15.106:6789/0 conn(0x7f56ac073020 legacy=0x7f56ac073db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:49.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.728+0000 7f56b1e37640 1 -- 172.21.15.106:0/3229699716 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56ac0c92f0 con 0x7f56ac0744c0 2024-04-25T10:31:49.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.729+0000 7f56b1e37640 1 --2- 172.21.15.106:0/3229699716 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56ac0744c0 0x7f56ac06f780 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f56a4009870 tx=0x7f56a402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa7ce696521f8b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:49.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.729+0000 7f56a3fff640 1 -- 172.21.15.106:0/3229699716 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a4037070 con 0x7f56ac0744c0 2024-04-25T10:31:49.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.729+0000 7f56a3fff640 1 -- 172.21.15.106:0/3229699716 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56a4032440 con 0x7f56ac0744c0 2024-04-25T10:31:49.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.729+0000 7f56a3fff640 1 -- 172.21.15.106:0/3229699716 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a4032810 con 0x7f56ac0744c0 2024-04-25T10:31:49.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.730+0000 7f56b389a640 1 -- 172.21.15.106:0/3229699716 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56ac0744c0 msgr2=0x7f56ac06f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:49.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.730+0000 7f56b389a640 1 --2- 172.21.15.106:0/3229699716 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56ac0744c0 0x7f56ac06f780 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f56a4009870 tx=0x7f56a402a2a0 comp rx=0 tx=0).stop 2024-04-25T10:31:49.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.730+0000 7f56b389a640 1 -- 172.21.15.106:0/3229699716 shutdown_connections 2024-04-25T10:31:49.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.730+0000 7f56b389a640 1 --2- 172.21.15.106:0/3229699716 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56ac0744c0 0x7f56ac06f780 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:49.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.730+0000 7f56b389a640 1 -- 172.21.15.106:0/3229699716 >> 172.21.15.106:0/3229699716 conn(0x7f56ac0ce570 msgr2=0x7f56ac0ce980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:49.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.730+0000 7f56b389a640 1 -- 172.21.15.106:0/3229699716 shutdown_connections 2024-04-25T10:31:49.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.730+0000 7f56b389a640 1 -- 172.21.15.106:0/3229699716 wait complete. 2024-04-25T10:31:49.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.731+0000 7f56b389a640 1 Processor -- start 2024-04-25T10:31:49.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.732+0000 7f56b389a640 1 -- start start 2024-04-25T10:31:49.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.733+0000 7f56b389a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56ac068600 0x7f56ac0c6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:49.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.733+0000 7f56b389a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56ac073020 0x7f56ac1f60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:49.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.733+0000 7f56b389a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f56ac0744c0 0x7f56ac1f6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:49.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.733+0000 7f56b389a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f56ac0c9ce0 con 0x7f56ac068600 2024-04-25T10:31:49.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.733+0000 7f56b389a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f56ac06fcc0 con 0x7f56ac073020 2024-04-25T10:31:49.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.733+0000 7f56b389a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f56ac150ed0 con 0x7f56ac0744c0 2024-04-25T10:31:49.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.733+0000 7f56b1636640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56ac068600 0x7f56ac0c6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:49.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.733+0000 7f56b1636640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56ac068600 0x7f56ac0c6150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42970/0 (socket says 172.21.15.106:42970) 2024-04-25T10:31:49.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.733+0000 7f56b1636640 1 -- 172.21.15.106:0/932273007 learned_addr learned my addr 172.21.15.106:0/932273007 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:31:49.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.733+0000 7f56b1e37640 1 --2- 172.21.15.106:0/932273007 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f56ac0744c0 0x7f56ac1f6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:49.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.733+0000 7f56b0e35640 1 --2- 172.21.15.106:0/932273007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56ac073020 0x7f56ac1f60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:49.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.733+0000 7f56b1636640 1 -- 172.21.15.106:0/932273007 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f56ac0744c0 msgr2=0x7f56ac1f6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:49.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.733+0000 7f56b1636640 1 --2- 172.21.15.106:0/932273007 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f56ac0744c0 0x7f56ac1f6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:49.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.733+0000 7f56b1636640 1 -- 172.21.15.106:0/932273007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56ac073020 msgr2=0x7f56ac1f60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:49.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.733+0000 7f56b1636640 1 --2- 172.21.15.106:0/932273007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56ac073020 0x7f56ac1f60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:49.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.733+0000 7f56b1636640 1 -- 172.21.15.106:0/932273007 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56ac0c4270 con 0x7f56ac068600 2024-04-25T10:31:49.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.733+0000 7f56b1636640 1 --2- 172.21.15.106:0/932273007 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56ac068600 0x7f56ac0c6150 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f56a800ba50 tx=0x7f56a800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:49.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.733+0000 7f56a1ffb640 1 -- 172.21.15.106:0/932273007 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a8019070 con 0x7f56ac068600 2024-04-25T10:31:49.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.733+0000 7f56a1ffb640 1 -- 172.21.15.106:0/932273007 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56a800cbb0 con 0x7f56ac068600 2024-04-25T10:31:49.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.734+0000 7f56b389a640 1 -- 172.21.15.106:0/932273007 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56ac0c4550 con 0x7f56ac068600 2024-04-25T10:31:49.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.734+0000 7f56a1ffb640 1 -- 172.21.15.106:0/932273007 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a8014440 con 0x7f56ac068600 2024-04-25T10:31:49.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.734+0000 7f56b389a640 1 -- 172.21.15.106:0/932273007 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56ac0c4b10 con 0x7f56ac068600 2024-04-25T10:31:49.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.735+0000 7f56b389a640 1 --2- 172.21.15.106:0/932273007 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56ac0c5310 0x7f56ac1f6b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:49.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.735+0000 7f56b389a640 1 -- 172.21.15.106:0/932273007 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f56ac0c4020 con 0x7f56ac0c5310 2024-04-25T10:31:49.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.735+0000 7f56b0e35640 1 --2- 172.21.15.106:0/932273007 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56ac0c5310 0x7f56ac1f6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:49.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.735+0000 7f56b0e35640 1 -- 172.21.15.106:0/932273007 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f56ac06a3d0 con 0x7f56ac0c5310 2024-04-25T10:31:49.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.735+0000 7f56b0e35640 1 --2- 172.21.15.106:0/932273007 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56ac0c5310 0x7f56ac1f6b50 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f56ac1f0790 tx=0x7f569c001680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:49.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.735+0000 7f56a1ffb640 1 -- 172.21.15.106:0/932273007 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f569c019070 con 0x7f56ac0c5310 2024-04-25T10:31:49.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.735+0000 7f56a1ffb640 1 -- 172.21.15.106:0/932273007 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f56a801c070 con 0x7f56ac068600 2024-04-25T10:31:49.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.736+0000 7f56a1ffb640 1 --2- 172.21.15.106:0/932273007 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f56900743b0 0x7f56900768c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:49.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.736+0000 7f56a1ffb640 1 -- 172.21.15.106:0/932273007 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f56a8095140 con 0x7f56ac068600 2024-04-25T10:31:49.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.736+0000 7f56b1e37640 1 --2- 172.21.15.106:0/932273007 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f56900743b0 0x7f56900768c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:49.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.736+0000 7f56a1ffb640 1 -- 172.21.15.106:0/932273007 <== mon.0 v2:172.21.15.106:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f569c0114f0 con 0x7f56ac0c5310 2024-04-25T10:31:49.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.736+0000 7f56a1ffb640 1 -- 172.21.15.106:0/932273007 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56ac0c5310 msgr2=0x7f56ac1f6b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:49.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.736+0000 7f56a1ffb640 1 --2- 172.21.15.106:0/932273007 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56ac0c5310 0x7f56ac1f6b50 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f56ac1f0790 tx=0x7f569c001680 comp rx=0 tx=0).stop 2024-04-25T10:31:49.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.736+0000 7f56a1ffb640 1 -- 172.21.15.106:0/932273007 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56ac0c5310 msgr2=0x7f56ac1f6b50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T10:31:49.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.736+0000 7f56a1ffb640 1 --2- 172.21.15.106:0/932273007 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56ac0c5310 0x7f56ac1f6b50 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:49.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.736+0000 7f56b1e37640 1 --2- 172.21.15.106:0/932273007 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f56900743b0 0x7f56900768c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f56a4034000 tx=0x7f56a402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:49.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.742+0000 7f56b389a640 1 --2- 172.21.15.106:0/932273007 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56ac15abb0 0x7f56ac14ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:49.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.742+0000 7f56b389a640 1 -- 172.21.15.106:0/932273007 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f56ac14b090 con 0x7f56ac15abb0 2024-04-25T10:31:49.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.743+0000 7f56b0e35640 1 --2- 172.21.15.106:0/932273007 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56ac15abb0 0x7f56ac14ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:49.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.743+0000 7f56b0e35640 1 -- 172.21.15.106:0/932273007 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f56ac073fe0 con 0x7f56ac15abb0 2024-04-25T10:31:49.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.743+0000 7f56b0e35640 1 --2- 172.21.15.106:0/932273007 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56ac15abb0 0x7f56ac14ab50 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f569c016000 tx=0x7f569c000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:49.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.743+0000 7f56a1ffb640 1 -- 172.21.15.106:0/932273007 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f569c01c8d0 con 0x7f56ac15abb0 2024-04-25T10:31:49.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.743+0000 7f56a1ffb640 1 -- 172.21.15.106:0/932273007 <== mon.0 v2:172.21.15.106:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5071 (secure 0 0 0) 0x7f569c01c8d0 con 0x7f56ac15abb0 2024-04-25T10:31:49.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.743+0000 7f56a1ffb640 1 -- 172.21.15.106:0/932273007 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56ac15abb0 msgr2=0x7f56ac14ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:49.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.743+0000 7f56a1ffb640 1 --2- 172.21.15.106:0/932273007 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56ac15abb0 0x7f56ac14ab50 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f569c016000 tx=0x7f569c000ed0 comp rx=0 tx=0).stop 2024-04-25T10:31:49.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.743+0000 7f56a1ffb640 1 -- 172.21.15.106:0/932273007 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56ac15abb0 msgr2=0x7f56ac14ab50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T10:31:49.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.743+0000 7f56a1ffb640 1 --2- 172.21.15.106:0/932273007 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56ac15abb0 0x7f56ac14ab50 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:49.746 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-25T10:31:49.746 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "a", 2024-04-25T10:31:49.746 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 0, 2024-04-25T10:31:49.746 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "leader", 2024-04-25T10:31:49.746 INFO:teuthology.orchestra.run.smithi106.stdout: "election_epoch": 10, 2024-04-25T10:31:49.746 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 101981, 2024-04-25T10:31:49.746 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum": [ 2024-04-25T10:31:49.746 INFO:teuthology.orchestra.run.smithi106.stdout: 0, 2024-04-25T10:31:49.746 INFO:teuthology.orchestra.run.smithi106.stdout: 1, 2024-04-25T10:31:49.746 INFO:teuthology.orchestra.run.smithi106.stdout: 2 2024-04-25T10:31:49.747 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:31:49.747 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_age": 0, 2024-04-25T10:31:49.747 INFO:teuthology.orchestra.run.smithi106.stdout: "features": { 2024-04-25T10:31:49.747 INFO:teuthology.orchestra.run.smithi106.stdout: "required_con": "2450521708007866372", 2024-04-25T10:31:49.747 INFO:teuthology.orchestra.run.smithi106.stdout: "required_mon": [ 2024-04-25T10:31:49.747 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:31:49.747 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:31:49.747 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:31:49.747 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:31:49.747 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:31:49.747 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:31:49.747 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:31:49.747 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:31:49.747 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:31:49.748 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:31:49.748 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:31:49.748 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:31:49.748 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_con": "4540701547738038271", 2024-04-25T10:31:49.748 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_mon": [ 2024-04-25T10:31:49.748 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:31:49.748 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:31:49.748 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:31:49.748 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:31:49.748 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:31:49.748 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:31:49.748 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:31:49.748 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:31:49.749 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:31:49.749 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:31:49.749 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:31:49.749 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:31:49.749 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:49.749 INFO:teuthology.orchestra.run.smithi106.stdout: "outside_quorum": [], 2024-04-25T10:31:49.749 INFO:teuthology.orchestra.run.smithi106.stdout: "extra_probe_peers": [], 2024-04-25T10:31:49.749 INFO:teuthology.orchestra.run.smithi106.stdout: "sync_provider": [], 2024-04-25T10:31:49.749 INFO:teuthology.orchestra.run.smithi106.stdout: "monmap": { 2024-04-25T10:31:49.749 INFO:teuthology.orchestra.run.smithi106.stdout: "epoch": 1, 2024-04-25T10:31:49.749 INFO:teuthology.orchestra.run.smithi106.stdout: "fsid": "3ee77985-e8e4-41cf-81e9-7c774cf7891f", 2024-04-25T10:31:49.749 INFO:teuthology.orchestra.run.smithi106.stdout: "modified": "2024-04-25T10:29:33.446228Z", 2024-04-25T10:31:49.749 INFO:teuthology.orchestra.run.smithi106.stdout: "created": "2024-04-25T10:29:33.446228Z", 2024-04-25T10:31:49.749 INFO:teuthology.orchestra.run.smithi106.stdout: "min_mon_release": 19, 2024-04-25T10:31:49.750 INFO:teuthology.orchestra.run.smithi106.stdout: "min_mon_release_name": "squid", 2024-04-25T10:31:49.750 INFO:teuthology.orchestra.run.smithi106.stdout: "election_strategy": 1, 2024-04-25T10:31:49.750 INFO:teuthology.orchestra.run.smithi106.stdout: "disallowed_leaders: ": "", 2024-04-25T10:31:49.750 INFO:teuthology.orchestra.run.smithi106.stdout: "stretch_mode": false, 2024-04-25T10:31:49.750 INFO:teuthology.orchestra.run.smithi106.stdout: "tiebreaker_mon": "", 2024-04-25T10:31:49.750 INFO:teuthology.orchestra.run.smithi106.stdout: "removed_ranks: ": "", 2024-04-25T10:31:49.750 INFO:teuthology.orchestra.run.smithi106.stdout: "features": { 2024-04-25T10:31:49.750 INFO:teuthology.orchestra.run.smithi106.stdout: "persistent": [ 2024-04-25T10:31:49.750 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:31:49.750 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:31:49.750 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:31:49.750 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:31:49.751 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:31:49.751 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:31:49.751 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:31:49.751 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:31:49.751 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:31:49.751 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:31:49.751 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:31:49.751 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:31:49.751 INFO:teuthology.orchestra.run.smithi106.stdout: "optional": [] 2024-04-25T10:31:49.751 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:49.751 INFO:teuthology.orchestra.run.smithi106.stdout: "mons": [ 2024-04-25T10:31:49.751 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:49.751 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 0, 2024-04-25T10:31:49.752 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "a", 2024-04-25T10:31:49.752 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:31:49.752 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:31:49.752 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:49.752 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:31:49.752 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:3300", 2024-04-25T10:31:49.752 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:31:49.752 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:49.752 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:49.752 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:31:49.752 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:6789", 2024-04-25T10:31:49.753 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:31:49.753 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:31:49.753 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:31:49.753 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:49.753 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:6789/0", 2024-04-25T10:31:49.753 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.106:6789/0", 2024-04-25T10:31:49.753 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:31:49.753 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:31:49.753 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:31:49.753 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:49.753 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:49.753 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 1, 2024-04-25T10:31:49.753 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "b", 2024-04-25T10:31:49.753 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:31:49.753 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:31:49.754 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:49.754 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:31:49.754 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:3300", 2024-04-25T10:31:49.754 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:31:49.754 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:49.754 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:49.754 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:31:49.754 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6789", 2024-04-25T10:31:49.754 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:31:49.754 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:31:49.754 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:31:49.754 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:49.754 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6789/0", 2024-04-25T10:31:49.754 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.134:6789/0", 2024-04-25T10:31:49.755 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:31:49.755 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:31:49.755 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:31:49.755 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:49.755 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:49.755 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 2, 2024-04-25T10:31:49.755 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "c", 2024-04-25T10:31:49.755 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:31:49.755 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:31:49.755 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:49.755 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:31:49.755 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:3301", 2024-04-25T10:31:49.755 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:31:49.756 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:49.756 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:49.756 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:31:49.756 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6790", 2024-04-25T10:31:49.756 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:31:49.756 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:31:49.756 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:31:49.756 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:49.756 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6790/0", 2024-04-25T10:31:49.756 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.134:6790/0", 2024-04-25T10:31:49.756 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:31:49.756 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:31:49.756 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:31:49.757 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:31:49.757 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:31:49.757 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:49.757 INFO:teuthology.orchestra.run.smithi106.stdout: "feature_map": { 2024-04-25T10:31:49.757 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": [ 2024-04-25T10:31:49.757 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:49.757 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:31:49.757 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:31:49.757 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 1 2024-04-25T10:31:49.757 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:31:49.757 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:31:49.757 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": [ 2024-04-25T10:31:49.757 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:49.758 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:31:49.758 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:31:49.758 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 6 2024-04-25T10:31:49.758 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:31:49.758 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:31:49.758 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": [ 2024-04-25T10:31:49.758 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:49.758 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:31:49.758 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:31:49.758 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 4 2024-04-25T10:31:49.758 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:31:49.758 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:31:49.758 INFO:teuthology.orchestra.run.smithi106.stdout: "client": [ 2024-04-25T10:31:49.758 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:49.759 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:31:49.759 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:31:49.759 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 7 2024-04-25T10:31:49.759 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:31:49.759 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:31:49.759 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": [ 2024-04-25T10:31:49.759 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:49.759 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:31:49.759 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:31:49.759 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 2 2024-04-25T10:31:49.759 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:31:49.759 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:31:49.759 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:49.759 INFO:teuthology.orchestra.run.smithi106.stdout: "stretch_mode": false 2024-04-25T10:31:49.760 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-25T10:31:49.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.744+0000 7f56b389a640 1 -- 172.21.15.106:0/932273007 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f56900743b0 msgr2=0x7f56900768c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:49.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.744+0000 7f56b389a640 1 --2- 172.21.15.106:0/932273007 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f56900743b0 0x7f56900768c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f56a4034000 tx=0x7f56a402e000 comp rx=0 tx=0).stop 2024-04-25T10:31:49.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.744+0000 7f56b389a640 1 -- 172.21.15.106:0/932273007 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56ac068600 msgr2=0x7f56ac0c6150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:49.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.744+0000 7f56b389a640 1 --2- 172.21.15.106:0/932273007 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56ac068600 0x7f56ac0c6150 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f56a800ba50 tx=0x7f56a800bf10 comp rx=0 tx=0).stop 2024-04-25T10:31:49.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.744+0000 7f56b1e37640 1 -- 172.21.15.106:0/932273007 reap_dead start 2024-04-25T10:31:49.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.744+0000 7f56b1e37640 1 -- 172.21.15.106:0/932273007 reap_dead start 2024-04-25T10:31:49.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.744+0000 7f56b389a640 1 -- 172.21.15.106:0/932273007 shutdown_connections 2024-04-25T10:31:49.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.744+0000 7f56b389a640 1 -- 172.21.15.106:0/932273007 >> 172.21.15.106:0/932273007 conn(0x7f56ac0ce570 msgr2=0x7f56ac0ce980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:49.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.744+0000 7f56b389a640 1 -- 172.21.15.106:0/932273007 shutdown_connections 2024-04-25T10:31:49.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.744+0000 7f56b389a640 1 -- 172.21.15.106:0/932273007 wait complete. 2024-04-25T10:31:49.762 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T10:31:49.762 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-04-25T10:31:49.762 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-25T10:31:49.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.868+0000 7f0237dee640 1 Processor -- start 2024-04-25T10:31:49.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.869+0000 7f0237dee640 1 -- start start 2024-04-25T10:31:49.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.870+0000 7f0237dee640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f02300cb270 0x7f0230156ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:49.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.870+0000 7f0237dee640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f023015dae0 con 0x7f02300cce00 2024-04-25T10:31:49.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.870+0000 7f0237dee640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02301604b0 con 0x7f02300cbff0 2024-04-25T10:31:49.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.870+0000 7f0237dee640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f0230161270 con 0x7f02300cb270 2024-04-25T10:31:49.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.870+0000 7f023638b640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f02300cce00 0x7f0230167630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:48562/0 (socket says 172.21.15.106:48562) 2024-04-25T10:31:49.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.870+0000 7f023638b640 1 -- 172.21.15.106:0/4178806843 learned_addr learned my addr 172.21.15.106:0/4178806843 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:31:49.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.870+0000 7f0235389640 1 --2- 172.21.15.106:0/4178806843 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f02300cb270 0x7f0230156ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:49.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.870+0000 7f0235389640 1 -- 172.21.15.106:0/4178806843 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f02300cb270 msgr2=0x7f0230156ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T10:31:49.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.870+0000 7f0235389640 1 -- 172.21.15.106:0/4178806843 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f02300cb270 msgr2=0x7f0230156ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:31:49.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.870+0000 7f0234b88640 1 -- 172.21.15.106:0/4178806843 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2329586473 0 0) 0x7f023015dae0 con 0x7f02300cce00 2024-04-25T10:31:49.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.870+0000 7f0235389640 1 --2- 172.21.15.106:0/4178806843 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f02300cb270 0x7f0230156ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:31:49.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.870+0000 7f0234b88640 1 -- 172.21.15.106:0/4178806843 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f021c003520 con 0x7f02300cce00 2024-04-25T10:31:49.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.870+0000 7f0235389640 1 --2- 172.21.15.106:0/4178806843 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f02300cb270 0x7f0230156ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:31:49.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.870+0000 7f0234b88640 1 -- 172.21.15.106:0/4178806843 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3090181058 0 0) 0x7f02301604b0 con 0x7f02300cbff0 2024-04-25T10:31:49.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.870+0000 7f0234b88640 1 -- 172.21.15.106:0/4178806843 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f023015dae0 con 0x7f02300cbff0 2024-04-25T10:31:49.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.870+0000 7f0234b88640 1 -- 172.21.15.106:0/4178806843 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3759159112 0 0) 0x7f021c003520 con 0x7f02300cce00 2024-04-25T10:31:49.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.870+0000 7f0234b88640 1 -- 172.21.15.106:0/4178806843 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f02300cb270 msgr2=0x7f0230156ec0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:31:49.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.870+0000 7f0234b88640 1 --2- 172.21.15.106:0/4178806843 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f02300cb270 0x7f0230156ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:49.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.870+0000 7f0234b88640 1 -- 172.21.15.106:0/4178806843 >> v1:172.21.15.134:6789/0 conn(0x7f02300cbff0 legacy=0x7f02300c61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:49.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.870+0000 7f0234b88640 1 -- 172.21.15.106:0/4178806843 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0230160790 con 0x7f02300cce00 2024-04-25T10:31:49.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.870+0000 7f0234b88640 1 -- 172.21.15.106:0/4178806843 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3776459350 0 0) 0x7f0228001690 con 0x7f02300cce00 2024-04-25T10:31:49.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.870+0000 7f0234b88640 1 -- 172.21.15.106:0/4178806843 >> v1:172.21.15.106:6789/0 conn(0x7f02300cce00 legacy=0x7f0230167630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:49.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.871+0000 7f0234b88640 1 --2- 172.21.15.106:0/4178806843 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f021c003b70 0x7f021c003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:49.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.871+0000 7f0234b88640 1 --2- 172.21.15.106:0/4178806843 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f02300cb270 0x7f021c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:49.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.871+0000 7f0234b88640 1 --2- 172.21.15.106:0/4178806843 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f021c004d40 0x7f021c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:49.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.871+0000 7f0234b88640 1 -- 172.21.15.106:0/4178806843 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0230161270 con 0x7f021c003b70 2024-04-25T10:31:49.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.871+0000 7f0234b88640 1 -- 172.21.15.106:0/4178806843 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f021c00bba0 con 0x7f021c004d40 2024-04-25T10:31:49.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.871+0000 7f0234b88640 1 -- 172.21.15.106:0/4178806843 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f021c00bda0 con 0x7f02300cb270 2024-04-25T10:31:49.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.871+0000 7f0235b8a640 1 --2- 172.21.15.106:0/4178806843 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f021c003b70 0x7f021c003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:49.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.871+0000 7f023638b640 1 --2- 172.21.15.106:0/4178806843 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f021c004d40 0x7f021c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:49.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.871+0000 7f0235389640 1 --2- 172.21.15.106:0/4178806843 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f02300cb270 0x7f021c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:49.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.871+0000 7f0235b8a640 1 -- 172.21.15.106:0/4178806843 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f02300cb270 msgr2=0x7f021c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:49.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.871+0000 7f0235b8a640 1 --2- 172.21.15.106:0/4178806843 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f02300cb270 0x7f021c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:49.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.871+0000 7f0235b8a640 1 -- 172.21.15.106:0/4178806843 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f021c004d40 msgr2=0x7f021c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:49.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.871+0000 7f0235b8a640 1 --2- 172.21.15.106:0/4178806843 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f021c004d40 0x7f021c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:49.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.871+0000 7f0235b8a640 1 -- 172.21.15.106:0/4178806843 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f021c003520 con 0x7f021c003b70 2024-04-25T10:31:49.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.872+0000 7f0235b8a640 1 --2- 172.21.15.106:0/4178806843 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f021c003b70 0x7f021c003fc0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f022c001fd0 tx=0x7f021c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d60c737c3ec0072 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:49.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.872+0000 7f0234b88640 1 -- 172.21.15.106:0/4178806843 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022c001bf0 con 0x7f021c003b70 2024-04-25T10:31:49.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.872+0000 7f0234b88640 1 -- 172.21.15.106:0/4178806843 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f022c003b70 con 0x7f021c003b70 2024-04-25T10:31:49.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.872+0000 7f0234b88640 1 -- 172.21.15.106:0/4178806843 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022c0050d0 con 0x7f021c003b70 2024-04-25T10:31:49.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.872+0000 7f0237dee640 1 -- 172.21.15.106:0/4178806843 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f021c003b70 msgr2=0x7f021c003fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:49.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.872+0000 7f0237dee640 1 --2- 172.21.15.106:0/4178806843 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f021c003b70 0x7f021c003fc0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f022c001fd0 tx=0x7f021c005830 comp rx=0 tx=0).stop 2024-04-25T10:31:49.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.872+0000 7f023638b640 1 -- 172.21.15.106:0/4178806843 reap_dead start 2024-04-25T10:31:49.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.872+0000 7f0237dee640 1 -- 172.21.15.106:0/4178806843 shutdown_connections 2024-04-25T10:31:49.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.872+0000 7f0237dee640 1 -- 172.21.15.106:0/4178806843 >> 172.21.15.106:0/4178806843 conn(0x7f02300727d0 msgr2=0x7f0230072c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:49.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.873+0000 7f0237dee640 1 -- 172.21.15.106:0/4178806843 shutdown_connections 2024-04-25T10:31:49.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.873+0000 7f0237dee640 1 -- 172.21.15.106:0/4178806843 wait complete. 2024-04-25T10:31:49.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.874+0000 7f0237dee640 1 Processor -- start 2024-04-25T10:31:49.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.874+0000 7f0237dee640 1 -- start start 2024-04-25T10:31:49.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.875+0000 7f0237dee640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f021c003b70 0x7f02301f3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:49.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.875+0000 7f0237dee640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f02300cf1b0 0x7f02300cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:49.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.875+0000 7f0237dee640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f023014b9f0 0x7f023014be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:49.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.875+0000 7f0237dee640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f023015d730 con 0x7f02300cf1b0 2024-04-25T10:31:49.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.875+0000 7f0237dee640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0230160d50 con 0x7f021c003b70 2024-04-25T10:31:49.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.875+0000 7f0237dee640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f023014dfb0 con 0x7f023014b9f0 2024-04-25T10:31:49.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.876+0000 7f0235389640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f02300cf1b0 0x7f02300cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:49.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.876+0000 7f0235389640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f02300cf1b0 0x7f02300cdf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43012/0 (socket says 172.21.15.106:43012) 2024-04-25T10:31:49.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.876+0000 7f0235389640 1 -- 172.21.15.106:0/2090367768 learned_addr learned my addr 172.21.15.106:0/2090367768 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:31:49.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.876+0000 7f0235b8a640 1 --2- 172.21.15.106:0/2090367768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f021c003b70 0x7f02301f3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:49.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.876+0000 7f023638b640 1 --2- 172.21.15.106:0/2090367768 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f023014b9f0 0x7f023014be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:49.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.876+0000 7f0235389640 1 -- 172.21.15.106:0/2090367768 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f023014b9f0 msgr2=0x7f023014be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:49.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.876+0000 7f0235389640 1 --2- 172.21.15.106:0/2090367768 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f023014b9f0 0x7f023014be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:49.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.876+0000 7f0235389640 1 -- 172.21.15.106:0/2090367768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f021c003b70 msgr2=0x7f02301f3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:49.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.876+0000 7f0235389640 1 --2- 172.21.15.106:0/2090367768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f021c003b70 0x7f02301f3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:49.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.876+0000 7f0235389640 1 -- 172.21.15.106:0/2090367768 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02301f3550 con 0x7f02300cf1b0 2024-04-25T10:31:49.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.876+0000 7f0235b8a640 1 --2- 172.21.15.106:0/2090367768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f021c003b70 0x7f02301f3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:31:49.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.876+0000 7f023638b640 1 --2- 172.21.15.106:0/2090367768 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f023014b9f0 0x7f023014be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:31:49.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.876+0000 7f0235389640 1 --2- 172.21.15.106:0/2090367768 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f02300cf1b0 0x7f02300cdf40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f02200108d0 tx=0x7f0220010d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:49.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.876+0000 7f02267fc640 1 -- 172.21.15.106:0/2090367768 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022001a020 con 0x7f02300cf1b0 2024-04-25T10:31:49.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.876+0000 7f02267fc640 1 -- 172.21.15.106:0/2090367768 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f022000e960 con 0x7f02300cf1b0 2024-04-25T10:31:49.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.876+0000 7f0237dee640 1 -- 172.21.15.106:0/2090367768 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f023014c340 con 0x7f02300cf1b0 2024-04-25T10:31:49.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.876+0000 7f02267fc640 1 -- 172.21.15.106:0/2090367768 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022000ed50 con 0x7f02300cf1b0 2024-04-25T10:31:49.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.876+0000 7f0237dee640 1 -- 172.21.15.106:0/2090367768 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02301f3760 con 0x7f02300cf1b0 2024-04-25T10:31:49.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.877+0000 7f0237dee640 1 --2- 172.21.15.106:0/2090367768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02300cfa80 0x7f02301f5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:49.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.877+0000 7f0237dee640 1 -- 172.21.15.106:0/2090367768 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f023015d8b0 con 0x7f02300cfa80 2024-04-25T10:31:49.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.877+0000 7f0235b8a640 1 --2- 172.21.15.106:0/2090367768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02300cfa80 0x7f02301f5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:49.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.877+0000 7f0235b8a640 1 -- 172.21.15.106:0/2090367768 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f02300cfe90 con 0x7f02300cfa80 2024-04-25T10:31:49.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.878+0000 7f0235b8a640 1 --2- 172.21.15.106:0/2090367768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02300cfa80 0x7f02301f5da0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f022c001fa0 tx=0x7f022c001650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:49.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.878+0000 7f02267fc640 1 -- 172.21.15.106:0/2090367768 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f022001d020 con 0x7f02300cf1b0 2024-04-25T10:31:49.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.878+0000 7f02267fc640 1 --2- 172.21.15.106:0/2090367768 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f02140744e0 0x7f02140769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:49.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.878+0000 7f023638b640 1 --2- 172.21.15.106:0/2090367768 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f02140744e0 0x7f02140769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:49.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.878+0000 7f02267fc640 1 -- 172.21.15.106:0/2090367768 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f0220095b50 con 0x7f02300cf1b0 2024-04-25T10:31:49.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.878+0000 7f02267fc640 1 -- 172.21.15.106:0/2090367768 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022c040030 con 0x7f02300cfa80 2024-04-25T10:31:49.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.878+0000 7f02267fc640 1 -- 172.21.15.106:0/2090367768 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f022c0052e0 con 0x7f02300cfa80 2024-04-25T10:31:49.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.878+0000 7f02267fc640 1 -- 172.21.15.106:0/2090367768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02300cfa80 msgr2=0x7f02301f5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:49.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.878+0000 7f02267fc640 1 --2- 172.21.15.106:0/2090367768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02300cfa80 0x7f02301f5da0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f022c001fa0 tx=0x7f022c001650 comp rx=0 tx=0).stop 2024-04-25T10:31:49.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.878+0000 7f02267fc640 1 -- 172.21.15.106:0/2090367768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02300cfa80 msgr2=0x7f02301f5da0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T10:31:49.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.878+0000 7f02267fc640 1 --2- 172.21.15.106:0/2090367768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02300cfa80 0x7f02301f5da0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:49.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.879+0000 7f023638b640 1 --2- 172.21.15.106:0/2090367768 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f02140744e0 0x7f02140769f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0230073f20 tx=0x7f0228009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:49.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.885+0000 7f0237dee640 1 --2- 172.21.15.106:0/2090367768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02301f62e0 0x7f02301f66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:49.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.885+0000 7f0237dee640 1 -- 172.21.15.106:0/2090367768 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f02301f6c30 con 0x7f02301f62e0 2024-04-25T10:31:49.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.885+0000 7f0235b8a640 1 --2- 172.21.15.106:0/2090367768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02301f62e0 0x7f02301f66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:49.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.885+0000 7f0235b8a640 1 -- 172.21.15.106:0/2090367768 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f023006a3d0 con 0x7f02301f62e0 2024-04-25T10:31:49.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.886+0000 7f0235b8a640 1 --2- 172.21.15.106:0/2090367768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02301f62e0 0x7f02301f66f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f023006c380 tx=0x7f022c030000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:49.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.886+0000 7f02267fc640 1 -- 172.21.15.106:0/2090367768 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022c040070 con 0x7f02301f62e0 2024-04-25T10:31:49.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.886+0000 7f02267fc640 1 -- 172.21.15.106:0/2090367768 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4729 (secure 0 0 0) 0x7f022c03c850 con 0x7f02301f62e0 2024-04-25T10:31:49.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.886+0000 7f02267fc640 1 -- 172.21.15.106:0/2090367768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02301f62e0 msgr2=0x7f02301f66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:49.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.886+0000 7f02267fc640 1 --2- 172.21.15.106:0/2090367768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02301f62e0 0x7f02301f66f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f023006c380 tx=0x7f022c030000 comp rx=0 tx=0).stop 2024-04-25T10:31:49.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.886+0000 7f02267fc640 1 -- 172.21.15.106:0/2090367768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02301f62e0 msgr2=0x7f02301f66f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T10:31:49.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.886+0000 7f02267fc640 1 --2- 172.21.15.106:0/2090367768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02301f62e0 0x7f02301f66f0 secure :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f023006c380 tx=0x7f022c030000 comp rx=0 tx=0).stop 2024-04-25T10:31:49.888 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-25T10:31:49.888 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "b", 2024-04-25T10:31:49.888 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 1, 2024-04-25T10:31:49.888 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "peon", 2024-04-25T10:31:49.889 INFO:teuthology.orchestra.run.smithi106.stdout: "election_epoch": 10, 2024-04-25T10:31:49.889 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 102081, 2024-04-25T10:31:49.889 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum": [ 2024-04-25T10:31:49.889 INFO:teuthology.orchestra.run.smithi106.stdout: 0, 2024-04-25T10:31:49.889 INFO:teuthology.orchestra.run.smithi106.stdout: 1, 2024-04-25T10:31:49.889 INFO:teuthology.orchestra.run.smithi106.stdout: 2 2024-04-25T10:31:49.889 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:31:49.889 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_age": 0, 2024-04-25T10:31:49.889 INFO:teuthology.orchestra.run.smithi106.stdout: "features": { 2024-04-25T10:31:49.889 INFO:teuthology.orchestra.run.smithi106.stdout: "required_con": "2450521708007866372", 2024-04-25T10:31:49.889 INFO:teuthology.orchestra.run.smithi106.stdout: "required_mon": [ 2024-04-25T10:31:49.889 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:31:49.889 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:31:49.890 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:31:49.890 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:31:49.890 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:31:49.890 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:31:49.890 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:31:49.890 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:31:49.890 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:31:49.890 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:31:49.890 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:31:49.890 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:31:49.890 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_con": "4540701547738038271", 2024-04-25T10:31:49.890 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_mon": [ 2024-04-25T10:31:49.891 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:31:49.891 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:31:49.891 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:31:49.891 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:31:49.891 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:31:49.891 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:31:49.891 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:31:49.891 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:31:49.891 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:31:49.891 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:31:49.891 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:31:49.891 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:31:49.891 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:49.891 INFO:teuthology.orchestra.run.smithi106.stdout: "outside_quorum": [], 2024-04-25T10:31:49.892 INFO:teuthology.orchestra.run.smithi106.stdout: "extra_probe_peers": [], 2024-04-25T10:31:49.892 INFO:teuthology.orchestra.run.smithi106.stdout: "sync_provider": [], 2024-04-25T10:31:49.892 INFO:teuthology.orchestra.run.smithi106.stdout: "monmap": { 2024-04-25T10:31:49.892 INFO:teuthology.orchestra.run.smithi106.stdout: "epoch": 1, 2024-04-25T10:31:49.892 INFO:teuthology.orchestra.run.smithi106.stdout: "fsid": "3ee77985-e8e4-41cf-81e9-7c774cf7891f", 2024-04-25T10:31:49.892 INFO:teuthology.orchestra.run.smithi106.stdout: "modified": "2024-04-25T10:29:33.446228Z", 2024-04-25T10:31:49.892 INFO:teuthology.orchestra.run.smithi106.stdout: "created": "2024-04-25T10:29:33.446228Z", 2024-04-25T10:31:49.892 INFO:teuthology.orchestra.run.smithi106.stdout: "min_mon_release": 19, 2024-04-25T10:31:49.892 INFO:teuthology.orchestra.run.smithi106.stdout: "min_mon_release_name": "squid", 2024-04-25T10:31:49.892 INFO:teuthology.orchestra.run.smithi106.stdout: "election_strategy": 1, 2024-04-25T10:31:49.892 INFO:teuthology.orchestra.run.smithi106.stdout: "disallowed_leaders: ": "", 2024-04-25T10:31:49.892 INFO:teuthology.orchestra.run.smithi106.stdout: "stretch_mode": false, 2024-04-25T10:31:49.892 INFO:teuthology.orchestra.run.smithi106.stdout: "tiebreaker_mon": "", 2024-04-25T10:31:49.893 INFO:teuthology.orchestra.run.smithi106.stdout: "removed_ranks: ": "", 2024-04-25T10:31:49.893 INFO:teuthology.orchestra.run.smithi106.stdout: "features": { 2024-04-25T10:31:49.893 INFO:teuthology.orchestra.run.smithi106.stdout: "persistent": [ 2024-04-25T10:31:49.893 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:31:49.893 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:31:49.893 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:31:49.893 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:31:49.893 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:31:49.893 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:31:49.893 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:31:49.893 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:31:49.893 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:31:49.893 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:31:49.894 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:31:49.894 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:31:49.894 INFO:teuthology.orchestra.run.smithi106.stdout: "optional": [] 2024-04-25T10:31:49.894 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:49.894 INFO:teuthology.orchestra.run.smithi106.stdout: "mons": [ 2024-04-25T10:31:49.894 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:49.894 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 0, 2024-04-25T10:31:49.894 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "a", 2024-04-25T10:31:49.894 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:31:49.894 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:31:49.894 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:49.894 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:31:49.894 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:3300", 2024-04-25T10:31:49.895 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:31:49.895 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:49.895 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:49.895 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:31:49.895 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:6789", 2024-04-25T10:31:49.895 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:31:49.895 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:31:49.895 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:31:49.895 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:49.895 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:6789/0", 2024-04-25T10:31:49.895 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.106:6789/0", 2024-04-25T10:31:49.895 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:31:49.895 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:31:49.895 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:31:49.896 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:49.896 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:49.896 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 1, 2024-04-25T10:31:49.896 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "b", 2024-04-25T10:31:49.896 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:31:49.896 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:31:49.896 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:49.896 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:31:49.896 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:3300", 2024-04-25T10:31:49.896 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:31:49.896 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:49.896 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:49.896 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:31:49.897 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6789", 2024-04-25T10:31:49.897 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:31:49.897 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:31:49.897 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:31:49.897 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:49.897 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6789/0", 2024-04-25T10:31:49.897 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.134:6789/0", 2024-04-25T10:31:49.897 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:31:49.897 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:31:49.897 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:31:49.897 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:49.897 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:49.897 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 2, 2024-04-25T10:31:49.898 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "c", 2024-04-25T10:31:49.898 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:31:49.898 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:31:49.898 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:49.898 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:31:49.898 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:3301", 2024-04-25T10:31:49.898 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:31:49.898 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:49.898 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:49.898 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:31:49.898 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6790", 2024-04-25T10:31:49.898 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:31:49.899 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:31:49.899 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:31:49.899 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:49.899 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6790/0", 2024-04-25T10:31:49.899 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.134:6790/0", 2024-04-25T10:31:49.899 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:31:49.899 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:31:49.899 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:31:49.899 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:31:49.899 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:31:49.899 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:49.899 INFO:teuthology.orchestra.run.smithi106.stdout: "feature_map": { 2024-04-25T10:31:49.899 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": [ 2024-04-25T10:31:49.899 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:49.900 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:31:49.900 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:31:49.900 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 1 2024-04-25T10:31:49.900 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:31:49.900 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:31:49.900 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": [ 2024-04-25T10:31:49.900 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:49.900 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:31:49.900 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:31:49.900 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 4 2024-04-25T10:31:49.900 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:31:49.900 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:31:49.900 INFO:teuthology.orchestra.run.smithi106.stdout: "client": [ 2024-04-25T10:31:49.900 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:49.901 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:31:49.901 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:31:49.901 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 1 2024-04-25T10:31:49.901 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:31:49.901 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:31:49.901 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:49.901 INFO:teuthology.orchestra.run.smithi106.stdout: "stretch_mode": false 2024-04-25T10:31:49.901 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-25T10:31:49.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.886+0000 7f0237dee640 1 -- 172.21.15.106:0/2090367768 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f02140744e0 msgr2=0x7f02140769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:49.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.886+0000 7f0237dee640 1 --2- 172.21.15.106:0/2090367768 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f02140744e0 0x7f02140769f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0230073f20 tx=0x7f0228009040 comp rx=0 tx=0).stop 2024-04-25T10:31:49.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.886+0000 7f0237dee640 1 -- 172.21.15.106:0/2090367768 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f02300cf1b0 msgr2=0x7f02300cdf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:49.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.886+0000 7f0237dee640 1 --2- 172.21.15.106:0/2090367768 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f02300cf1b0 0x7f02300cdf40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f02200108d0 tx=0x7f0220010d90 comp rx=0 tx=0).stop 2024-04-25T10:31:49.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.886+0000 7f023638b640 1 -- 172.21.15.106:0/2090367768 reap_dead start 2024-04-25T10:31:49.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.886+0000 7f023638b640 1 -- 172.21.15.106:0/2090367768 reap_dead start 2024-04-25T10:31:49.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.886+0000 7f0237dee640 1 -- 172.21.15.106:0/2090367768 shutdown_connections 2024-04-25T10:31:49.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.886+0000 7f0237dee640 1 -- 172.21.15.106:0/2090367768 >> 172.21.15.106:0/2090367768 conn(0x7f02300727d0 msgr2=0x7f0230072c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:49.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.887+0000 7f0237dee640 1 -- 172.21.15.106:0/2090367768 shutdown_connections 2024-04-25T10:31:49.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:49.887+0000 7f0237dee640 1 -- 172.21.15.106:0/2090367768 wait complete. 2024-04-25T10:31:49.907 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T10:31:49.907 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-04-25T10:31:49.907 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-25T10:31:50.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.014+0000 7f866aa6e640 1 Processor -- start 2024-04-25T10:31:50.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.014+0000 7f866aa6e640 1 -- start start 2024-04-25T10:31:50.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.015+0000 7f866aa6e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8664072870 0x7f866415a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:50.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.015+0000 7f866aa6e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f866415d9a0 0x7f8664153710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:50.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.015+0000 7f866aa6e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86640c5690 con 0x7f86640c5280 2024-04-25T10:31:50.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.015+0000 7f866aa6e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f86640cb590 con 0x7f866415d9a0 2024-04-25T10:31:50.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.015+0000 7f866aa6e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f86640c7c40 con 0x7f8664072870 2024-04-25T10:31:50.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.015+0000 7f866900b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f866415d9a0 0x7f8664153710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:50.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.015+0000 7f866900b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f866415d9a0 0x7f8664153710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43026/0 (socket says 172.21.15.106:43026) 2024-04-25T10:31:50.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.015+0000 7f866900b640 1 -- 172.21.15.106:0/3360017847 learned_addr learned my addr 172.21.15.106:0/3360017847 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:31:50.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.015+0000 7f866880a640 1 --2- 172.21.15.106:0/3360017847 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8664072870 0x7f866415a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:50.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.016+0000 7f866880a640 1 -- 172.21.15.106:0/3360017847 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8664072870 msgr2=0x7f866415a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T10:31:50.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.016+0000 7f866880a640 1 -- 172.21.15.106:0/3360017847 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8664072870 msgr2=0x7f866415a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:31:50.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.016+0000 7f866880a640 1 --2- 172.21.15.106:0/3360017847 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8664072870 0x7f866415a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:31:50.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.016+0000 7f866900b640 1 -- 172.21.15.106:0/3360017847 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8664072870 msgr2=0x7f866415a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:50.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.016+0000 7f866880a640 1 --2- 172.21.15.106:0/3360017847 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8664072870 0x7f866415a590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:31:50.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.016+0000 7f866900b640 1 --2- 172.21.15.106:0/3360017847 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8664072870 0x7f866415a590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:50.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.016+0000 7f866900b640 1 -- 172.21.15.106:0/3360017847 >> v1:172.21.15.106:6789/0 conn(0x7f86640c5280 legacy=0x7f86640cae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:50.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.016+0000 7f866900b640 1 -- 172.21.15.106:0/3360017847 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86640c7f20 con 0x7f866415d9a0 2024-04-25T10:31:50.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.016+0000 7f866900b640 1 --2- 172.21.15.106:0/3360017847 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f866415d9a0 0x7f8664153710 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f865c00d150 tx=0x7f865c02c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9037a995abe0e530 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:50.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.016+0000 7f86637fe640 1 -- 172.21.15.106:0/3360017847 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865c039070 con 0x7f866415d9a0 2024-04-25T10:31:50.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.016+0000 7f86637fe640 1 -- 172.21.15.106:0/3360017847 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f865c009600 con 0x7f866415d9a0 2024-04-25T10:31:50.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.016+0000 7f86637fe640 1 -- 172.21.15.106:0/3360017847 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865c0099d0 con 0x7f866415d9a0 2024-04-25T10:31:50.020 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.017+0000 7f866aa6e640 1 -- 172.21.15.106:0/3360017847 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f866415d9a0 msgr2=0x7f8664153710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:50.020 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.017+0000 7f866aa6e640 1 --2- 172.21.15.106:0/3360017847 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f866415d9a0 0x7f8664153710 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f865c00d150 tx=0x7f865c02c890 comp rx=0 tx=0).stop 2024-04-25T10:31:50.020 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.017+0000 7f866aa6e640 1 -- 172.21.15.106:0/3360017847 shutdown_connections 2024-04-25T10:31:50.020 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.017+0000 7f866aa6e640 1 --2- 172.21.15.106:0/3360017847 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f866415d9a0 0x7f8664153710 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:50.020 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.017+0000 7f866aa6e640 1 --2- 172.21.15.106:0/3360017847 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8664072870 0x7f866415a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:50.020 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.017+0000 7f866aa6e640 1 -- 172.21.15.106:0/3360017847 >> 172.21.15.106:0/3360017847 conn(0x7f866406c9b0 msgr2=0x7f866406cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:50.020 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.017+0000 7f866aa6e640 1 -- 172.21.15.106:0/3360017847 shutdown_connections 2024-04-25T10:31:50.020 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.017+0000 7f866aa6e640 1 -- 172.21.15.106:0/3360017847 wait complete. 2024-04-25T10:31:50.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.019+0000 7f866aa6e640 1 Processor -- start 2024-04-25T10:31:50.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.019+0000 7f866aa6e640 1 -- start start 2024-04-25T10:31:50.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.020+0000 7f866aa6e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8664072870 0x7f866414d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:50.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.020+0000 7f866aa6e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f86640c5280 0x7f86640ce150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:50.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.020+0000 7f866aa6e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f866415d9a0 0x7f86640ce690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:50.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.020+0000 7f866aa6e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8664159d90 con 0x7f866415d9a0 2024-04-25T10:31:50.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.020+0000 7f866aa6e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8664159ab0 con 0x7f8664072870 2024-04-25T10:31:50.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.020+0000 7f866aa6e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f866415a070 con 0x7f86640c5280 2024-04-25T10:31:50.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.020+0000 7f866900b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f866415d9a0 0x7f86640ce690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:50.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.020+0000 7f866900b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f866415d9a0 0x7f86640ce690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43032/0 (socket says 172.21.15.106:43032) 2024-04-25T10:31:50.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.020+0000 7f866900b640 1 -- 172.21.15.106:0/3380696760 learned_addr learned my addr 172.21.15.106:0/3380696760 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:31:50.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.021+0000 7f866880a640 1 --2- 172.21.15.106:0/3380696760 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8664072870 0x7f866414d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:50.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.021+0000 7f8663fff640 1 --2- 172.21.15.106:0/3380696760 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f86640c5280 0x7f86640ce150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:50.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.021+0000 7f866900b640 1 -- 172.21.15.106:0/3380696760 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f86640c5280 msgr2=0x7f86640ce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:50.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.021+0000 7f866900b640 1 --2- 172.21.15.106:0/3380696760 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f86640c5280 0x7f86640ce150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:50.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.021+0000 7f866900b640 1 -- 172.21.15.106:0/3380696760 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8664072870 msgr2=0x7f866414d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:50.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.021+0000 7f866900b640 1 --2- 172.21.15.106:0/3380696760 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8664072870 0x7f866414d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:50.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.021+0000 7f866900b640 1 -- 172.21.15.106:0/3380696760 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86640cebd0 con 0x7f866415d9a0 2024-04-25T10:31:50.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.021+0000 7f866880a640 1 --2- 172.21.15.106:0/3380696760 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8664072870 0x7f866414d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:31:50.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.021+0000 7f866900b640 1 --2- 172.21.15.106:0/3380696760 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f866415d9a0 0x7f86640ce690 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f865c02ce80 tx=0x7f865c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:50.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.021+0000 7f86617fa640 1 -- 172.21.15.106:0/3380696760 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865c044020 con 0x7f866415d9a0 2024-04-25T10:31:50.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.021+0000 7f86617fa640 1 -- 172.21.15.106:0/3380696760 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f865c009d60 con 0x7f866415d9a0 2024-04-25T10:31:50.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.021+0000 7f866aa6e640 1 -- 172.21.15.106:0/3380696760 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86640c9e80 con 0x7f866415d9a0 2024-04-25T10:31:50.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.021+0000 7f86617fa640 1 -- 172.21.15.106:0/3380696760 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865c039040 con 0x7f866415d9a0 2024-04-25T10:31:50.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.021+0000 7f866aa6e640 1 -- 172.21.15.106:0/3380696760 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86640ccaf0 con 0x7f866415d9a0 2024-04-25T10:31:50.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.022+0000 7f866aa6e640 1 --2- 172.21.15.106:0/3380696760 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f866414d8d0 0x7f86641ef9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:50.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.022+0000 7f866aa6e640 1 -- 172.21.15.106:0/3380696760 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f86640c4920 con 0x7f866414d8d0 2024-04-25T10:31:50.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.022+0000 7f866880a640 1 --2- 172.21.15.106:0/3380696760 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f866414d8d0 0x7f86641ef9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:50.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.022+0000 7f866880a640 1 -- 172.21.15.106:0/3380696760 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f866414d8d0 msgr2=0x7f86641ef9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 20 2024-04-25T10:31:50.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.022+0000 7f866880a640 1 -- 172.21.15.106:0/3380696760 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f866414d8d0 msgr2=0x7f86641ef9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T10:31:50.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.022+0000 7f866880a640 1 --2- 172.21.15.106:0/3380696760 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f866414d8d0 0x7f86641ef9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:31:50.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.022+0000 7f866880a640 1 --2- 172.21.15.106:0/3380696760 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f866414d8d0 0x7f86641ef9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:31:50.026 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.023+0000 7f86617fa640 1 -- 172.21.15.106:0/3380696760 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f865c047020 con 0x7f866415d9a0 2024-04-25T10:31:50.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.023+0000 7f86617fa640 1 --2- 172.21.15.106:0/3380696760 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f86480744e0 0x7f86480769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:50.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.023+0000 7f86617fa640 1 -- 172.21.15.106:0/3380696760 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f865c0b7ee0 con 0x7f866415d9a0 2024-04-25T10:31:50.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.023+0000 7f8663fff640 1 --2- 172.21.15.106:0/3380696760 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f86480744e0 0x7f86480769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:50.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.023+0000 7f8663fff640 1 --2- 172.21.15.106:0/3380696760 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f86480744e0 0x7f86480769f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8650007fd0 tx=0x7f8650009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:50.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.223+0000 7f866880a640 1 --2- 172.21.15.106:0/3380696760 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f866414d8d0 0x7f86641ef9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:50.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.223+0000 7f866880a640 1 -- 172.21.15.106:0/3380696760 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f866414d8d0 msgr2=0x7f86641ef9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-04-25T10:31:50.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.223+0000 7f866880a640 1 -- 172.21.15.106:0/3380696760 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f866414d8d0 msgr2=0x7f86641ef9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T10:31:50.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.223+0000 7f866880a640 1 --2- 172.21.15.106:0/3380696760 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f866414d8d0 0x7f86641ef9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:31:50.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.223+0000 7f866880a640 1 --2- 172.21.15.106:0/3380696760 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f866414d8d0 0x7f86641ef9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T10:31:50.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.624+0000 7f866880a640 1 --2- 172.21.15.106:0/3380696760 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f866414d8d0 0x7f86641ef9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:50.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.624+0000 7f866880a640 1 -- 172.21.15.106:0/3380696760 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f866414d8d0 msgr2=0x7f86641ef9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-04-25T10:31:50.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.624+0000 7f866880a640 1 -- 172.21.15.106:0/3380696760 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f866414d8d0 msgr2=0x7f86641ef9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T10:31:50.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.624+0000 7f866880a640 1 --2- 172.21.15.106:0/3380696760 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f866414d8d0 0x7f86641ef9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:31:50.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:50.624+0000 7f866880a640 1 --2- 172.21.15.106:0/3380696760 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f866414d8d0 0x7f86641ef9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T10:31:51.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:51.424+0000 7f866880a640 1 --2- 172.21.15.106:0/3380696760 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f866414d8d0 0x7f86641ef9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:51.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:51.425+0000 7f866880a640 1 -- 172.21.15.106:0/3380696760 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f866414d8d0 msgr2=0x7f86641ef9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-04-25T10:31:51.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:51.425+0000 7f866880a640 1 -- 172.21.15.106:0/3380696760 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f866414d8d0 msgr2=0x7f86641ef9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T10:31:51.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:51.425+0000 7f866880a640 1 --2- 172.21.15.106:0/3380696760 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f866414d8d0 0x7f86641ef9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:31:51.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:51.425+0000 7f866880a640 1 --2- 172.21.15.106:0/3380696760 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f866414d8d0 0x7f86641ef9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T10:31:53.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:53.027+0000 7f866880a640 1 --2- 172.21.15.106:0/3380696760 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f866414d8d0 0x7f86641ef9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:53.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:53.027+0000 7f866880a640 1 -- 172.21.15.106:0/3380696760 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f866414d8d0 msgr2=0x7f86641ef9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-04-25T10:31:53.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:53.027+0000 7f866880a640 1 -- 172.21.15.106:0/3380696760 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f866414d8d0 msgr2=0x7f86641ef9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T10:31:53.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:53.027+0000 7f866880a640 1 --2- 172.21.15.106:0/3380696760 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f866414d8d0 0x7f86641ef9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:31:53.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:53.027+0000 7f866880a640 1 --2- 172.21.15.106:0/3380696760 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f866414d8d0 0x7f86641ef9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T10:31:54.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.020+0000 7f864ffff640 1 -- 172.21.15.106:0/3380696760 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f866414d8d0 msgr2=0x7f86641ef9b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:31:54.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.020+0000 7f864ffff640 1 --2- 172.21.15.106:0/3380696760 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f866414d8d0 0x7f86641ef9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:54.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.020+0000 7f864ffff640 1 --2- 172.21.15.106:0/3380696760 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f863c003590 0x7f863c005a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:54.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.020+0000 7f864ffff640 1 -- 172.21.15.106:0/3380696760 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f866414d8d0 msgr2=0x7f86641ef9b0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T10:31:54.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.020+0000 7f864ffff640 1 --2- 172.21.15.106:0/3380696760 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f866414d8d0 0x7f86641ef9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:54.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.020+0000 7f864ffff640 1 -- 172.21.15.106:0/3380696760 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f86640c4920 con 0x7f863c003590 2024-04-25T10:31:54.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.020+0000 7f866880a640 1 --2- 172.21.15.106:0/3380696760 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f863c003590 0x7f863c005a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:54.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.020+0000 7f866880a640 1 -- 172.21.15.106:0/3380696760 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f863c003590 msgr2=0x7f863c005a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-04-25T10:31:54.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.020+0000 7f866880a640 1 -- 172.21.15.106:0/3380696760 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f863c003590 msgr2=0x7f863c005a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T10:31:54.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.020+0000 7f866880a640 1 --2- 172.21.15.106:0/3380696760 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f863c003590 0x7f863c005a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:31:54.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.020+0000 7f866880a640 1 --2- 172.21.15.106:0/3380696760 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f863c003590 0x7f863c005a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:31:54.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.221+0000 7f866880a640 1 --2- 172.21.15.106:0/3380696760 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f863c003590 0x7f863c005a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:54.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.221+0000 7f866880a640 1 -- 172.21.15.106:0/3380696760 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f863c003590 msgr2=0x7f863c005a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-04-25T10:31:54.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.221+0000 7f866880a640 1 -- 172.21.15.106:0/3380696760 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f863c003590 msgr2=0x7f863c005a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T10:31:54.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.221+0000 7f866880a640 1 --2- 172.21.15.106:0/3380696760 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f863c003590 0x7f863c005a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:31:54.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.221+0000 7f866880a640 1 --2- 172.21.15.106:0/3380696760 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f863c003590 0x7f863c005a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T10:31:54.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.622+0000 7f866880a640 1 --2- 172.21.15.106:0/3380696760 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f863c003590 0x7f863c005a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:54.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.622+0000 7f866880a640 1 -- 172.21.15.106:0/3380696760 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f863c006150 con 0x7f863c003590 2024-04-25T10:31:54.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.623+0000 7f866880a640 1 --2- 172.21.15.106:0/3380696760 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f863c003590 0x7f863c005a70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f86580149d0 tx=0x7f8658014e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:54.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.623+0000 7f86617fa640 1 -- 172.21.15.106:0/3380696760 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865800f040 con 0x7f863c003590 2024-04-25T10:31:54.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.623+0000 7f86617fa640 1 -- 172.21.15.106:0/3380696760 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f8658016bf0 con 0x7f863c003590 2024-04-25T10:31:54.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.623+0000 7f86617fa640 1 -- 172.21.15.106:0/3380696760 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f863c003590 msgr2=0x7f863c005a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:54.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.623+0000 7f86617fa640 1 --2- 172.21.15.106:0/3380696760 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f863c003590 0x7f863c005a70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f86580149d0 tx=0x7f8658014e90 comp rx=0 tx=0).stop 2024-04-25T10:31:54.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.623+0000 7f86617fa640 1 -- 172.21.15.106:0/3380696760 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f863c003590 msgr2=0x7f863c005a70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T10:31:54.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.623+0000 7f86617fa640 1 --2- 172.21.15.106:0/3380696760 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f863c003590 0x7f863c005a70 secure :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f86580149d0 tx=0x7f8658014e90 comp rx=0 tx=0).stop 2024-04-25T10:31:54.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.631+0000 7f866aa6e640 1 --2- 172.21.15.106:0/3380696760 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f86640cccc0 0x7f86640cd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:54.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.631+0000 7f866aa6e640 1 -- 172.21.15.106:0/3380696760 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f866415d3e0 con 0x7f86640cccc0 2024-04-25T10:31:54.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.631+0000 7f866880a640 1 --2- 172.21.15.106:0/3380696760 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f86640cccc0 0x7f86640cd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:54.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.631+0000 7f866880a640 1 -- 172.21.15.106:0/3380696760 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f866406d5d0 con 0x7f86640cccc0 2024-04-25T10:31:54.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.632+0000 7f866880a640 1 --2- 172.21.15.106:0/3380696760 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f86640cccc0 0x7f86640cd0d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8658023000 tx=0x7f8658004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:54.634 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.632+0000 7f86617fa640 1 -- 172.21.15.106:0/3380696760 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8658018440 con 0x7f86640cccc0 2024-04-25T10:31:54.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.632+0000 7f86617fa640 1 -- 172.21.15.106:0/3380696760 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4557 (secure 0 0 0) 0x7f8658018440 con 0x7f86640cccc0 2024-04-25T10:31:54.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.632+0000 7f86617fa640 1 -- 172.21.15.106:0/3380696760 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f86640cccc0 msgr2=0x7f86640cd0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:54.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.632+0000 7f86617fa640 1 --2- 172.21.15.106:0/3380696760 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f86640cccc0 0x7f86640cd0d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8658023000 tx=0x7f8658004730 comp rx=0 tx=0).stop 2024-04-25T10:31:54.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.632+0000 7f86617fa640 1 -- 172.21.15.106:0/3380696760 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f86640cccc0 msgr2=0x7f86640cd0d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T10:31:54.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.632+0000 7f86617fa640 1 --2- 172.21.15.106:0/3380696760 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f86640cccc0 0x7f86640cd0d0 secure :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8658023000 tx=0x7f8658004730 comp rx=0 tx=0).stop 2024-04-25T10:31:54.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.632+0000 7f866900b640 1 -- 172.21.15.106:0/3380696760 reap_dead start 2024-04-25T10:31:54.636 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-25T10:31:54.636 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "c", 2024-04-25T10:31:54.636 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 2, 2024-04-25T10:31:54.636 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "peon", 2024-04-25T10:31:54.636 INFO:teuthology.orchestra.run.smithi106.stdout: "election_epoch": 10, 2024-04-25T10:31:54.636 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 5303, 2024-04-25T10:31:54.636 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum": [ 2024-04-25T10:31:54.636 INFO:teuthology.orchestra.run.smithi106.stdout: 0, 2024-04-25T10:31:54.636 INFO:teuthology.orchestra.run.smithi106.stdout: 1, 2024-04-25T10:31:54.636 INFO:teuthology.orchestra.run.smithi106.stdout: 2 2024-04-25T10:31:54.636 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:31:54.636 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_age": 5, 2024-04-25T10:31:54.636 INFO:teuthology.orchestra.run.smithi106.stdout: "features": { 2024-04-25T10:31:54.637 INFO:teuthology.orchestra.run.smithi106.stdout: "required_con": "2450521708007866372", 2024-04-25T10:31:54.637 INFO:teuthology.orchestra.run.smithi106.stdout: "required_mon": [ 2024-04-25T10:31:54.637 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:31:54.637 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:31:54.637 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:31:54.637 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:31:54.637 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:31:54.637 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:31:54.637 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:31:54.637 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:31:54.637 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:31:54.637 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:31:54.637 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:31:54.638 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:31:54.638 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_con": "4540701547738038271", 2024-04-25T10:31:54.638 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_mon": [ 2024-04-25T10:31:54.638 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:31:54.638 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:31:54.638 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:31:54.638 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:31:54.638 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:31:54.638 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:31:54.638 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:31:54.638 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:31:54.638 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:31:54.638 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:31:54.639 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:31:54.639 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:31:54.639 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:54.639 INFO:teuthology.orchestra.run.smithi106.stdout: "outside_quorum": [], 2024-04-25T10:31:54.639 INFO:teuthology.orchestra.run.smithi106.stdout: "extra_probe_peers": [], 2024-04-25T10:31:54.639 INFO:teuthology.orchestra.run.smithi106.stdout: "sync_provider": [], 2024-04-25T10:31:54.639 INFO:teuthology.orchestra.run.smithi106.stdout: "monmap": { 2024-04-25T10:31:54.639 INFO:teuthology.orchestra.run.smithi106.stdout: "epoch": 1, 2024-04-25T10:31:54.639 INFO:teuthology.orchestra.run.smithi106.stdout: "fsid": "3ee77985-e8e4-41cf-81e9-7c774cf7891f", 2024-04-25T10:31:54.639 INFO:teuthology.orchestra.run.smithi106.stdout: "modified": "2024-04-25T10:29:33.446228Z", 2024-04-25T10:31:54.639 INFO:teuthology.orchestra.run.smithi106.stdout: "created": "2024-04-25T10:29:33.446228Z", 2024-04-25T10:31:54.639 INFO:teuthology.orchestra.run.smithi106.stdout: "min_mon_release": 19, 2024-04-25T10:31:54.639 INFO:teuthology.orchestra.run.smithi106.stdout: "min_mon_release_name": "squid", 2024-04-25T10:31:54.639 INFO:teuthology.orchestra.run.smithi106.stdout: "election_strategy": 1, 2024-04-25T10:31:54.640 INFO:teuthology.orchestra.run.smithi106.stdout: "disallowed_leaders: ": "", 2024-04-25T10:31:54.640 INFO:teuthology.orchestra.run.smithi106.stdout: "stretch_mode": false, 2024-04-25T10:31:54.640 INFO:teuthology.orchestra.run.smithi106.stdout: "tiebreaker_mon": "", 2024-04-25T10:31:54.640 INFO:teuthology.orchestra.run.smithi106.stdout: "removed_ranks: ": "", 2024-04-25T10:31:54.640 INFO:teuthology.orchestra.run.smithi106.stdout: "features": { 2024-04-25T10:31:54.640 INFO:teuthology.orchestra.run.smithi106.stdout: "persistent": [ 2024-04-25T10:31:54.640 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:31:54.640 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:31:54.640 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:31:54.640 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:31:54.640 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:31:54.640 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:31:54.640 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:31:54.641 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:31:54.641 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:31:54.641 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:31:54.641 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:31:54.641 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:31:54.641 INFO:teuthology.orchestra.run.smithi106.stdout: "optional": [] 2024-04-25T10:31:54.641 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:54.641 INFO:teuthology.orchestra.run.smithi106.stdout: "mons": [ 2024-04-25T10:31:54.641 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:54.641 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 0, 2024-04-25T10:31:54.641 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "a", 2024-04-25T10:31:54.641 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:31:54.641 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:31:54.642 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:54.642 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:31:54.642 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:3300", 2024-04-25T10:31:54.642 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:31:54.642 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:54.642 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:54.642 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:31:54.642 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:6789", 2024-04-25T10:31:54.642 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:31:54.642 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:31:54.642 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:31:54.642 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:54.642 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:6789/0", 2024-04-25T10:31:54.642 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.106:6789/0", 2024-04-25T10:31:54.643 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:31:54.643 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:31:54.643 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:31:54.643 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:54.643 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:54.643 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 1, 2024-04-25T10:31:54.643 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "b", 2024-04-25T10:31:54.643 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:31:54.643 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:31:54.644 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:54.644 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:31:54.644 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:3300", 2024-04-25T10:31:54.644 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:31:54.644 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:54.644 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:54.644 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:31:54.644 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6789", 2024-04-25T10:31:54.644 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:31:54.644 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:31:54.644 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:31:54.644 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:54.644 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6789/0", 2024-04-25T10:31:54.645 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.134:6789/0", 2024-04-25T10:31:54.645 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:31:54.645 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:31:54.645 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:31:54.645 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:54.645 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:54.645 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 2, 2024-04-25T10:31:54.645 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "c", 2024-04-25T10:31:54.645 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:31:54.645 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:31:54.645 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:54.645 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:31:54.645 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:3301", 2024-04-25T10:31:54.646 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:31:54.646 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:54.646 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:54.646 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:31:54.646 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6790", 2024-04-25T10:31:54.646 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:31:54.646 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:31:54.646 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:31:54.646 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:54.646 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6790/0", 2024-04-25T10:31:54.646 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.134:6790/0", 2024-04-25T10:31:54.646 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:31:54.647 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:31:54.647 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:31:54.647 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:31:54.647 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:31:54.647 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:54.647 INFO:teuthology.orchestra.run.smithi106.stdout: "feature_map": { 2024-04-25T10:31:54.647 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": [ 2024-04-25T10:31:54.647 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:54.647 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:31:54.647 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:31:54.647 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 1 2024-04-25T10:31:54.647 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:31:54.647 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:31:54.647 INFO:teuthology.orchestra.run.smithi106.stdout: "client": [ 2024-04-25T10:31:54.648 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:31:54.648 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:31:54.648 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:31:54.648 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 1 2024-04-25T10:31:54.648 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:31:54.648 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:31:54.648 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:31:54.648 INFO:teuthology.orchestra.run.smithi106.stdout: "stretch_mode": false 2024-04-25T10:31:54.648 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-25T10:31:54.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.633+0000 7f866aa6e640 1 -- 172.21.15.106:0/3380696760 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f86480744e0 msgr2=0x7f86480769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:54.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.633+0000 7f866aa6e640 1 --2- 172.21.15.106:0/3380696760 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f86480744e0 0x7f86480769f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8650007fd0 tx=0x7f8650009040 comp rx=0 tx=0).stop 2024-04-25T10:31:54.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.633+0000 7f866aa6e640 1 -- 172.21.15.106:0/3380696760 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f866415d9a0 msgr2=0x7f86640ce690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:54.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.633+0000 7f866aa6e640 1 --2- 172.21.15.106:0/3380696760 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f866415d9a0 0x7f86640ce690 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f865c02ce80 tx=0x7f865c004a40 comp rx=0 tx=0).stop 2024-04-25T10:31:54.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.633+0000 7f866aa6e640 1 -- 172.21.15.106:0/3380696760 shutdown_connections 2024-04-25T10:31:54.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.633+0000 7f866aa6e640 1 --2- 172.21.15.106:0/3380696760 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f86480744e0 0x7f86480769f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:54.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.633+0000 7f866aa6e640 1 --2- 172.21.15.106:0/3380696760 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f866415d9a0 0x7f86640ce690 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:54.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.633+0000 7f866aa6e640 1 -- 172.21.15.106:0/3380696760 >> 172.21.15.106:0/3380696760 conn(0x7f866406c9b0 msgr2=0x7f8664072d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:54.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.633+0000 7f866aa6e640 1 -- 172.21.15.106:0/3380696760 shutdown_connections 2024-04-25T10:31:54.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.633+0000 7f866aa6e640 1 -- 172.21.15.106:0/3380696760 wait complete. 2024-04-25T10:31:54.651 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T10:31:54.651 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-04-25T10:31:54.651 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-04-25T10:31:54.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.776+0000 7fa5e6b72640 1 Processor -- start 2024-04-25T10:31:54.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.776+0000 7fa5e6b72640 1 -- start start 2024-04-25T10:31:54.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.777+0000 7fa5e6b72640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5e006f390 0x7fa5e006dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:54.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.777+0000 7fa5e6b72640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fa5e00c6250 0x7fa5e00c6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:54.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.777+0000 7fa5e6b72640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5e015d750 con 0x7fa5e01570d0 2024-04-25T10:31:54.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.777+0000 7fa5e6b72640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fa5e015dcf0 con 0x7fa5e00c6250 2024-04-25T10:31:54.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.777+0000 7fa5e6b72640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa5e006ef00 con 0x7fa5e006f390 2024-04-25T10:31:54.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.777+0000 7fa5dffff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fa5e01570d0 0x7fa5e00ca960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:47402/0 (socket says 172.21.15.106:47402) 2024-04-25T10:31:54.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.777+0000 7fa5dffff640 1 -- 172.21.15.106:0/1932625784 learned_addr learned my addr 172.21.15.106:0/1932625784 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:31:54.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.777+0000 7fa5e490e640 1 --2- 172.21.15.106:0/1932625784 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5e006f390 0x7fa5e006dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:54.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.777+0000 7fa5e510f640 1 --2- 172.21.15.106:0/1932625784 >> v2:172.21.15.134:3300/0 conn(0x7fa5e00c6250 0x7fa5e00c6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:54.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.777+0000 7fa5df7fe640 1 -- 172.21.15.106:0/1932625784 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2291018030 0 0) 0x7fa5e015d750 con 0x7fa5e01570d0 2024-04-25T10:31:54.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.778+0000 7fa5df7fe640 1 -- 172.21.15.106:0/1932625784 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa5cc003520 con 0x7fa5e01570d0 2024-04-25T10:31:54.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.778+0000 7fa5df7fe640 1 -- 172.21.15.106:0/1932625784 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3415644823 0 0) 0x7fa5cc003520 con 0x7fa5e01570d0 2024-04-25T10:31:54.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.778+0000 7fa5df7fe640 1 -- 172.21.15.106:0/1932625784 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5e006f390 msgr2=0x7fa5e006dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:54.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.778+0000 7fa5df7fe640 1 --2- 172.21.15.106:0/1932625784 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5e006f390 0x7fa5e006dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:54.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.778+0000 7fa5df7fe640 1 -- 172.21.15.106:0/1932625784 >> v2:172.21.15.134:3300/0 conn(0x7fa5e00c6250 msgr2=0x7fa5e00c6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:54.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.778+0000 7fa5df7fe640 1 --2- 172.21.15.106:0/1932625784 >> v2:172.21.15.134:3300/0 conn(0x7fa5e00c6250 0x7fa5e00c6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:54.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.778+0000 7fa5df7fe640 1 -- 172.21.15.106:0/1932625784 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5e015da10 con 0x7fa5e01570d0 2024-04-25T10:31:54.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.778+0000 7fa5df7fe640 1 -- 172.21.15.106:0/1932625784 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3776459350 0 0) 0x7fa5d0001690 con 0x7fa5e01570d0 2024-04-25T10:31:54.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.778+0000 7fa5df7fe640 1 -- 172.21.15.106:0/1932625784 >> v1:172.21.15.106:6789/0 conn(0x7fa5e01570d0 legacy=0x7fa5e00ca960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:54.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.778+0000 7fa5df7fe640 1 --2- 172.21.15.106:0/1932625784 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5cc0039b0 0x7fa5cc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:54.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.778+0000 7fa5df7fe640 1 --2- 172.21.15.106:0/1932625784 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5cc004800 0x7fa5cc004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:54.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.778+0000 7fa5df7fe640 1 --2- 172.21.15.106:0/1932625784 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5e006f390 0x7fa5cc005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:54.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.778+0000 7fa5df7fe640 1 -- 172.21.15.106:0/1932625784 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e015dcf0 con 0x7fa5cc0039b0 2024-04-25T10:31:54.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.778+0000 7fa5df7fe640 1 -- 172.21.15.106:0/1932625784 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e006ef00 con 0x7fa5cc004800 2024-04-25T10:31:54.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.778+0000 7fa5df7fe640 1 -- 172.21.15.106:0/1932625784 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa5cc00ba40 con 0x7fa5e006f390 2024-04-25T10:31:54.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.778+0000 7fa5e490e640 1 --2- 172.21.15.106:0/1932625784 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5cc0039b0 0x7fa5cc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:54.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.778+0000 7fa5dffff640 1 --2- 172.21.15.106:0/1932625784 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5cc004800 0x7fa5cc004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:54.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.778+0000 7fa5e510f640 1 --2- 172.21.15.106:0/1932625784 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5e006f390 0x7fa5cc005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:54.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.778+0000 7fa5e490e640 1 -- 172.21.15.106:0/1932625784 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5e006f390 msgr2=0x7fa5cc005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:54.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.778+0000 7fa5e490e640 1 --2- 172.21.15.106:0/1932625784 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5e006f390 0x7fa5cc005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:54.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.778+0000 7fa5e490e640 1 -- 172.21.15.106:0/1932625784 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5cc004800 msgr2=0x7fa5cc004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:54.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.778+0000 7fa5e490e640 1 --2- 172.21.15.106:0/1932625784 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5cc004800 0x7fa5cc004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:54.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.778+0000 7fa5e490e640 1 -- 172.21.15.106:0/1932625784 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5cc003520 con 0x7fa5cc0039b0 2024-04-25T10:31:54.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.779+0000 7fa5e490e640 1 --2- 172.21.15.106:0/1932625784 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5cc0039b0 0x7fa5cc003e20 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa5c8009ac0 tx=0x7fa5cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df6bdc67c6fa63ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:54.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.779+0000 7fa5df7fe640 1 -- 172.21.15.106:0/1932625784 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c8036070 con 0x7fa5cc0039b0 2024-04-25T10:31:54.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.779+0000 7fa5df7fe640 1 -- 172.21.15.106:0/1932625784 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5c8031400 con 0x7fa5cc0039b0 2024-04-25T10:31:54.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.779+0000 7fa5df7fe640 1 -- 172.21.15.106:0/1932625784 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c8009690 con 0x7fa5cc0039b0 2024-04-25T10:31:54.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.780+0000 7fa5e6b72640 1 -- 172.21.15.106:0/1932625784 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5cc0039b0 msgr2=0x7fa5cc003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:54.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.780+0000 7fa5e6b72640 1 --2- 172.21.15.106:0/1932625784 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5cc0039b0 0x7fa5cc003e20 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa5c8009ac0 tx=0x7fa5cc005830 comp rx=0 tx=0).stop 2024-04-25T10:31:54.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.780+0000 7fa5e510f640 1 -- 172.21.15.106:0/1932625784 reap_dead start 2024-04-25T10:31:54.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.780+0000 7fa5e6b72640 1 -- 172.21.15.106:0/1932625784 shutdown_connections 2024-04-25T10:31:54.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.780+0000 7fa5e6b72640 1 -- 172.21.15.106:0/1932625784 >> 172.21.15.106:0/1932625784 conn(0x7fa5e00cff60 msgr2=0x7fa5e00d0370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:54.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.780+0000 7fa5e6b72640 1 -- 172.21.15.106:0/1932625784 shutdown_connections 2024-04-25T10:31:54.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.780+0000 7fa5e6b72640 1 -- 172.21.15.106:0/1932625784 wait complete. 2024-04-25T10:31:54.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.782+0000 7fa5e6b72640 1 Processor -- start 2024-04-25T10:31:54.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.782+0000 7fa5e6b72640 1 -- start start 2024-04-25T10:31:54.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.783+0000 7fa5e6b72640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5cc0039b0 0x7fa5e0152aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:54.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.783+0000 7fa5e6b72640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5e0160510 0x7fa5e01f6f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:54.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.783+0000 7fa5e6b72640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5e0152370 0x7fa5e00ce000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:54.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.783+0000 7fa5e6b72640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e006eb90 con 0x7fa5cc0039b0 2024-04-25T10:31:54.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.783+0000 7fa5e6b72640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e006e8b0 con 0x7fa5e0160510 2024-04-25T10:31:54.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.783+0000 7fa5e6b72640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa5e006d840 con 0x7fa5e0152370 2024-04-25T10:31:54.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.783+0000 7fa5e490e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5cc0039b0 0x7fa5e0152aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:54.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.783+0000 7fa5e490e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5cc0039b0 0x7fa5e0152aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49526/0 (socket says 172.21.15.106:49526) 2024-04-25T10:31:54.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.783+0000 7fa5e490e640 1 -- 172.21.15.106:0/933009378 learned_addr learned my addr 172.21.15.106:0/933009378 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:31:54.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.783+0000 7fa5dffff640 1 --2- 172.21.15.106:0/933009378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5e0160510 0x7fa5e01f6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:54.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.783+0000 7fa5e510f640 1 --2- 172.21.15.106:0/933009378 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5e0152370 0x7fa5e00ce000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:54.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.784+0000 7fa5e490e640 1 -- 172.21.15.106:0/933009378 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5e0152370 msgr2=0x7fa5e00ce000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:54.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.784+0000 7fa5e490e640 1 --2- 172.21.15.106:0/933009378 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5e0152370 0x7fa5e00ce000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:54.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.784+0000 7fa5e490e640 1 -- 172.21.15.106:0/933009378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5e0160510 msgr2=0x7fa5e01f6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:54.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.784+0000 7fa5e490e640 1 --2- 172.21.15.106:0/933009378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5e0160510 0x7fa5e01f6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:54.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.784+0000 7fa5e490e640 1 -- 172.21.15.106:0/933009378 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5e0160fe0 con 0x7fa5cc0039b0 2024-04-25T10:31:54.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.784+0000 7fa5e490e640 1 --2- 172.21.15.106:0/933009378 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5cc0039b0 0x7fa5e0152aa0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa5c8009be0 tx=0x7fa5c8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:54.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.784+0000 7fa5dd7fa640 1 -- 172.21.15.106:0/933009378 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c8040020 con 0x7fa5cc0039b0 2024-04-25T10:31:54.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.784+0000 7fa5dd7fa640 1 -- 172.21.15.106:0/933009378 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5c8003bc0 con 0x7fa5cc0039b0 2024-04-25T10:31:54.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.784+0000 7fa5e6b72640 1 -- 172.21.15.106:0/933009378 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5e01f64e0 con 0x7fa5cc0039b0 2024-04-25T10:31:54.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.784+0000 7fa5dd7fa640 1 -- 172.21.15.106:0/933009378 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c8036040 con 0x7fa5cc0039b0 2024-04-25T10:31:54.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.784+0000 7fa5e6b72640 1 -- 172.21.15.106:0/933009378 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5e01f6780 con 0x7fa5cc0039b0 2024-04-25T10:31:54.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.785+0000 7fa5e6b72640 1 -- 172.21.15.106:0/933009378 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5e006a3d0 con 0x7fa5cc0039b0 2024-04-25T10:31:54.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.786+0000 7fa5dd7fa640 1 -- 172.21.15.106:0/933009378 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa5c8043020 con 0x7fa5cc0039b0 2024-04-25T10:31:54.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.786+0000 7fa5dd7fa640 1 --2- 172.21.15.106:0/933009378 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fa5c00744b0 0x7fa5c00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:54.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.786+0000 7fa5dffff640 1 --2- 172.21.15.106:0/933009378 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fa5c00744b0 0x7fa5c00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:54.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.786+0000 7fa5dd7fa640 1 -- 172.21.15.106:0/933009378 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fa5c80b52d0 con 0x7fa5cc0039b0 2024-04-25T10:31:54.791 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.787+0000 7fa5dffff640 1 --2- 172.21.15.106:0/933009378 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fa5c00744b0 0x7fa5c00769c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa5e0153df0 tx=0x7fa5d0007970 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:54.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:54.791+0000 7fa5dd7fa640 1 -- 172.21.15.106:0/933009378 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa5c804a070 con 0x7fa5cc0039b0 2024-04-25T10:31:55.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:55.034+0000 7fa5e6b72640 1 -- 172.21.15.106:0/933009378 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7fa5e015a6f0 con 0x7fa5cc0039b0 2024-04-25T10:31:55.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:55.034+0000 7fa5dd7fa640 1 -- 172.21.15.106:0/933009378 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7fa5c80044e0 con 0x7fa5cc0039b0 2024-04-25T10:31:55.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:55.036+0000 7fa5e6b72640 1 -- 172.21.15.106:0/933009378 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fa5c00744b0 msgr2=0x7fa5c00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:55.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:55.036+0000 7fa5e6b72640 1 --2- 172.21.15.106:0/933009378 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fa5c00744b0 0x7fa5c00769c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa5e0153df0 tx=0x7fa5d0007970 comp rx=0 tx=0).stop 2024-04-25T10:31:55.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:55.036+0000 7fa5e6b72640 1 -- 172.21.15.106:0/933009378 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5cc0039b0 msgr2=0x7fa5e0152aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:55.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:55.036+0000 7fa5e6b72640 1 --2- 172.21.15.106:0/933009378 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5cc0039b0 0x7fa5e0152aa0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa5c8009be0 tx=0x7fa5c8004730 comp rx=0 tx=0).stop 2024-04-25T10:31:55.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:55.036+0000 7fa5e6b72640 1 -- 172.21.15.106:0/933009378 shutdown_connections 2024-04-25T10:31:55.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:55.036+0000 7fa5e6b72640 1 --2- 172.21.15.106:0/933009378 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fa5c00744b0 0x7fa5c00769c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:55.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:55.036+0000 7fa5e6b72640 1 --2- 172.21.15.106:0/933009378 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5e0152370 0x7fa5e00ce000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:55.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:55.036+0000 7fa5e6b72640 1 --2- 172.21.15.106:0/933009378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5e0160510 0x7fa5e01f6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:55.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:55.036+0000 7fa5e6b72640 1 --2- 172.21.15.106:0/933009378 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5cc0039b0 0x7fa5e0152aa0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:55.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:55.036+0000 7fa5e6b72640 1 -- 172.21.15.106:0/933009378 >> 172.21.15.106:0/933009378 conn(0x7fa5e00cff60 msgr2=0x7fa5e01562d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:55.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:55.036+0000 7fa5e6b72640 1 -- 172.21.15.106:0/933009378 shutdown_connections 2024-04-25T10:31:55.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:31:55.036+0000 7fa5e6b72640 1 -- 172.21.15.106:0/933009378 wait complete. 2024-04-25T10:31:55.050 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-04-25T10:32:02.026 DEBUG:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:32:02.029 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:32:05.051 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-04-25T10:32:05.051 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T10:32:05.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.157+0000 7f666b8aa640 1 Processor -- start 2024-04-25T10:32:05.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.157+0000 7f666b8aa640 1 -- start start 2024-04-25T10:32:05.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.159+0000 7f666b8aa640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f66640c9920 0x7f6664156c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:05.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.159+0000 7f666b8aa640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6664074630 0x7f66640738d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:05.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.159+0000 7f666b8aa640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6664160340 con 0x7f666415d650 2024-04-25T10:32:05.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.159+0000 7f666b8aa640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f666415dbe0 con 0x7f66640c9920 2024-04-25T10:32:05.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.159+0000 7f666b8aa640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6664160620 con 0x7f6664074630 2024-04-25T10:32:05.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.159+0000 7f6668e45640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6664074630 0x7f66640738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:05.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.159+0000 7f6669646640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f66640c9920 0x7f6664156c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:05.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.159+0000 7f6669e47640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f666415d650 0x7f66640c7a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.106:45768/0 (socket says 172.21.15.106:45768) 2024-04-25T10:32:05.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.159+0000 7f6669e47640 1 -- 172.21.15.106:0/2196246284 learned_addr learned my addr 172.21.15.106:0/2196246284 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:32:05.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.159+0000 7f665bfff640 1 -- 172.21.15.106:0/2196246284 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2773861218 0 0) 0x7f6664160340 con 0x7f666415d650 2024-04-25T10:32:05.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.159+0000 7f665bfff640 1 -- 172.21.15.106:0/2196246284 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6650003520 con 0x7f666415d650 2024-04-25T10:32:05.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.159+0000 7f6669646640 1 -- 172.21.15.106:0/2196246284 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6664074630 msgr2=0x7f66640738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:05.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.159+0000 7f6669646640 1 --2- 172.21.15.106:0/2196246284 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6664074630 0x7f66640738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:05.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.159+0000 7f6669646640 1 -- 172.21.15.106:0/2196246284 >> v1:172.21.15.134:6789/0 conn(0x7f666415d650 legacy=0x7f66640c7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:05.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.159+0000 7f6669646640 1 -- 172.21.15.106:0/2196246284 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6664160900 con 0x7f66640c9920 2024-04-25T10:32:05.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.159+0000 7f6668e45640 1 --2- 172.21.15.106:0/2196246284 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6664074630 0x7f66640738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:32:05.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.160+0000 7f6669646640 1 --2- 172.21.15.106:0/2196246284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66640c9920 0x7f6664156c50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f66600097a0 tx=0x7f666002a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=574b39313a043cf3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:05.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.160+0000 7f665bfff640 1 -- 172.21.15.106:0/2196246284 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6660037070 con 0x7f66640c9920 2024-04-25T10:32:05.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.160+0000 7f665bfff640 1 -- 172.21.15.106:0/2196246284 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f666002ae50 con 0x7f66640c9920 2024-04-25T10:32:05.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.160+0000 7f665bfff640 1 -- 172.21.15.106:0/2196246284 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6660032640 con 0x7f66640c9920 2024-04-25T10:32:05.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.161+0000 7f666b8aa640 1 -- 172.21.15.106:0/2196246284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66640c9920 msgr2=0x7f6664156c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:05.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.161+0000 7f666b8aa640 1 --2- 172.21.15.106:0/2196246284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66640c9920 0x7f6664156c50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f66600097a0 tx=0x7f666002a100 comp rx=0 tx=0).stop 2024-04-25T10:32:05.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.161+0000 7f666b8aa640 1 -- 172.21.15.106:0/2196246284 shutdown_connections 2024-04-25T10:32:05.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.161+0000 7f666b8aa640 1 --2- 172.21.15.106:0/2196246284 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6664074630 0x7f66640738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:05.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.161+0000 7f666b8aa640 1 --2- 172.21.15.106:0/2196246284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66640c9920 0x7f6664156c50 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:05.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.161+0000 7f666b8aa640 1 -- 172.21.15.106:0/2196246284 >> 172.21.15.106:0/2196246284 conn(0x7f666406d6d0 msgr2=0x7f666406dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:05.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.161+0000 7f666b8aa640 1 -- 172.21.15.106:0/2196246284 shutdown_connections 2024-04-25T10:32:05.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.161+0000 7f666b8aa640 1 -- 172.21.15.106:0/2196246284 wait complete. 2024-04-25T10:32:05.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.163+0000 7f666b8aa640 1 Processor -- start 2024-04-25T10:32:05.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.163+0000 7f666b8aa640 1 -- start start 2024-04-25T10:32:05.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.164+0000 7f666b8aa640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6664074630 0x7f6664151230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:05.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.164+0000 7f666b8aa640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66640c9920 0x7f666415e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:05.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.164+0000 7f666b8aa640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f666415d650 0x7f666414c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:05.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.164+0000 7f666b8aa640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f66640c8190 con 0x7f666415d650 2024-04-25T10:32:05.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.164+0000 7f666b8aa640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f666406e280 con 0x7f66640c9920 2024-04-25T10:32:05.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.164+0000 7f666b8aa640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f66640c8310 con 0x7f6664074630 2024-04-25T10:32:05.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.165+0000 7f6669e47640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f666415d650 0x7f666414c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:05.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.165+0000 7f6669e47640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f666415d650 0x7f666414c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53032/0 (socket says 172.21.15.106:53032) 2024-04-25T10:32:05.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.165+0000 7f6669e47640 1 -- 172.21.15.106:0/3714729900 learned_addr learned my addr 172.21.15.106:0/3714729900 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:32:05.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.165+0000 7f6668e45640 1 --2- 172.21.15.106:0/3714729900 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66640c9920 0x7f666415e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:05.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.165+0000 7f6669646640 1 --2- 172.21.15.106:0/3714729900 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6664074630 0x7f6664151230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:05.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.165+0000 7f6669e47640 1 -- 172.21.15.106:0/3714729900 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6664074630 msgr2=0x7f6664151230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:05.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.165+0000 7f6669e47640 1 --2- 172.21.15.106:0/3714729900 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6664074630 0x7f6664151230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:05.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.165+0000 7f6669e47640 1 -- 172.21.15.106:0/3714729900 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66640c9920 msgr2=0x7f666415e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:05.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.165+0000 7f6669e47640 1 --2- 172.21.15.106:0/3714729900 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66640c9920 0x7f666415e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:05.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.165+0000 7f6669e47640 1 -- 172.21.15.106:0/3714729900 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6664151770 con 0x7f666415d650 2024-04-25T10:32:05.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.165+0000 7f6669e47640 1 --2- 172.21.15.106:0/3714729900 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f666415d650 0x7f666414c910 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f665c00caa0 tx=0x7f665c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:05.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.165+0000 7f6659ffb640 1 -- 172.21.15.106:0/3714729900 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665c017070 con 0x7f666415d650 2024-04-25T10:32:05.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.165+0000 7f6659ffb640 1 -- 172.21.15.106:0/3714729900 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f665c00ab40 con 0x7f666415d650 2024-04-25T10:32:05.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.165+0000 7f666b8aa640 1 -- 172.21.15.106:0/3714729900 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f666415f3b0 con 0x7f666415d650 2024-04-25T10:32:05.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.165+0000 7f6659ffb640 1 -- 172.21.15.106:0/3714729900 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665c012400 con 0x7f666415d650 2024-04-25T10:32:05.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.166+0000 7f666b8aa640 1 -- 172.21.15.106:0/3714729900 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6664150a30 con 0x7f666415d650 2024-04-25T10:32:05.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.167+0000 7f666b8aa640 1 -- 172.21.15.106:0/3714729900 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f666406a3d0 con 0x7f666415d650 2024-04-25T10:32:05.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.168+0000 7f6659ffb640 1 -- 172.21.15.106:0/3714729900 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f665c01a070 con 0x7f666415d650 2024-04-25T10:32:05.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.169+0000 7f6659ffb640 1 --2- 172.21.15.106:0/3714729900 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f66480744b0 0x7f66480769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:05.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.169+0000 7f6669646640 1 --2- 172.21.15.106:0/3714729900 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f66480744b0 0x7f66480769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:05.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.169+0000 7f6659ffb640 1 -- 172.21.15.106:0/3714729900 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f665c094ea0 con 0x7f666415d650 2024-04-25T10:32:05.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.169+0000 7f6669646640 1 --2- 172.21.15.106:0/3714729900 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f66480744b0 0x7f66480769c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6660034000 tx=0x7f666002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:05.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.174+0000 7f6659ffb640 1 -- 172.21.15.106:0/3714729900 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f665c05eff0 con 0x7f666415d650 2024-04-25T10:32:05.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.427+0000 7f666b8aa640 1 -- 172.21.15.106:0/3714729900 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f6664074410 con 0x7f666415d650 2024-04-25T10:32:05.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.427+0000 7f6659ffb640 1 -- 172.21.15.106:0/3714729900 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1569 (secure 0 0 0) 0x7f665c0632f0 con 0x7f666415d650 2024-04-25T10:32:05.430 INFO:teuthology.orchestra.run.smithi106.stdout:{"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"3ee77985-e8e4-41cf-81e9-7c774cf7891f","modified":"2024-04-25T10:29:33.446228Z","created":"2024-04-25T10:29:33.446228Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:3300","nonce":0},{"type":"v1","addr":"172.21.15.106:6789","nonce":0}]},"addr":"172.21.15.106:6789/0","public_addr":"172.21.15.106:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T10:32:05.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.429+0000 7f666b8aa640 1 -- 172.21.15.106:0/3714729900 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f66480744b0 msgr2=0x7f66480769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:05.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.429+0000 7f666b8aa640 1 --2- 172.21.15.106:0/3714729900 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f66480744b0 0x7f66480769c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6660034000 tx=0x7f666002e000 comp rx=0 tx=0).stop 2024-04-25T10:32:05.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.429+0000 7f666b8aa640 1 -- 172.21.15.106:0/3714729900 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f666415d650 msgr2=0x7f666414c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:05.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.429+0000 7f666b8aa640 1 --2- 172.21.15.106:0/3714729900 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f666415d650 0x7f666414c910 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f665c00caa0 tx=0x7f665c00cf60 comp rx=0 tx=0).stop 2024-04-25T10:32:05.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.429+0000 7f666b8aa640 1 -- 172.21.15.106:0/3714729900 shutdown_connections 2024-04-25T10:32:05.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.429+0000 7f666b8aa640 1 --2- 172.21.15.106:0/3714729900 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f66480744b0 0x7f66480769c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:05.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.429+0000 7f666b8aa640 1 --2- 172.21.15.106:0/3714729900 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f666415d650 0x7f666414c910 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:05.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.429+0000 7f666b8aa640 1 --2- 172.21.15.106:0/3714729900 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66640c9920 0x7f666415e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:05.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.429+0000 7f666b8aa640 1 --2- 172.21.15.106:0/3714729900 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6664074630 0x7f6664151230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:05.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.429+0000 7f666b8aa640 1 -- 172.21.15.106:0/3714729900 >> 172.21.15.106:0/3714729900 conn(0x7f666406d6d0 msgr2=0x7f66640c8b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:05.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.429+0000 7f666b8aa640 1 -- 172.21.15.106:0/3714729900 shutdown_connections 2024-04-25T10:32:05.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.429+0000 7f666b8aa640 1 -- 172.21.15.106:0/3714729900 wait complete. 2024-04-25T10:32:05.443 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-04-25T10:32:05.443 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-04-25T10:32:05.443 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-04-25T10:32:05.443 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-25T10:32:05.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.550+0000 7f66b4c30640 1 Processor -- start 2024-04-25T10:32:05.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.550+0000 7f66b4c30640 1 -- start start 2024-04-25T10:32:05.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.552+0000 7f66b4c30640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f66b00c8d90 0x7f66b0156ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:05.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.552+0000 7f66b4c30640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f66b00b4ce0 0x7f66b015ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:05.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.552+0000 7f66b4c30640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66b00ce630 con 0x7f66b00c7e30 2024-04-25T10:32:05.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.552+0000 7f66b4c30640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f66b00b50f0 con 0x7f66b00b4ce0 2024-04-25T10:32:05.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.552+0000 7f66b4c30640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f66b00c7c60 con 0x7f66b00c8d90 2024-04-25T10:32:05.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.552+0000 7f66aed9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f66b00b4ce0 0x7f66b015ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:05.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.552+0000 7f66aed9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f66b00b4ce0 0x7f66b015ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53038/0 (socket says 172.21.15.106:53038) 2024-04-25T10:32:05.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.552+0000 7f66aed9d640 1 -- 172.21.15.106:0/3548388613 learned_addr learned my addr 172.21.15.106:0/3548388613 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:32:05.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.552+0000 7f66ae59c640 1 --1- 172.21.15.106:0/3548388613 >> v1:172.21.15.106:6789/0 conn(0x7f66b00c7e30 0x7f66b006d000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:57822/0 (socket says 172.21.15.106:57822) 2024-04-25T10:32:05.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.552+0000 7f66add9b640 1 --2- 172.21.15.106:0/3548388613 >> v2:172.21.15.134:3300/0 conn(0x7f66b00c8d90 0x7f66b0156ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:05.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.552+0000 7f66aed9d640 1 -- 172.21.15.106:0/3548388613 >> v2:172.21.15.134:3300/0 conn(0x7f66b00c8d90 msgr2=0x7f66b0156ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:05.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.552+0000 7f66aed9d640 1 --2- 172.21.15.106:0/3548388613 >> v2:172.21.15.134:3300/0 conn(0x7f66b00c8d90 0x7f66b0156ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:05.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.552+0000 7f66aed9d640 1 -- 172.21.15.106:0/3548388613 >> v1:172.21.15.106:6789/0 conn(0x7f66b00c7e30 legacy=0x7f66b006d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:05.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.552+0000 7f66aed9d640 1 -- 172.21.15.106:0/3548388613 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66b00c7980 con 0x7f66b00b4ce0 2024-04-25T10:32:05.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.553+0000 7f66aed9d640 1 --2- 172.21.15.106:0/3548388613 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66b00b4ce0 0x7f66b015ce30 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f66b008ffe0 tx=0x7f66a802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fc0eee82806cd67 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:05.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.553+0000 7f66ad59a640 1 -- 172.21.15.106:0/3548388613 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a8036070 con 0x7f66b00b4ce0 2024-04-25T10:32:05.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.553+0000 7f66ad59a640 1 -- 172.21.15.106:0/3548388613 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66a8031600 con 0x7f66b00b4ce0 2024-04-25T10:32:05.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.553+0000 7f66ad59a640 1 -- 172.21.15.106:0/3548388613 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a80319d0 con 0x7f66b00b4ce0 2024-04-25T10:32:05.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.554+0000 7f66b4c30640 1 -- 172.21.15.106:0/3548388613 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66b00b4ce0 msgr2=0x7f66b015ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:05.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.554+0000 7f66b4c30640 1 --2- 172.21.15.106:0/3548388613 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66b00b4ce0 0x7f66b015ce30 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f66b008ffe0 tx=0x7f66a802a6a0 comp rx=0 tx=0).stop 2024-04-25T10:32:05.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.554+0000 7f66b4c30640 1 -- 172.21.15.106:0/3548388613 shutdown_connections 2024-04-25T10:32:05.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.554+0000 7f66b4c30640 1 --2- 172.21.15.106:0/3548388613 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66b00b4ce0 0x7f66b015ce30 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:05.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.554+0000 7f66b4c30640 1 --2- 172.21.15.106:0/3548388613 >> v2:172.21.15.134:3300/0 conn(0x7f66b00c8d90 0x7f66b0156ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:05.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.554+0000 7f66b4c30640 1 -- 172.21.15.106:0/3548388613 >> 172.21.15.106:0/3548388613 conn(0x7f66b0068b10 msgr2=0x7f66b00d1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:05.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.554+0000 7f66b4c30640 1 -- 172.21.15.106:0/3548388613 shutdown_connections 2024-04-25T10:32:05.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.554+0000 7f66b4c30640 1 -- 172.21.15.106:0/3548388613 wait complete. 2024-04-25T10:32:05.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.556+0000 7f66b4c30640 1 Processor -- start 2024-04-25T10:32:05.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.556+0000 7f66b4c30640 1 -- start start 2024-04-25T10:32:05.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.557+0000 7f66b4c30640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66b00b4ce0 0x7f66b014fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:05.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.557+0000 7f66b4c30640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66b00c7e30 0x7f66b01f2c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:05.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.557+0000 7f66b4c30640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66b00c8d90 0x7f66b00cd020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:05.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.557+0000 7f66b4c30640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f66b006d710 con 0x7f66b00b4ce0 2024-04-25T10:32:05.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.557+0000 7f66b4c30640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f66b00c72f0 con 0x7f66b00c8d90 2024-04-25T10:32:05.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.557+0000 7f66b4c30640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f66b006d890 con 0x7f66b00c7e30 2024-04-25T10:32:05.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.557+0000 7f66ae59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66b00b4ce0 0x7f66b014fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:05.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.557+0000 7f66ae59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66b00b4ce0 0x7f66b014fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53052/0 (socket says 172.21.15.106:53052) 2024-04-25T10:32:05.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.557+0000 7f66ae59c640 1 -- 172.21.15.106:0/1857513737 learned_addr learned my addr 172.21.15.106:0/1857513737 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:32:05.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.557+0000 7f66aed9d640 1 --2- 172.21.15.106:0/1857513737 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66b00c8d90 0x7f66b00cd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:05.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.557+0000 7f66add9b640 1 --2- 172.21.15.106:0/1857513737 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66b00c7e30 0x7f66b01f2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:05.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.558+0000 7f66ae59c640 1 -- 172.21.15.106:0/1857513737 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66b00c7e30 msgr2=0x7f66b01f2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:05.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.558+0000 7f66ae59c640 1 --2- 172.21.15.106:0/1857513737 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66b00c7e30 0x7f66b01f2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:05.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.558+0000 7f66ae59c640 1 -- 172.21.15.106:0/1857513737 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66b00c8d90 msgr2=0x7f66b00cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:05.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.558+0000 7f66ae59c640 1 --2- 172.21.15.106:0/1857513737 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66b00c8d90 0x7f66b00cd020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:05.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.558+0000 7f66ae59c640 1 -- 172.21.15.106:0/1857513737 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66b015b360 con 0x7f66b00b4ce0 2024-04-25T10:32:05.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.558+0000 7f66ae59c640 1 --2- 172.21.15.106:0/1857513737 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66b00b4ce0 0x7f66b014fff0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f66a000c3c0 tx=0x7f66a000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:05.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.558+0000 7f669effd640 1 -- 172.21.15.106:0/1857513737 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a00074e0 con 0x7f66b00b4ce0 2024-04-25T10:32:05.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.558+0000 7f669effd640 1 -- 172.21.15.106:0/1857513737 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66a0007680 con 0x7f66b00b4ce0 2024-04-25T10:32:05.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.558+0000 7f669effd640 1 -- 172.21.15.106:0/1857513737 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a0004a00 con 0x7f66b00b4ce0 2024-04-25T10:32:05.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.558+0000 7f66b4c30640 1 -- 172.21.15.106:0/1857513737 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66b01577c0 con 0x7f66b00b4ce0 2024-04-25T10:32:05.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.558+0000 7f66b4c30640 1 -- 172.21.15.106:0/1857513737 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66b015b8e0 con 0x7f66b00b4ce0 2024-04-25T10:32:05.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.559+0000 7f66b4c30640 1 --2- 172.21.15.106:0/1857513737 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66b015bab0 0x7f66b00cd560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:05.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.559+0000 7f66b4c30640 1 -- 172.21.15.106:0/1857513737 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f66b015abf0 con 0x7f66b015bab0 2024-04-25T10:32:05.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.559+0000 7f66add9b640 1 --2- 172.21.15.106:0/1857513737 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66b015bab0 0x7f66b00cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:05.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.559+0000 7f66add9b640 1 -- 172.21.15.106:0/1857513737 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f66b006a3d0 con 0x7f66b015bab0 2024-04-25T10:32:05.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.560+0000 7f66add9b640 1 --2- 172.21.15.106:0/1857513737 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66b015bab0 0x7f66b00cd560 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f66b006d070 tx=0x7f66a4011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:05.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.560+0000 7f669effd640 1 -- 172.21.15.106:0/1857513737 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f66a001a020 con 0x7f66b00b4ce0 2024-04-25T10:32:05.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.560+0000 7f669effd640 1 --2- 172.21.15.106:0/1857513737 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f66900744e0 0x7f66900769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:05.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.560+0000 7f669effd640 1 -- 172.21.15.106:0/1857513737 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f66a0094820 con 0x7f66b00b4ce0 2024-04-25T10:32:05.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.560+0000 7f66aed9d640 1 --2- 172.21.15.106:0/1857513737 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f66900744e0 0x7f66900769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:05.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.560+0000 7f669effd640 1 -- 172.21.15.106:0/1857513737 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a4019070 con 0x7f66b015bab0 2024-04-25T10:32:05.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.560+0000 7f669effd640 1 -- 172.21.15.106:0/1857513737 <== mon.0 v2:172.21.15.106:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f66a4004240 con 0x7f66b015bab0 2024-04-25T10:32:05.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.560+0000 7f669effd640 1 -- 172.21.15.106:0/1857513737 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66b015bab0 msgr2=0x7f66b00cd560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:05.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.560+0000 7f669effd640 1 --2- 172.21.15.106:0/1857513737 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66b015bab0 0x7f66b00cd560 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f66b006d070 tx=0x7f66a4011cf0 comp rx=0 tx=0).stop 2024-04-25T10:32:05.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.560+0000 7f669effd640 1 -- 172.21.15.106:0/1857513737 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66b015bab0 msgr2=0x7f66b00cd560 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T10:32:05.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.560+0000 7f669effd640 1 --2- 172.21.15.106:0/1857513737 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66b015bab0 0x7f66b00cd560 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:05.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.561+0000 7f66aed9d640 1 --2- 172.21.15.106:0/1857513737 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f66900744e0 0x7f66900769f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f66a8033000 tx=0x7f66a8009000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:05.575 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.573+0000 7f66b4c30640 1 --2- 172.21.15.106:0/1857513737 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66b015ae40 0x7f66b00d0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:05.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.573+0000 7f66b4c30640 1 -- 172.21.15.106:0/1857513737 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f66b015a5a0 con 0x7f66b015ae40 2024-04-25T10:32:05.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.573+0000 7f66add9b640 1 --2- 172.21.15.106:0/1857513737 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66b015ae40 0x7f66b00d0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:05.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.574+0000 7f66add9b640 1 -- 172.21.15.106:0/1857513737 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f66b00ce2e0 con 0x7f66b015ae40 2024-04-25T10:32:05.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.574+0000 7f66add9b640 1 --2- 172.21.15.106:0/1857513737 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66b015ae40 0x7f66b00d0320 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f66b006d070 tx=0x7f66a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:05.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.574+0000 7f669effd640 1 -- 172.21.15.106:0/1857513737 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a4019040 con 0x7f66b015ae40 2024-04-25T10:32:05.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.574+0000 7f669effd640 1 -- 172.21.15.106:0/1857513737 <== mon.0 v2:172.21.15.106:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5072 (secure 0 0 0) 0x7f66a4003dc0 con 0x7f66b015ae40 2024-04-25T10:32:05.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.574+0000 7f669effd640 1 -- 172.21.15.106:0/1857513737 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66b015ae40 msgr2=0x7f66b00d0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:05.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.574+0000 7f669effd640 1 --2- 172.21.15.106:0/1857513737 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66b015ae40 0x7f66b00d0320 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f66b006d070 tx=0x7f66a40046c0 comp rx=0 tx=0).stop 2024-04-25T10:32:05.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.574+0000 7f669effd640 1 -- 172.21.15.106:0/1857513737 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66b015ae40 msgr2=0x7f66b00d0320 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T10:32:05.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.574+0000 7f669effd640 1 --2- 172.21.15.106:0/1857513737 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66b015ae40 0x7f66b00d0320 secure :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f66b006d070 tx=0x7f66a40046c0 comp rx=0 tx=0).stop 2024-04-25T10:32:05.577 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-25T10:32:05.578 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "a", 2024-04-25T10:32:05.578 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 0, 2024-04-25T10:32:05.578 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "leader", 2024-04-25T10:32:05.578 INFO:teuthology.orchestra.run.smithi106.stdout: "election_epoch": 10, 2024-04-25T10:32:05.578 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 117813, 2024-04-25T10:32:05.578 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum": [ 2024-04-25T10:32:05.578 INFO:teuthology.orchestra.run.smithi106.stdout: 0, 2024-04-25T10:32:05.578 INFO:teuthology.orchestra.run.smithi106.stdout: 1, 2024-04-25T10:32:05.578 INFO:teuthology.orchestra.run.smithi106.stdout: 2 2024-04-25T10:32:05.578 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:32:05.578 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_age": 16, 2024-04-25T10:32:05.578 INFO:teuthology.orchestra.run.smithi106.stdout: "features": { 2024-04-25T10:32:05.578 INFO:teuthology.orchestra.run.smithi106.stdout: "required_con": "2450521708007866372", 2024-04-25T10:32:05.579 INFO:teuthology.orchestra.run.smithi106.stdout: "required_mon": [ 2024-04-25T10:32:05.579 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:32:05.579 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:32:05.579 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:32:05.579 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:32:05.579 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:32:05.579 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:32:05.579 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:32:05.579 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:32:05.579 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:32:05.579 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:32:05.579 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:32:05.579 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:32:05.579 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_con": "4540701547738038271", 2024-04-25T10:32:05.579 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_mon": [ 2024-04-25T10:32:05.580 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:32:05.580 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:32:05.580 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:32:05.580 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:32:05.580 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:32:05.580 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:32:05.580 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:32:05.580 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:32:05.580 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:32:05.580 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:32:05.580 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:32:05.580 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:32:05.580 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:05.580 INFO:teuthology.orchestra.run.smithi106.stdout: "outside_quorum": [], 2024-04-25T10:32:05.581 INFO:teuthology.orchestra.run.smithi106.stdout: "extra_probe_peers": [], 2024-04-25T10:32:05.581 INFO:teuthology.orchestra.run.smithi106.stdout: "sync_provider": [], 2024-04-25T10:32:05.581 INFO:teuthology.orchestra.run.smithi106.stdout: "monmap": { 2024-04-25T10:32:05.581 INFO:teuthology.orchestra.run.smithi106.stdout: "epoch": 1, 2024-04-25T10:32:05.581 INFO:teuthology.orchestra.run.smithi106.stdout: "fsid": "3ee77985-e8e4-41cf-81e9-7c774cf7891f", 2024-04-25T10:32:05.581 INFO:teuthology.orchestra.run.smithi106.stdout: "modified": "2024-04-25T10:29:33.446228Z", 2024-04-25T10:32:05.581 INFO:teuthology.orchestra.run.smithi106.stdout: "created": "2024-04-25T10:29:33.446228Z", 2024-04-25T10:32:05.581 INFO:teuthology.orchestra.run.smithi106.stdout: "min_mon_release": 19, 2024-04-25T10:32:05.581 INFO:teuthology.orchestra.run.smithi106.stdout: "min_mon_release_name": "squid", 2024-04-25T10:32:05.581 INFO:teuthology.orchestra.run.smithi106.stdout: "election_strategy": 1, 2024-04-25T10:32:05.581 INFO:teuthology.orchestra.run.smithi106.stdout: "disallowed_leaders: ": "", 2024-04-25T10:32:05.582 INFO:teuthology.orchestra.run.smithi106.stdout: "stretch_mode": false, 2024-04-25T10:32:05.582 INFO:teuthology.orchestra.run.smithi106.stdout: "tiebreaker_mon": "", 2024-04-25T10:32:05.582 INFO:teuthology.orchestra.run.smithi106.stdout: "removed_ranks: ": "", 2024-04-25T10:32:05.582 INFO:teuthology.orchestra.run.smithi106.stdout: "features": { 2024-04-25T10:32:05.582 INFO:teuthology.orchestra.run.smithi106.stdout: "persistent": [ 2024-04-25T10:32:05.582 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:32:05.582 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:32:05.582 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:32:05.582 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:32:05.582 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:32:05.582 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:32:05.582 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:32:05.582 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:32:05.583 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:32:05.583 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:32:05.583 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:32:05.583 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:32:05.583 INFO:teuthology.orchestra.run.smithi106.stdout: "optional": [] 2024-04-25T10:32:05.583 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:05.583 INFO:teuthology.orchestra.run.smithi106.stdout: "mons": [ 2024-04-25T10:32:05.583 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:05.583 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 0, 2024-04-25T10:32:05.583 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "a", 2024-04-25T10:32:05.583 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:32:05.583 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:32:05.583 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:05.583 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:32:05.584 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:3300", 2024-04-25T10:32:05.584 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:32:05.584 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:05.584 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:05.584 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:32:05.584 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:6789", 2024-04-25T10:32:05.584 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:32:05.585 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:05.585 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:32:05.585 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:05.585 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:6789/0", 2024-04-25T10:32:05.585 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.106:6789/0", 2024-04-25T10:32:05.585 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:32:05.585 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:32:05.585 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:32:05.585 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:05.585 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:05.585 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 1, 2024-04-25T10:32:05.585 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "b", 2024-04-25T10:32:05.585 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:32:05.585 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:32:05.586 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:05.586 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:32:05.586 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:3300", 2024-04-25T10:32:05.586 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:32:05.586 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:05.586 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:05.586 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:32:05.586 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6789", 2024-04-25T10:32:05.586 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:32:05.586 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:05.586 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:32:05.586 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:05.586 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6789/0", 2024-04-25T10:32:05.587 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.134:6789/0", 2024-04-25T10:32:05.587 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:32:05.587 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:32:05.587 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:32:05.587 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:05.587 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:05.587 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 2, 2024-04-25T10:32:05.587 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "c", 2024-04-25T10:32:05.587 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:32:05.587 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:32:05.587 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:05.587 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:32:05.587 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:3301", 2024-04-25T10:32:05.587 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:32:05.588 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:05.588 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:05.588 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:32:05.588 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6790", 2024-04-25T10:32:05.588 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:32:05.588 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:05.588 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:32:05.588 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:05.588 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6790/0", 2024-04-25T10:32:05.588 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.134:6790/0", 2024-04-25T10:32:05.588 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:32:05.588 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:32:05.588 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:32:05.588 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:05.588 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:32:05.589 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:05.589 INFO:teuthology.orchestra.run.smithi106.stdout: "feature_map": { 2024-04-25T10:32:05.589 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": [ 2024-04-25T10:32:05.589 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:05.589 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:32:05.589 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:32:05.589 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 1 2024-04-25T10:32:05.589 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:05.589 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:32:05.589 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": [ 2024-04-25T10:32:05.589 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:05.589 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:32:05.589 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:32:05.589 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 6 2024-04-25T10:32:05.589 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:05.590 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:32:05.590 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": [ 2024-04-25T10:32:05.590 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:05.590 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:32:05.590 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:32:05.590 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 4 2024-04-25T10:32:05.590 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:05.590 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:32:05.590 INFO:teuthology.orchestra.run.smithi106.stdout: "client": [ 2024-04-25T10:32:05.590 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:05.590 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:32:05.591 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:32:05.591 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 7 2024-04-25T10:32:05.591 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:05.591 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:32:05.591 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": [ 2024-04-25T10:32:05.591 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:05.591 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:32:05.591 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:32:05.591 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 2 2024-04-25T10:32:05.591 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:05.591 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:32:05.591 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:05.591 INFO:teuthology.orchestra.run.smithi106.stdout: "stretch_mode": false 2024-04-25T10:32:05.591 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-25T10:32:05.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.575+0000 7f66b4c30640 1 -- 172.21.15.106:0/1857513737 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f66900744e0 msgr2=0x7f66900769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:05.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.575+0000 7f66b4c30640 1 --2- 172.21.15.106:0/1857513737 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f66900744e0 0x7f66900769f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f66a8033000 tx=0x7f66a8009000 comp rx=0 tx=0).stop 2024-04-25T10:32:05.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.575+0000 7f66b4c30640 1 -- 172.21.15.106:0/1857513737 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66b00b4ce0 msgr2=0x7f66b014fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:05.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.575+0000 7f66b4c30640 1 --2- 172.21.15.106:0/1857513737 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66b00b4ce0 0x7f66b014fff0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f66a000c3c0 tx=0x7f66a000c880 comp rx=0 tx=0).stop 2024-04-25T10:32:05.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.575+0000 7f66aed9d640 1 -- 172.21.15.106:0/1857513737 reap_dead start 2024-04-25T10:32:05.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.575+0000 7f66b4c30640 1 -- 172.21.15.106:0/1857513737 shutdown_connections 2024-04-25T10:32:05.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.575+0000 7f66b4c30640 1 --2- 172.21.15.106:0/1857513737 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66b00b4ce0 0x7f66b014fff0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:05.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.575+0000 7f66b4c30640 1 -- 172.21.15.106:0/1857513737 >> 172.21.15.106:0/1857513737 conn(0x7f66b0068b10 msgr2=0x7f66b0154f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:05.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.575+0000 7f66b4c30640 1 -- 172.21.15.106:0/1857513737 shutdown_connections 2024-04-25T10:32:05.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.575+0000 7f66b4c30640 1 -- 172.21.15.106:0/1857513737 wait complete. 2024-04-25T10:32:05.594 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T10:32:05.594 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-04-25T10:32:05.594 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-25T10:32:05.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.701+0000 7f13300af640 1 Processor -- start 2024-04-25T10:32:05.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.701+0000 7f13300af640 1 -- start start 2024-04-25T10:32:05.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.702+0000 7f13300af640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f1328154e30 0x7f13280c57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:05.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.702+0000 7f13300af640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f13280d00a0 0x7f13280d04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:05.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.702+0000 7f13300af640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13280c7380 con 0x7f13280c51e0 2024-04-25T10:32:05.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.702+0000 7f13300af640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f13280cb850 con 0x7f13280d00a0 2024-04-25T10:32:05.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.702+0000 7f13300af640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f13280ccb50 con 0x7f1328154e30 2024-04-25T10:32:05.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.702+0000 7f132e64c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f13280d00a0 0x7f13280d04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:05.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.702+0000 7f132e64c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f13280d00a0 0x7f13280d04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53078/0 (socket says 172.21.15.106:53078) 2024-04-25T10:32:05.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.702+0000 7f132e64c640 1 -- 172.21.15.106:0/1988616219 learned_addr learned my addr 172.21.15.106:0/1988616219 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:32:05.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.702+0000 7f132d64a640 1 --2- 172.21.15.106:0/1988616219 >> v2:172.21.15.134:3300/0 conn(0x7f1328154e30 0x7f13280c57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:05.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.702+0000 7f132e64c640 1 -- 172.21.15.106:0/1988616219 >> v2:172.21.15.134:3300/0 conn(0x7f1328154e30 msgr2=0x7f13280c57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:05.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.702+0000 7f132e64c640 1 --2- 172.21.15.106:0/1988616219 >> v2:172.21.15.134:3300/0 conn(0x7f1328154e30 0x7f13280c57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:05.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.702+0000 7f132e64c640 1 -- 172.21.15.106:0/1988616219 >> v1:172.21.15.106:6789/0 conn(0x7f13280c51e0 legacy=0x7f132815b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:05.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.703+0000 7f132e64c640 1 -- 172.21.15.106:0/1988616219 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13280ccdf0 con 0x7f13280d00a0 2024-04-25T10:32:05.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.703+0000 7f132e64c640 1 --2- 172.21.15.106:0/1988616219 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13280d00a0 0x7f13280d04b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1320009870 tx=0x7f132002a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bd8e19bc824cbc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:05.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.703+0000 7f132ce49640 1 -- 172.21.15.106:0/1988616219 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1320037070 con 0x7f13280d00a0 2024-04-25T10:32:05.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.703+0000 7f132ce49640 1 -- 172.21.15.106:0/1988616219 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1320032640 con 0x7f13280d00a0 2024-04-25T10:32:05.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.703+0000 7f132ce49640 1 -- 172.21.15.106:0/1988616219 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1320032a10 con 0x7f13280d00a0 2024-04-25T10:32:05.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.704+0000 7f13300af640 1 -- 172.21.15.106:0/1988616219 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13280d00a0 msgr2=0x7f13280d04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:05.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.704+0000 7f13300af640 1 --2- 172.21.15.106:0/1988616219 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13280d00a0 0x7f13280d04b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1320009870 tx=0x7f132002a330 comp rx=0 tx=0).stop 2024-04-25T10:32:05.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.704+0000 7f13300af640 1 -- 172.21.15.106:0/1988616219 shutdown_connections 2024-04-25T10:32:05.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.704+0000 7f13300af640 1 --2- 172.21.15.106:0/1988616219 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13280d00a0 0x7f13280d04b0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:05.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.704+0000 7f13300af640 1 --2- 172.21.15.106:0/1988616219 >> v2:172.21.15.134:3300/0 conn(0x7f1328154e30 0x7f13280c57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:05.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.704+0000 7f13300af640 1 -- 172.21.15.106:0/1988616219 >> 172.21.15.106:0/1988616219 conn(0x7f1328072730 msgr2=0x7f1328072b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:05.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.704+0000 7f13300af640 1 -- 172.21.15.106:0/1988616219 shutdown_connections 2024-04-25T10:32:05.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.704+0000 7f13300af640 1 -- 172.21.15.106:0/1988616219 wait complete. 2024-04-25T10:32:05.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.706+0000 7f13300af640 1 Processor -- start 2024-04-25T10:32:05.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.706+0000 7f13300af640 1 -- start start 2024-04-25T10:32:05.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.707+0000 7f13300af640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13280c51e0 0x7f13281514d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:05.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.707+0000 7f13300af640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13280d00a0 0x7f132814b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:05.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.707+0000 7f13300af640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1328154e30 0x7f13280b5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:05.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.707+0000 7f13300af640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f13280cbaf0 con 0x7f1328154e30 2024-04-25T10:32:05.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.707+0000 7f13300af640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f13280cd3b0 con 0x7f13280d00a0 2024-04-25T10:32:05.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.707+0000 7f13300af640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f13280cd8d0 con 0x7f13280c51e0 2024-04-25T10:32:05.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.707+0000 7f132e64c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1328154e30 0x7f13280b5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:05.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.707+0000 7f132e64c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1328154e30 0x7f13280b5460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53094/0 (socket says 172.21.15.106:53094) 2024-04-25T10:32:05.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.707+0000 7f132e64c640 1 -- 172.21.15.106:0/647013735 learned_addr learned my addr 172.21.15.106:0/647013735 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:32:05.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.707+0000 7f132d64a640 1 --2- 172.21.15.106:0/647013735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13280d00a0 0x7f132814b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:05.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.707+0000 7f132de4b640 1 --2- 172.21.15.106:0/647013735 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13280c51e0 0x7f13281514d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:05.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.707+0000 7f132e64c640 1 -- 172.21.15.106:0/647013735 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13280c51e0 msgr2=0x7f13281514d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:05.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.707+0000 7f132e64c640 1 --2- 172.21.15.106:0/647013735 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13280c51e0 0x7f13281514d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:05.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.707+0000 7f132e64c640 1 -- 172.21.15.106:0/647013735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13280d00a0 msgr2=0x7f132814b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:05.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.707+0000 7f132e64c640 1 --2- 172.21.15.106:0/647013735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13280d00a0 0x7f132814b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:05.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.707+0000 7f132e64c640 1 -- 172.21.15.106:0/647013735 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f132814d3d0 con 0x7f1328154e30 2024-04-25T10:32:05.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.707+0000 7f132d64a640 1 --2- 172.21.15.106:0/647013735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13280d00a0 0x7f132814b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:32:05.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.707+0000 7f132de4b640 1 --2- 172.21.15.106:0/647013735 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13280c51e0 0x7f13281514d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:32:05.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.708+0000 7f132e64c640 1 --2- 172.21.15.106:0/647013735 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1328154e30 0x7f13280b5460 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1320009870 tx=0x7f13200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:05.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.708+0000 7f131e7fc640 1 -- 172.21.15.106:0/647013735 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1320037040 con 0x7f1328154e30 2024-04-25T10:32:05.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.708+0000 7f131e7fc640 1 -- 172.21.15.106:0/647013735 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1320042070 con 0x7f1328154e30 2024-04-25T10:32:05.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.708+0000 7f13300af640 1 -- 172.21.15.106:0/647013735 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f132814da20 con 0x7f1328154e30 2024-04-25T10:32:05.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.708+0000 7f131e7fc640 1 -- 172.21.15.106:0/647013735 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1320030990 con 0x7f1328154e30 2024-04-25T10:32:05.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.708+0000 7f13300af640 1 -- 172.21.15.106:0/647013735 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f132814d0a0 con 0x7f1328154e30 2024-04-25T10:32:05.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.709+0000 7f13300af640 1 --2- 172.21.15.106:0/647013735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13281f0c00 0x7f13281f1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:05.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.709+0000 7f13300af640 1 -- 172.21.15.106:0/647013735 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f132814d880 con 0x7f13281f0c00 2024-04-25T10:32:05.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.709+0000 7f132de4b640 1 --2- 172.21.15.106:0/647013735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13281f0c00 0x7f13281f1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:05.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.709+0000 7f132de4b640 1 -- 172.21.15.106:0/647013735 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f132806a3d0 con 0x7f13281f0c00 2024-04-25T10:32:05.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.709+0000 7f132de4b640 1 --2- 172.21.15.106:0/647013735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13281f0c00 0x7f13281f1010 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f132400e7c0 tx=0x7f132400ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:05.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.709+0000 7f131e7fc640 1 -- 172.21.15.106:0/647013735 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1320030b30 con 0x7f1328154e30 2024-04-25T10:32:05.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.710+0000 7f131e7fc640 1 --2- 172.21.15.106:0/647013735 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f130c0744b0 0x7f130c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:05.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.710+0000 7f131e7fc640 1 -- 172.21.15.106:0/647013735 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f13200b5cd0 con 0x7f1328154e30 2024-04-25T10:32:05.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.710+0000 7f132d64a640 1 --2- 172.21.15.106:0/647013735 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f130c0744b0 0x7f130c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:05.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.710+0000 7f131e7fc640 1 -- 172.21.15.106:0/647013735 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1324003b80 con 0x7f13281f0c00 2024-04-25T10:32:05.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.710+0000 7f131e7fc640 1 -- 172.21.15.106:0/647013735 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f1324003d20 con 0x7f13281f0c00 2024-04-25T10:32:05.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.710+0000 7f131e7fc640 1 -- 172.21.15.106:0/647013735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13281f0c00 msgr2=0x7f13281f1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:05.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.710+0000 7f131e7fc640 1 --2- 172.21.15.106:0/647013735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13281f0c00 0x7f13281f1010 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f132400e7c0 tx=0x7f132400ec80 comp rx=0 tx=0).stop 2024-04-25T10:32:05.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.710+0000 7f131e7fc640 1 -- 172.21.15.106:0/647013735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13281f0c00 msgr2=0x7f13281f1010 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T10:32:05.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.710+0000 7f131e7fc640 1 --2- 172.21.15.106:0/647013735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13281f0c00 0x7f13281f1010 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:05.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.710+0000 7f132d64a640 1 --2- 172.21.15.106:0/647013735 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f130c0744b0 0x7f130c0769c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f132814c3a0 tx=0x7f1318008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:05.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.717+0000 7f13300af640 1 --2- 172.21.15.106:0/647013735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1328153180 0x7f1328153590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:05.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.717+0000 7f13300af640 1 -- 172.21.15.106:0/647013735 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f13280cbd50 con 0x7f1328153180 2024-04-25T10:32:05.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.717+0000 7f132de4b640 1 --2- 172.21.15.106:0/647013735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1328153180 0x7f1328153590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:05.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.717+0000 7f132de4b640 1 -- 172.21.15.106:0/647013735 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f13280d08a0 con 0x7f1328153180 2024-04-25T10:32:05.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.717+0000 7f132de4b640 1 --2- 172.21.15.106:0/647013735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1328153180 0x7f1328153590 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f132400e7c0 tx=0x7f1324001250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:05.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.718+0000 7f131e7fc640 1 -- 172.21.15.106:0/647013735 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1324003b80 con 0x7f1328153180 2024-04-25T10:32:05.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.718+0000 7f131e7fc640 1 -- 172.21.15.106:0/647013735 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4730 (secure 0 0 0) 0x7f13240050d0 con 0x7f1328153180 2024-04-25T10:32:05.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.718+0000 7f131e7fc640 1 -- 172.21.15.106:0/647013735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1328153180 msgr2=0x7f1328153590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:05.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.718+0000 7f131e7fc640 1 --2- 172.21.15.106:0/647013735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1328153180 0x7f1328153590 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f132400e7c0 tx=0x7f1324001250 comp rx=0 tx=0).stop 2024-04-25T10:32:05.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.718+0000 7f131e7fc640 1 -- 172.21.15.106:0/647013735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1328153180 msgr2=0x7f1328153590 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T10:32:05.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.718+0000 7f131e7fc640 1 --2- 172.21.15.106:0/647013735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1328153180 0x7f1328153590 secure :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f132400e7c0 tx=0x7f1324001250 comp rx=0 tx=0).stop 2024-04-25T10:32:05.721 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-25T10:32:05.721 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "b", 2024-04-25T10:32:05.721 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 1, 2024-04-25T10:32:05.721 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "peon", 2024-04-25T10:32:05.721 INFO:teuthology.orchestra.run.smithi106.stdout: "election_epoch": 10, 2024-04-25T10:32:05.721 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 117914, 2024-04-25T10:32:05.721 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum": [ 2024-04-25T10:32:05.721 INFO:teuthology.orchestra.run.smithi106.stdout: 0, 2024-04-25T10:32:05.722 INFO:teuthology.orchestra.run.smithi106.stdout: 1, 2024-04-25T10:32:05.722 INFO:teuthology.orchestra.run.smithi106.stdout: 2 2024-04-25T10:32:05.722 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:32:05.722 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_age": 16, 2024-04-25T10:32:05.722 INFO:teuthology.orchestra.run.smithi106.stdout: "features": { 2024-04-25T10:32:05.722 INFO:teuthology.orchestra.run.smithi106.stdout: "required_con": "2450521708007866372", 2024-04-25T10:32:05.722 INFO:teuthology.orchestra.run.smithi106.stdout: "required_mon": [ 2024-04-25T10:32:05.722 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:32:05.722 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:32:05.722 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:32:05.722 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:32:05.722 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:32:05.722 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:32:05.722 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:32:05.723 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:32:05.723 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:32:05.723 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:32:05.723 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:32:05.723 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:32:05.723 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_con": "4540701547738038271", 2024-04-25T10:32:05.723 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_mon": [ 2024-04-25T10:32:05.723 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:32:05.723 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:32:05.723 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:32:05.723 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:32:05.723 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:32:05.723 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:32:05.723 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:32:05.723 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:32:05.724 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:32:05.724 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:32:05.724 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:32:05.724 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:32:05.724 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:05.724 INFO:teuthology.orchestra.run.smithi106.stdout: "outside_quorum": [], 2024-04-25T10:32:05.724 INFO:teuthology.orchestra.run.smithi106.stdout: "extra_probe_peers": [], 2024-04-25T10:32:05.724 INFO:teuthology.orchestra.run.smithi106.stdout: "sync_provider": [], 2024-04-25T10:32:05.724 INFO:teuthology.orchestra.run.smithi106.stdout: "monmap": { 2024-04-25T10:32:05.724 INFO:teuthology.orchestra.run.smithi106.stdout: "epoch": 1, 2024-04-25T10:32:05.724 INFO:teuthology.orchestra.run.smithi106.stdout: "fsid": "3ee77985-e8e4-41cf-81e9-7c774cf7891f", 2024-04-25T10:32:05.724 INFO:teuthology.orchestra.run.smithi106.stdout: "modified": "2024-04-25T10:29:33.446228Z", 2024-04-25T10:32:05.724 INFO:teuthology.orchestra.run.smithi106.stdout: "created": "2024-04-25T10:29:33.446228Z", 2024-04-25T10:32:05.724 INFO:teuthology.orchestra.run.smithi106.stdout: "min_mon_release": 19, 2024-04-25T10:32:05.725 INFO:teuthology.orchestra.run.smithi106.stdout: "min_mon_release_name": "squid", 2024-04-25T10:32:05.725 INFO:teuthology.orchestra.run.smithi106.stdout: "election_strategy": 1, 2024-04-25T10:32:05.725 INFO:teuthology.orchestra.run.smithi106.stdout: "disallowed_leaders: ": "", 2024-04-25T10:32:05.725 INFO:teuthology.orchestra.run.smithi106.stdout: "stretch_mode": false, 2024-04-25T10:32:05.725 INFO:teuthology.orchestra.run.smithi106.stdout: "tiebreaker_mon": "", 2024-04-25T10:32:05.725 INFO:teuthology.orchestra.run.smithi106.stdout: "removed_ranks: ": "", 2024-04-25T10:32:05.725 INFO:teuthology.orchestra.run.smithi106.stdout: "features": { 2024-04-25T10:32:05.725 INFO:teuthology.orchestra.run.smithi106.stdout: "persistent": [ 2024-04-25T10:32:05.725 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:32:05.725 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:32:05.725 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:32:05.725 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:32:05.725 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:32:05.725 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:32:05.726 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:32:05.726 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:32:05.726 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:32:05.726 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:32:05.726 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:32:05.726 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:32:05.726 INFO:teuthology.orchestra.run.smithi106.stdout: "optional": [] 2024-04-25T10:32:05.726 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:05.726 INFO:teuthology.orchestra.run.smithi106.stdout: "mons": [ 2024-04-25T10:32:05.726 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:05.726 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 0, 2024-04-25T10:32:05.726 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "a", 2024-04-25T10:32:05.726 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:32:05.726 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:32:05.727 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:05.727 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:32:05.727 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:3300", 2024-04-25T10:32:05.727 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:32:05.727 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:05.727 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:05.727 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:32:05.727 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:6789", 2024-04-25T10:32:05.727 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:32:05.727 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:05.727 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:32:05.727 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:05.727 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:6789/0", 2024-04-25T10:32:05.728 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.106:6789/0", 2024-04-25T10:32:05.728 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:32:05.728 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:32:05.728 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:32:05.728 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:05.728 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:05.728 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 1, 2024-04-25T10:32:05.728 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "b", 2024-04-25T10:32:05.728 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:32:05.728 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:32:05.728 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:05.728 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:32:05.728 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:3300", 2024-04-25T10:32:05.728 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:32:05.729 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:05.729 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:05.729 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:32:05.729 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6789", 2024-04-25T10:32:05.729 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:32:05.729 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:05.729 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:32:05.729 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:05.729 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6789/0", 2024-04-25T10:32:05.729 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.134:6789/0", 2024-04-25T10:32:05.729 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:32:05.729 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:32:05.729 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:32:05.729 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:05.729 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:05.730 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 2, 2024-04-25T10:32:05.730 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "c", 2024-04-25T10:32:05.730 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:32:05.730 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:32:05.730 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:05.730 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:32:05.730 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:3301", 2024-04-25T10:32:05.730 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:32:05.730 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:05.730 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:05.730 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:32:05.730 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6790", 2024-04-25T10:32:05.730 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:32:05.730 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:05.731 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:32:05.731 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:05.731 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6790/0", 2024-04-25T10:32:05.731 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.134:6790/0", 2024-04-25T10:32:05.731 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:32:05.731 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:32:05.731 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:32:05.731 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:05.731 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:32:05.731 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:05.731 INFO:teuthology.orchestra.run.smithi106.stdout: "feature_map": { 2024-04-25T10:32:05.731 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": [ 2024-04-25T10:32:05.731 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:05.731 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:32:05.732 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:32:05.732 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 1 2024-04-25T10:32:05.732 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:05.732 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:32:05.732 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": [ 2024-04-25T10:32:05.732 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:05.732 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:32:05.732 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:32:05.732 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 4 2024-04-25T10:32:05.732 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:05.732 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:32:05.732 INFO:teuthology.orchestra.run.smithi106.stdout: "client": [ 2024-04-25T10:32:05.732 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:05.732 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:32:05.733 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:32:05.733 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 1 2024-04-25T10:32:05.733 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:05.733 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:32:05.733 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:05.733 INFO:teuthology.orchestra.run.smithi106.stdout: "stretch_mode": false 2024-04-25T10:32:05.733 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-25T10:32:05.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.718+0000 7f13300af640 1 -- 172.21.15.106:0/647013735 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f130c0744b0 msgr2=0x7f130c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:05.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.718+0000 7f13300af640 1 --2- 172.21.15.106:0/647013735 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f130c0744b0 0x7f130c0769c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f132814c3a0 tx=0x7f1318008040 comp rx=0 tx=0).stop 2024-04-25T10:32:05.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.718+0000 7f13300af640 1 -- 172.21.15.106:0/647013735 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1328154e30 msgr2=0x7f13280b5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:05.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.718+0000 7f13300af640 1 --2- 172.21.15.106:0/647013735 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1328154e30 0x7f13280b5460 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1320009870 tx=0x7f13200046c0 comp rx=0 tx=0).stop 2024-04-25T10:32:05.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.718+0000 7f132e64c640 1 -- 172.21.15.106:0/647013735 reap_dead start 2024-04-25T10:32:05.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.718+0000 7f132e64c640 1 -- 172.21.15.106:0/647013735 reap_dead start 2024-04-25T10:32:05.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.718+0000 7f13300af640 1 -- 172.21.15.106:0/647013735 shutdown_connections 2024-04-25T10:32:05.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.718+0000 7f13300af640 1 -- 172.21.15.106:0/647013735 >> 172.21.15.106:0/647013735 conn(0x7f1328072730 msgr2=0x7f132814ff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:05.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.718+0000 7f13300af640 1 -- 172.21.15.106:0/647013735 shutdown_connections 2024-04-25T10:32:05.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.718+0000 7f13300af640 1 -- 172.21.15.106:0/647013735 wait complete. 2024-04-25T10:32:05.735 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T10:32:05.736 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-04-25T10:32:05.736 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-25T10:32:05.845 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.843+0000 7f265d5f3640 1 Processor -- start 2024-04-25T10:32:05.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.843+0000 7f265d5f3640 1 -- start start 2024-04-25T10:32:05.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.844+0000 7f265d5f3640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f26580c7e30 0x7f26580c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:05.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.844+0000 7f265d5f3640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26580ce630 con 0x7f26580b4ce0 2024-04-25T10:32:05.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.844+0000 7f265d5f3640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26580c7980 con 0x7f2658156ef0 2024-04-25T10:32:05.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.844+0000 7f265d5f3640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f26580b50f0 con 0x7f26580c7e30 2024-04-25T10:32:05.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.845+0000 7f2656ffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f26580c7e30 0x7f26580c8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:05.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.845+0000 7f2656ffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f26580c7e30 0x7f26580c8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53108/0 (socket says 172.21.15.106:53108) 2024-04-25T10:32:05.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.845+0000 7f2656ffd640 1 -- 172.21.15.106:0/2671080213 learned_addr learned my addr 172.21.15.106:0/2671080213 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:32:05.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.845+0000 7f26577fe640 1 --1- 172.21.15.106:0/2671080213 >> v1:172.21.15.106:6789/0 conn(0x7f26580b4ce0 0x7f265815ce30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:57836/0 (socket says 172.21.15.106:57836) 2024-04-25T10:32:05.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.845+0000 7f2656ffd640 1 -- 172.21.15.106:0/2671080213 >> v1:172.21.15.134:6789/0 conn(0x7f2658156ef0 legacy=0x7f265806d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:05.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.845+0000 7f2656ffd640 1 -- 172.21.15.106:0/2671080213 >> v1:172.21.15.106:6789/0 conn(0x7f26580b4ce0 legacy=0x7f265815ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:05.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.845+0000 7f2656ffd640 1 -- 172.21.15.106:0/2671080213 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26580c7010 con 0x7f26580c7e30 2024-04-25T10:32:05.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.846+0000 7f2656ffd640 1 --2- 172.21.15.106:0/2671080213 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26580c7e30 0x7f26580c8d90 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2648009e50 tx=0x7f264802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a78cd3b57f218b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:05.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.846+0000 7f2655ffb640 1 -- 172.21.15.106:0/2671080213 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2648036070 con 0x7f26580c7e30 2024-04-25T10:32:05.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.846+0000 7f2655ffb640 1 -- 172.21.15.106:0/2671080213 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2648031600 con 0x7f26580c7e30 2024-04-25T10:32:05.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.846+0000 7f2655ffb640 1 -- 172.21.15.106:0/2671080213 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26480319d0 con 0x7f26580c7e30 2024-04-25T10:32:05.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.846+0000 7f265d5f3640 1 -- 172.21.15.106:0/2671080213 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26580c7e30 msgr2=0x7f26580c8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:05.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.846+0000 7f265d5f3640 1 --2- 172.21.15.106:0/2671080213 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26580c7e30 0x7f26580c8d90 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2648009e50 tx=0x7f264802a730 comp rx=0 tx=0).stop 2024-04-25T10:32:05.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.846+0000 7f265d5f3640 1 -- 172.21.15.106:0/2671080213 shutdown_connections 2024-04-25T10:32:05.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.846+0000 7f265d5f3640 1 --2- 172.21.15.106:0/2671080213 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26580c7e30 0x7f26580c8d90 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:05.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.846+0000 7f265d5f3640 1 -- 172.21.15.106:0/2671080213 >> 172.21.15.106:0/2671080213 conn(0x7f2658068b10 msgr2=0x7f26580d1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:05.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.847+0000 7f265d5f3640 1 -- 172.21.15.106:0/2671080213 shutdown_connections 2024-04-25T10:32:05.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.847+0000 7f265d5f3640 1 -- 172.21.15.106:0/2671080213 wait complete. 2024-04-25T10:32:05.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.848+0000 7f265d5f3640 1 Processor -- start 2024-04-25T10:32:05.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.848+0000 7f265d5f3640 1 -- start start 2024-04-25T10:32:05.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.850+0000 7f265d5f3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26580b4ce0 0x7f2658157eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:05.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.850+0000 7f265d5f3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26580c7e30 0x7f26580d1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:05.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.850+0000 7f265d5f3640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2658156ef0 0x7f26581f73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:05.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.850+0000 7f265d5f3640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f265806d710 con 0x7f26580b4ce0 2024-04-25T10:32:05.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.850+0000 7f265d5f3640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f26580c7c60 con 0x7f26580c7e30 2024-04-25T10:32:05.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.850+0000 7f265d5f3640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f265806d890 con 0x7f2658156ef0 2024-04-25T10:32:05.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.850+0000 7f2656ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26580b4ce0 0x7f2658157eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:05.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.850+0000 7f2656ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26580b4ce0 0x7f2658157eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53112/0 (socket says 172.21.15.106:53112) 2024-04-25T10:32:05.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.850+0000 7f2656ffd640 1 -- 172.21.15.106:0/947968731 learned_addr learned my addr 172.21.15.106:0/947968731 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:32:05.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.850+0000 7f26567fc640 1 --2- 172.21.15.106:0/947968731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26580c7e30 0x7f26580d1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:05.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.850+0000 7f26577fe640 1 --2- 172.21.15.106:0/947968731 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2658156ef0 0x7f26581f73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:05.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.850+0000 7f2656ffd640 1 -- 172.21.15.106:0/947968731 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2658156ef0 msgr2=0x7f26581f73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:05.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.850+0000 7f2656ffd640 1 --2- 172.21.15.106:0/947968731 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2658156ef0 0x7f26581f73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:05.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.850+0000 7f2656ffd640 1 -- 172.21.15.106:0/947968731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26580c7e30 msgr2=0x7f26580d1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:05.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.850+0000 7f2656ffd640 1 --2- 172.21.15.106:0/947968731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26580c7e30 0x7f26580d1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:05.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.850+0000 7f2656ffd640 1 -- 172.21.15.106:0/947968731 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26581f7900 con 0x7f26580b4ce0 2024-04-25T10:32:05.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.850+0000 7f26577fe640 1 --2- 172.21.15.106:0/947968731 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2658156ef0 0x7f26581f73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:32:05.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.850+0000 7f2656ffd640 1 --2- 172.21.15.106:0/947968731 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26580b4ce0 0x7f2658157eb0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f264802ad20 tx=0x7f2648004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:05.854 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.850+0000 7f264f7fe640 1 -- 172.21.15.106:0/947968731 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264803f020 con 0x7f26580b4ce0 2024-04-25T10:32:05.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.850+0000 7f264f7fe640 1 -- 172.21.15.106:0/947968731 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2648037040 con 0x7f26580b4ce0 2024-04-25T10:32:05.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.850+0000 7f265d5f3640 1 -- 172.21.15.106:0/947968731 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26581f3470 con 0x7f26580b4ce0 2024-04-25T10:32:05.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.850+0000 7f264f7fe640 1 -- 172.21.15.106:0/947968731 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2648036070 con 0x7f26580b4ce0 2024-04-25T10:32:05.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.851+0000 7f265d5f3640 1 -- 172.21.15.106:0/947968731 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26581f3730 con 0x7f26580b4ce0 2024-04-25T10:32:05.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.851+0000 7f265d5f3640 1 --2- 172.21.15.106:0/947968731 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f26581f3930 0x7f2658158920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:05.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.851+0000 7f265d5f3640 1 -- 172.21.15.106:0/947968731 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f265806a3d0 con 0x7f26581f3930 2024-04-25T10:32:05.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.852+0000 7f26567fc640 1 --2- 172.21.15.106:0/947968731 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f26581f3930 0x7f2658158920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:05.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.852+0000 7f26567fc640 1 -- 172.21.15.106:0/947968731 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f265815a5a0 con 0x7f26581f3930 2024-04-25T10:32:05.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.852+0000 7f264f7fe640 1 -- 172.21.15.106:0/947968731 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2648042020 con 0x7f26580b4ce0 2024-04-25T10:32:05.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.852+0000 7f26567fc640 1 --2- 172.21.15.106:0/947968731 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f26581f3930 0x7f2658158920 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2640007fd0 tx=0x7f264000f920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:05.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.853+0000 7f264f7fe640 1 --2- 172.21.15.106:0/947968731 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f26380744e0 0x7f26380769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:05.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.853+0000 7f26577fe640 1 --2- 172.21.15.106:0/947968731 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f26380744e0 0x7f26380769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:05.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.853+0000 7f264f7fe640 1 -- 172.21.15.106:0/947968731 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f26480b5860 con 0x7f26580b4ce0 2024-04-25T10:32:05.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.853+0000 7f264f7fe640 1 -- 172.21.15.106:0/947968731 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26400052e0 con 0x7f26581f3930 2024-04-25T10:32:05.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.853+0000 7f264f7fe640 1 -- 172.21.15.106:0/947968731 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f2640018070 con 0x7f26581f3930 2024-04-25T10:32:05.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.853+0000 7f264f7fe640 1 -- 172.21.15.106:0/947968731 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f26581f3930 msgr2=0x7f2658158920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:05.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.853+0000 7f264f7fe640 1 --2- 172.21.15.106:0/947968731 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f26581f3930 0x7f2658158920 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2640007fd0 tx=0x7f264000f920 comp rx=0 tx=0).stop 2024-04-25T10:32:05.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.853+0000 7f264f7fe640 1 -- 172.21.15.106:0/947968731 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f26581f3930 msgr2=0x7f2658158920 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T10:32:05.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.853+0000 7f264f7fe640 1 --2- 172.21.15.106:0/947968731 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f26581f3930 0x7f2658158920 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:05.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.853+0000 7f26577fe640 1 --2- 172.21.15.106:0/947968731 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f26380744e0 0x7f26380769f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2658156b00 tx=0x7f2650007cc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:05.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.859+0000 7f265d5f3640 1 --2- 172.21.15.106:0/947968731 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2658151ef0 0x7f265815c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:05.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.859+0000 7f265d5f3640 1 -- 172.21.15.106:0/947968731 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f26580b72b0 con 0x7f2658151ef0 2024-04-25T10:32:05.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.860+0000 7f26567fc640 1 --2- 172.21.15.106:0/947968731 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2658151ef0 0x7f265815c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:05.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.860+0000 7f26567fc640 1 -- 172.21.15.106:0/947968731 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f26580cd6d0 con 0x7f2658151ef0 2024-04-25T10:32:05.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.860+0000 7f26567fc640 1 --2- 172.21.15.106:0/947968731 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2658151ef0 0x7f265815c000 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2640017000 tx=0x7f2640000f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:05.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.860+0000 7f264f7fe640 1 -- 172.21.15.106:0/947968731 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2640003b20 con 0x7f2658151ef0 2024-04-25T10:32:05.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.861+0000 7f264f7fe640 1 -- 172.21.15.106:0/947968731 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4559 (secure 0 0 0) 0x7f2640003b20 con 0x7f2658151ef0 2024-04-25T10:32:05.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.861+0000 7f264f7fe640 1 -- 172.21.15.106:0/947968731 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2658151ef0 msgr2=0x7f265815c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:05.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.861+0000 7f264f7fe640 1 --2- 172.21.15.106:0/947968731 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2658151ef0 0x7f265815c000 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2640017000 tx=0x7f2640000f20 comp rx=0 tx=0).stop 2024-04-25T10:32:05.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.861+0000 7f264f7fe640 1 -- 172.21.15.106:0/947968731 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2658151ef0 msgr2=0x7f265815c000 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T10:32:05.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.861+0000 7f264f7fe640 1 --2- 172.21.15.106:0/947968731 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2658151ef0 0x7f265815c000 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:05.864 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-25T10:32:05.864 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "c", 2024-04-25T10:32:05.864 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 2, 2024-04-25T10:32:05.864 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "peon", 2024-04-25T10:32:05.864 INFO:teuthology.orchestra.run.smithi106.stdout: "election_epoch": 10, 2024-04-25T10:32:05.864 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 16531, 2024-04-25T10:32:05.864 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum": [ 2024-04-25T10:32:05.864 INFO:teuthology.orchestra.run.smithi106.stdout: 0, 2024-04-25T10:32:05.864 INFO:teuthology.orchestra.run.smithi106.stdout: 1, 2024-04-25T10:32:05.864 INFO:teuthology.orchestra.run.smithi106.stdout: 2 2024-04-25T10:32:05.865 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:32:05.865 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_age": 16, 2024-04-25T10:32:05.865 INFO:teuthology.orchestra.run.smithi106.stdout: "features": { 2024-04-25T10:32:05.865 INFO:teuthology.orchestra.run.smithi106.stdout: "required_con": "2450521708007866372", 2024-04-25T10:32:05.865 INFO:teuthology.orchestra.run.smithi106.stdout: "required_mon": [ 2024-04-25T10:32:05.865 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:32:05.865 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:32:05.865 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:32:05.865 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:32:05.865 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:32:05.865 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:32:05.865 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:32:05.866 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:32:05.866 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:32:05.866 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:32:05.866 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:32:05.866 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:32:05.866 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_con": "4540701547738038271", 2024-04-25T10:32:05.866 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_mon": [ 2024-04-25T10:32:05.866 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:32:05.866 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:32:05.866 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:32:05.866 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:32:05.867 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:32:05.867 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:32:05.867 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:32:05.867 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:32:05.867 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:32:05.867 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:32:05.867 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:32:05.867 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:32:05.867 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:05.867 INFO:teuthology.orchestra.run.smithi106.stdout: "outside_quorum": [], 2024-04-25T10:32:05.868 INFO:teuthology.orchestra.run.smithi106.stdout: "extra_probe_peers": [], 2024-04-25T10:32:05.868 INFO:teuthology.orchestra.run.smithi106.stdout: "sync_provider": [], 2024-04-25T10:32:05.868 INFO:teuthology.orchestra.run.smithi106.stdout: "monmap": { 2024-04-25T10:32:05.868 INFO:teuthology.orchestra.run.smithi106.stdout: "epoch": 1, 2024-04-25T10:32:05.868 INFO:teuthology.orchestra.run.smithi106.stdout: "fsid": "3ee77985-e8e4-41cf-81e9-7c774cf7891f", 2024-04-25T10:32:05.868 INFO:teuthology.orchestra.run.smithi106.stdout: "modified": "2024-04-25T10:29:33.446228Z", 2024-04-25T10:32:05.868 INFO:teuthology.orchestra.run.smithi106.stdout: "created": "2024-04-25T10:29:33.446228Z", 2024-04-25T10:32:05.868 INFO:teuthology.orchestra.run.smithi106.stdout: "min_mon_release": 19, 2024-04-25T10:32:05.868 INFO:teuthology.orchestra.run.smithi106.stdout: "min_mon_release_name": "squid", 2024-04-25T10:32:05.868 INFO:teuthology.orchestra.run.smithi106.stdout: "election_strategy": 1, 2024-04-25T10:32:05.868 INFO:teuthology.orchestra.run.smithi106.stdout: "disallowed_leaders: ": "", 2024-04-25T10:32:05.869 INFO:teuthology.orchestra.run.smithi106.stdout: "stretch_mode": false, 2024-04-25T10:32:05.869 INFO:teuthology.orchestra.run.smithi106.stdout: "tiebreaker_mon": "", 2024-04-25T10:32:05.869 INFO:teuthology.orchestra.run.smithi106.stdout: "removed_ranks: ": "", 2024-04-25T10:32:05.869 INFO:teuthology.orchestra.run.smithi106.stdout: "features": { 2024-04-25T10:32:05.869 INFO:teuthology.orchestra.run.smithi106.stdout: "persistent": [ 2024-04-25T10:32:05.869 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:32:05.869 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:32:05.869 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:32:05.869 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:32:05.869 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:32:05.869 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:32:05.869 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:32:05.870 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:32:05.870 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:32:05.870 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:32:05.870 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:32:05.870 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:32:05.874 INFO:teuthology.orchestra.run.smithi106.stdout: "optional": [] 2024-04-25T10:32:05.874 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:05.874 INFO:teuthology.orchestra.run.smithi106.stdout: "mons": [ 2024-04-25T10:32:05.874 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:05.874 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 0, 2024-04-25T10:32:05.874 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "a", 2024-04-25T10:32:05.874 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:32:05.874 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:32:05.875 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:05.875 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:32:05.875 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:3300", 2024-04-25T10:32:05.875 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:32:05.875 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:05.875 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:05.875 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:32:05.875 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:6789", 2024-04-25T10:32:05.875 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:32:05.875 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:05.876 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:32:05.876 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:05.876 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:6789/0", 2024-04-25T10:32:05.876 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.106:6789/0", 2024-04-25T10:32:05.876 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:32:05.876 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:32:05.876 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:32:05.876 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:05.876 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:05.876 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 1, 2024-04-25T10:32:05.877 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "b", 2024-04-25T10:32:05.877 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:32:05.877 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:32:05.877 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:05.877 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:32:05.877 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:3300", 2024-04-25T10:32:05.877 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:32:05.877 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:05.877 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:05.877 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:32:05.877 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6789", 2024-04-25T10:32:05.878 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:32:05.878 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:05.878 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:32:05.878 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:05.878 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6789/0", 2024-04-25T10:32:05.878 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.134:6789/0", 2024-04-25T10:32:05.878 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:32:05.878 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:32:05.878 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:32:05.878 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:05.879 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:05.879 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 2, 2024-04-25T10:32:05.879 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "c", 2024-04-25T10:32:05.879 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:32:05.879 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:32:05.879 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:05.879 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:32:05.879 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:3301", 2024-04-25T10:32:05.879 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:32:05.879 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:05.880 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:05.880 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:32:05.880 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6790", 2024-04-25T10:32:05.880 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:32:05.880 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:05.880 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:32:05.880 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:05.880 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6790/0", 2024-04-25T10:32:05.880 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.134:6790/0", 2024-04-25T10:32:05.881 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:32:05.881 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:32:05.881 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:32:05.881 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:05.881 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:32:05.881 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:05.881 INFO:teuthology.orchestra.run.smithi106.stdout: "feature_map": { 2024-04-25T10:32:05.881 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": [ 2024-04-25T10:32:05.881 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:05.881 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:32:05.882 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:32:05.882 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 1 2024-04-25T10:32:05.882 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:05.882 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:32:05.882 INFO:teuthology.orchestra.run.smithi106.stdout: "client": [ 2024-04-25T10:32:05.882 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:05.882 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:32:05.882 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:32:05.882 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 1 2024-04-25T10:32:05.882 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:05.882 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:32:05.882 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:05.882 INFO:teuthology.orchestra.run.smithi106.stdout: "stretch_mode": false 2024-04-25T10:32:05.882 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-25T10:32:05.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.861+0000 7f265d5f3640 1 -- 172.21.15.106:0/947968731 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f26380744e0 msgr2=0x7f26380769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:05.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.861+0000 7f265d5f3640 1 --2- 172.21.15.106:0/947968731 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f26380744e0 0x7f26380769f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2658156b00 tx=0x7f2650007cc0 comp rx=0 tx=0).stop 2024-04-25T10:32:05.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.861+0000 7f265d5f3640 1 -- 172.21.15.106:0/947968731 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26580b4ce0 msgr2=0x7f2658157eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:05.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.861+0000 7f265d5f3640 1 --2- 172.21.15.106:0/947968731 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26580b4ce0 0x7f2658157eb0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f264802ad20 tx=0x7f2648004a40 comp rx=0 tx=0).stop 2024-04-25T10:32:05.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.861+0000 7f26577fe640 1 -- 172.21.15.106:0/947968731 reap_dead start 2024-04-25T10:32:05.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.861+0000 7f26577fe640 1 -- 172.21.15.106:0/947968731 reap_dead start 2024-04-25T10:32:05.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.861+0000 7f265d5f3640 1 -- 172.21.15.106:0/947968731 shutdown_connections 2024-04-25T10:32:05.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.861+0000 7f265d5f3640 1 -- 172.21.15.106:0/947968731 >> 172.21.15.106:0/947968731 conn(0x7f2658068b10 msgr2=0x7f265815ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:05.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.861+0000 7f265d5f3640 1 -- 172.21.15.106:0/947968731 shutdown_connections 2024-04-25T10:32:05.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:05.861+0000 7f265d5f3640 1 -- 172.21.15.106:0/947968731 wait complete. 2024-04-25T10:32:05.885 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T10:32:05.886 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c'] 2024-04-25T10:32:05.886 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-04-25T10:32:05.886 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-04-25T10:32:05.886 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-04-25T10:32:05.886 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-25T10:32:05.886 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:32:05.899 INFO:tasks.ceph.mon.c:Stopped 2024-04-25T10:32:05.899 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-04-25T10:32:05.899 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T10:32:06.007 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.005+0000 7fe1bf96c640 1 Processor -- start 2024-04-25T10:32:06.008 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.005+0000 7fe1bf96c640 1 -- start start 2024-04-25T10:32:06.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.007+0000 7fe1bf96c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1b80cc250 0x7fe1b8159a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:06.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.007+0000 7fe1bf96c640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fe1b80cc6b0 0x7fe1b8153c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:06.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.007+0000 7fe1bf96c640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1b80c9d40 con 0x7fe1b80c6a80 2024-04-25T10:32:06.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.007+0000 7fe1bf96c640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fe1b8159fa0 con 0x7fe1b80cc6b0 2024-04-25T10:32:06.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.007+0000 7fe1bf96c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe1b80c64c0 con 0x7fe1b80cc250 2024-04-25T10:32:06.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.007+0000 7fe1bd708640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1b80cc250 msgr2=0x7fe1b8159a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-04-25T10:32:06.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.007+0000 7fe1bd708640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1b80cc250 0x7fe1b8159a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:32:06.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.007+0000 7fe1bcf07640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fe1b80cc6b0 0x7fe1b8153c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:06.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.007+0000 7fe1bdf09640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fe1b80c6a80 0x7fe1b80c8a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.106:45790/0 (socket says 172.21.15.106:45790) 2024-04-25T10:32:06.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.007+0000 7fe1bdf09640 1 -- 172.21.15.106:0/1172110502 learned_addr learned my addr 172.21.15.106:0/1172110502 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:32:06.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.007+0000 7fe1bcf07640 1 -- 172.21.15.106:0/1172110502 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1b80cc250 msgr2=0x7fe1b8159a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:32:06.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.007+0000 7fe1bcf07640 1 --2- 172.21.15.106:0/1172110502 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1b80cc250 0x7fe1b8159a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:06.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.007+0000 7fe1bcf07640 1 -- 172.21.15.106:0/1172110502 >> v1:172.21.15.134:6789/0 conn(0x7fe1b80c6a80 legacy=0x7fe1b80c8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:06.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.008+0000 7fe1bcf07640 1 -- 172.21.15.106:0/1172110502 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1b80c67a0 con 0x7fe1b80cc6b0 2024-04-25T10:32:06.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.008+0000 7fe1bcf07640 1 --2- 172.21.15.106:0/1172110502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1b80cc6b0 0x7fe1b8153c20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe1a80097a0 tx=0x7fe1a802a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=583f1c4b4dba3171 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:06.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.008+0000 7fe1affff640 1 -- 172.21.15.106:0/1172110502 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a8037070 con 0x7fe1b80cc6b0 2024-04-25T10:32:06.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.008+0000 7fe1affff640 1 -- 172.21.15.106:0/1172110502 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1a8032440 con 0x7fe1b80cc6b0 2024-04-25T10:32:06.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.008+0000 7fe1affff640 1 -- 172.21.15.106:0/1172110502 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a8032810 con 0x7fe1b80cc6b0 2024-04-25T10:32:06.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.009+0000 7fe1bf96c640 1 -- 172.21.15.106:0/1172110502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1b80cc6b0 msgr2=0x7fe1b8153c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:06.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.009+0000 7fe1bf96c640 1 --2- 172.21.15.106:0/1172110502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1b80cc6b0 0x7fe1b8153c20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe1a80097a0 tx=0x7fe1a802a1a0 comp rx=0 tx=0).stop 2024-04-25T10:32:06.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.009+0000 7fe1bf96c640 1 -- 172.21.15.106:0/1172110502 shutdown_connections 2024-04-25T10:32:06.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.009+0000 7fe1bf96c640 1 --2- 172.21.15.106:0/1172110502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1b80cc6b0 0x7fe1b8153c20 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:06.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.009+0000 7fe1bf96c640 1 --2- 172.21.15.106:0/1172110502 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1b80cc250 0x7fe1b8159a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:06.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.009+0000 7fe1bf96c640 1 -- 172.21.15.106:0/1172110502 >> 172.21.15.106:0/1172110502 conn(0x7fe1b806d6d0 msgr2=0x7fe1b806dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:06.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.009+0000 7fe1bf96c640 1 -- 172.21.15.106:0/1172110502 shutdown_connections 2024-04-25T10:32:06.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.009+0000 7fe1bf96c640 1 -- 172.21.15.106:0/1172110502 wait complete. 2024-04-25T10:32:06.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.011+0000 7fe1bf96c640 1 Processor -- start 2024-04-25T10:32:06.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.011+0000 7fe1bf96c640 1 -- start start 2024-04-25T10:32:06.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.012+0000 7fe1bf96c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1b80c6a80 0x7fe1b81496d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:06.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.012+0000 7fe1bf96c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1b8147d80 0x7fe1b8146460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:06.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.012+0000 7fe1bf96c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1b815cfe0 0x7fe1b815d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:06.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.012+0000 7fe1bf96c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b80c7450 con 0x7fe1b815cfe0 2024-04-25T10:32:06.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.012+0000 7fe1bf96c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b8154160 con 0x7fe1b80c6a80 2024-04-25T10:32:06.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.012+0000 7fe1bf96c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe1b80c9180 con 0x7fe1b8147d80 2024-04-25T10:32:06.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.012+0000 7fe1bcf07640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1b8147d80 msgr2=0x7fe1b8146460 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-04-25T10:32:06.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.012+0000 7fe1bcf07640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1b8147d80 0x7fe1b8146460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:32:06.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.012+0000 7fe1bdf09640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1b815cfe0 0x7fe1b815d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:06.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.012+0000 7fe1bdf09640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1b815cfe0 0x7fe1b815d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53118/0 (socket says 172.21.15.106:53118) 2024-04-25T10:32:06.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.012+0000 7fe1bdf09640 1 -- 172.21.15.106:0/3675298521 learned_addr learned my addr 172.21.15.106:0/3675298521 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:32:06.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.012+0000 7fe1bd708640 1 --2- 172.21.15.106:0/3675298521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1b80c6a80 0x7fe1b81496d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:06.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.013+0000 7fe1bdf09640 1 -- 172.21.15.106:0/3675298521 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1b8147d80 msgr2=0x7fe1b8146460 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:32:06.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.013+0000 7fe1bdf09640 1 --2- 172.21.15.106:0/3675298521 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1b8147d80 0x7fe1b8146460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:06.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.013+0000 7fe1bdf09640 1 -- 172.21.15.106:0/3675298521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1b80c6a80 msgr2=0x7fe1b81496d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:06.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.013+0000 7fe1bdf09640 1 --2- 172.21.15.106:0/3675298521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1b80c6a80 0x7fe1b81496d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:06.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.013+0000 7fe1bdf09640 1 -- 172.21.15.106:0/3675298521 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1b81f0b90 con 0x7fe1b815cfe0 2024-04-25T10:32:06.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.013+0000 7fe1bdf09640 1 --2- 172.21.15.106:0/3675298521 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1b815cfe0 0x7fe1b815d3f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe1b000ca80 tx=0x7fe1b000cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:06.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.013+0000 7fe1adffb640 1 -- 172.21.15.106:0/3675298521 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b0017070 con 0x7fe1b815cfe0 2024-04-25T10:32:06.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.013+0000 7fe1bf96c640 1 -- 172.21.15.106:0/3675298521 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1b81f0e70 con 0x7fe1b815cfe0 2024-04-25T10:32:06.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.013+0000 7fe1adffb640 1 -- 172.21.15.106:0/3675298521 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1b000ac30 con 0x7fe1b815cfe0 2024-04-25T10:32:06.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.013+0000 7fe1adffb640 1 -- 172.21.15.106:0/3675298521 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b0012600 con 0x7fe1b815cfe0 2024-04-25T10:32:06.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.013+0000 7fe1bf96c640 1 -- 172.21.15.106:0/3675298521 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1b81f1430 con 0x7fe1b815cfe0 2024-04-25T10:32:06.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.014+0000 7fe1bf96c640 1 -- 172.21.15.106:0/3675298521 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1b80cfdb0 con 0x7fe1b815cfe0 2024-04-25T10:32:06.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.015+0000 7fe1adffb640 1 -- 172.21.15.106:0/3675298521 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe1b001a070 con 0x7fe1b815cfe0 2024-04-25T10:32:06.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.015+0000 7fe1adffb640 1 --2- 172.21.15.106:0/3675298521 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fe19c074460 0x7fe19c076970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:06.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.015+0000 7fe1adffb640 1 -- 172.21.15.106:0/3675298521 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fe1b0094f50 con 0x7fe1b815cfe0 2024-04-25T10:32:06.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.015+0000 7fe1bd708640 1 --2- 172.21.15.106:0/3675298521 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fe19c074460 0x7fe19c076970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:06.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.015+0000 7fe1bd708640 1 --2- 172.21.15.106:0/3675298521 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fe19c074460 0x7fe19c076970 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe1b40093d0 tx=0x7fe1b4009360 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:06.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.020+0000 7fe1adffb640 1 -- 172.21.15.106:0/3675298521 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe1b80cfdb0 con 0x7fe1b815cfe0 2024-04-25T10:32:06.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.294+0000 7fe1bf96c640 1 -- 172.21.15.106:0/3675298521 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fe1b806a3d0 con 0x7fe1b815cfe0 2024-04-25T10:32:06.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.294+0000 7fe1adffb640 1 -- 172.21.15.106:0/3675298521 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1569 (secure 0 0 0) 0x7fe1b000ae20 con 0x7fe1b815cfe0 2024-04-25T10:32:06.297 INFO:teuthology.orchestra.run.smithi106.stdout:{"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":16,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"3ee77985-e8e4-41cf-81e9-7c774cf7891f","modified":"2024-04-25T10:29:33.446228Z","created":"2024-04-25T10:29:33.446228Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:3300","nonce":0},{"type":"v1","addr":"172.21.15.106:6789","nonce":0}]},"addr":"172.21.15.106:6789/0","public_addr":"172.21.15.106:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T10:32:06.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.297+0000 7fe1bf96c640 1 -- 172.21.15.106:0/3675298521 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fe19c074460 msgr2=0x7fe19c076970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:06.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.297+0000 7fe1bf96c640 1 --2- 172.21.15.106:0/3675298521 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fe19c074460 0x7fe19c076970 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe1b40093d0 tx=0x7fe1b4009360 comp rx=0 tx=0).stop 2024-04-25T10:32:06.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.297+0000 7fe1bf96c640 1 -- 172.21.15.106:0/3675298521 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1b815cfe0 msgr2=0x7fe1b815d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:06.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.297+0000 7fe1bf96c640 1 --2- 172.21.15.106:0/3675298521 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1b815cfe0 0x7fe1b815d3f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe1b000ca80 tx=0x7fe1b000cf40 comp rx=0 tx=0).stop 2024-04-25T10:32:06.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.297+0000 7fe1bf96c640 1 -- 172.21.15.106:0/3675298521 shutdown_connections 2024-04-25T10:32:06.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.297+0000 7fe1bf96c640 1 --2- 172.21.15.106:0/3675298521 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fe19c074460 0x7fe19c076970 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:06.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.297+0000 7fe1bf96c640 1 --2- 172.21.15.106:0/3675298521 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1b815cfe0 0x7fe1b815d3f0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:06.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.297+0000 7fe1bf96c640 1 --2- 172.21.15.106:0/3675298521 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1b8147d80 0x7fe1b8146460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:06.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.297+0000 7fe1bf96c640 1 --2- 172.21.15.106:0/3675298521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1b80c6a80 0x7fe1b81496d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:06.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.297+0000 7fe1bf96c640 1 -- 172.21.15.106:0/3675298521 >> 172.21.15.106:0/3675298521 conn(0x7fe1b806d6d0 msgr2=0x7fe1b80cac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:06.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.297+0000 7fe1bf96c640 1 -- 172.21.15.106:0/3675298521 shutdown_connections 2024-04-25T10:32:06.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:06.297+0000 7fe1bf96c640 1 -- 172.21.15.106:0/3675298521 wait complete. 2024-04-25T10:32:09.311 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T10:32:09.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.417+0000 7faca1a49640 1 Processor -- start 2024-04-25T10:32:09.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.417+0000 7faca1a49640 1 -- start start 2024-04-25T10:32:09.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.418+0000 7faca1a49640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fac9c0c7e30 0x7fac9c0c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:09.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.418+0000 7faca1a49640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fac9c156ef0 0x7fac9c0b4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:09.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.418+0000 7faca1a49640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac9c0ce630 con 0x7fac9c06d000 2024-04-25T10:32:09.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.418+0000 7faca1a49640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fac9c0c7c60 con 0x7fac9c0c7e30 2024-04-25T10:32:09.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.418+0000 7faca1a49640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fac9c0c7980 con 0x7fac9c156ef0 2024-04-25T10:32:09.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.418+0000 7fac9affd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fac9c0c7e30 0x7fac9c0c8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:09.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.418+0000 7fac9a7fc640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fac9c156ef0 msgr2=0x7fac9c0b4ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-04-25T10:32:09.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.418+0000 7fac9affd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fac9c0c7e30 0x7fac9c0c8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53134/0 (socket says 172.21.15.106:53134) 2024-04-25T10:32:09.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.418+0000 7fac9a7fc640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fac9c156ef0 0x7fac9c0b4ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:32:09.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.418+0000 7fac9affd640 1 -- 172.21.15.106:0/114653882 learned_addr learned my addr 172.21.15.106:0/114653882 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:32:09.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.418+0000 7fac9affd640 1 -- 172.21.15.106:0/114653882 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fac9c156ef0 msgr2=0x7fac9c0b4ce0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:32:09.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.418+0000 7fac9affd640 1 --2- 172.21.15.106:0/114653882 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fac9c156ef0 0x7fac9c0b4ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:09.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.418+0000 7fac9affd640 1 -- 172.21.15.106:0/114653882 >> v1:172.21.15.134:6789/0 conn(0x7fac9c06d000 legacy=0x7fac9c15ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:09.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.419+0000 7fac9affd640 1 -- 172.21.15.106:0/114653882 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac9c0c7010 con 0x7fac9c0c7e30 2024-04-25T10:32:09.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.419+0000 7fac9affd640 1 --2- 172.21.15.106:0/114653882 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fac9c0c7e30 0x7fac9c0c8d90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fac9c0c7de0 tx=0x7fac9402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e644f8c1db65221 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:09.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.419+0000 7fac99ffb640 1 -- 172.21.15.106:0/114653882 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac94036070 con 0x7fac9c0c7e30 2024-04-25T10:32:09.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.419+0000 7fac99ffb640 1 -- 172.21.15.106:0/114653882 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac94031600 con 0x7fac9c0c7e30 2024-04-25T10:32:09.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.419+0000 7fac99ffb640 1 -- 172.21.15.106:0/114653882 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac940319d0 con 0x7fac9c0c7e30 2024-04-25T10:32:09.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.420+0000 7faca1a49640 1 -- 172.21.15.106:0/114653882 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fac9c0c7e30 msgr2=0x7fac9c0c8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:09.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.420+0000 7faca1a49640 1 --2- 172.21.15.106:0/114653882 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fac9c0c7e30 0x7fac9c0c8d90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fac9c0c7de0 tx=0x7fac9402a6a0 comp rx=0 tx=0).stop 2024-04-25T10:32:09.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.420+0000 7faca1a49640 1 -- 172.21.15.106:0/114653882 shutdown_connections 2024-04-25T10:32:09.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.420+0000 7faca1a49640 1 --2- 172.21.15.106:0/114653882 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fac9c156ef0 0x7fac9c0b4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:09.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.420+0000 7faca1a49640 1 --2- 172.21.15.106:0/114653882 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fac9c0c7e30 0x7fac9c0c8d90 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:09.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.420+0000 7faca1a49640 1 -- 172.21.15.106:0/114653882 >> 172.21.15.106:0/114653882 conn(0x7fac9c068b10 msgr2=0x7fac9c0d1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:09.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.420+0000 7faca1a49640 1 -- 172.21.15.106:0/114653882 shutdown_connections 2024-04-25T10:32:09.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.420+0000 7faca1a49640 1 -- 172.21.15.106:0/114653882 wait complete. 2024-04-25T10:32:09.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.422+0000 7faca1a49640 1 Processor -- start 2024-04-25T10:32:09.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.422+0000 7faca1a49640 1 -- start start 2024-04-25T10:32:09.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.423+0000 7faca1a49640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac9c06d000 0x7fac9c15a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:09.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.423+0000 7faca1a49640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fac9c0c7e30 0x7fac9c0d11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:09.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.423+0000 7faca1a49640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fac9c156ef0 0x7fac9c148840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:09.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.423+0000 7faca1a49640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fac9c15d6c0 con 0x7fac9c0c7e30 2024-04-25T10:32:09.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.423+0000 7faca1a49640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fac9c15d540 con 0x7fac9c06d000 2024-04-25T10:32:09.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.423+0000 7faca1a49640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fac9c1483f0 con 0x7fac9c156ef0 2024-04-25T10:32:09.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.424+0000 7fac9a7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fac9c0c7e30 0x7fac9c0d11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:09.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.424+0000 7fac9b7fe640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fac9c156ef0 msgr2=0x7fac9c148840 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-04-25T10:32:09.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.424+0000 7fac9a7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fac9c0c7e30 0x7fac9c0d11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53142/0 (socket says 172.21.15.106:53142) 2024-04-25T10:32:09.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.424+0000 7fac9a7fc640 1 -- 172.21.15.106:0/1966857492 learned_addr learned my addr 172.21.15.106:0/1966857492 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:32:09.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.424+0000 7fac9b7fe640 1 --2- 172.21.15.106:0/1966857492 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fac9c156ef0 0x7fac9c148840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:32:09.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.424+0000 7fac9affd640 1 --2- 172.21.15.106:0/1966857492 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac9c06d000 0x7fac9c15a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:09.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.424+0000 7fac9a7fc640 1 -- 172.21.15.106:0/1966857492 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fac9c156ef0 msgr2=0x7fac9c148840 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:32:09.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.424+0000 7fac9a7fc640 1 --2- 172.21.15.106:0/1966857492 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fac9c156ef0 0x7fac9c148840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:09.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.424+0000 7fac9a7fc640 1 -- 172.21.15.106:0/1966857492 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac9c06d000 msgr2=0x7fac9c15a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:09.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.424+0000 7fac9a7fc640 1 --2- 172.21.15.106:0/1966857492 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac9c06d000 0x7fac9c15a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:09.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.424+0000 7fac9a7fc640 1 -- 172.21.15.106:0/1966857492 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac9c153460 con 0x7fac9c0c7e30 2024-04-25T10:32:09.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.424+0000 7fac9a7fc640 1 --2- 172.21.15.106:0/1966857492 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fac9c0c7e30 0x7fac9c0d11a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fac8400b530 tx=0x7fac8400b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:09.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.424+0000 7fac93fff640 1 -- 172.21.15.106:0/1966857492 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac84013020 con 0x7fac9c0c7e30 2024-04-25T10:32:09.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.424+0000 7fac93fff640 1 -- 172.21.15.106:0/1966857492 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac84003e10 con 0x7fac9c0c7e30 2024-04-25T10:32:09.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.424+0000 7fac93fff640 1 -- 172.21.15.106:0/1966857492 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac840052b0 con 0x7fac9c0c7e30 2024-04-25T10:32:09.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.424+0000 7faca1a49640 1 -- 172.21.15.106:0/1966857492 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac9c1f6710 con 0x7fac9c0c7e30 2024-04-25T10:32:09.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.424+0000 7faca1a49640 1 -- 172.21.15.106:0/1966857492 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac9c147eb0 con 0x7fac9c0c7e30 2024-04-25T10:32:09.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.425+0000 7faca1a49640 1 -- 172.21.15.106:0/1966857492 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac9c1531b0 con 0x7fac9c0c7e30 2024-04-25T10:32:09.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.426+0000 7fac93fff640 1 -- 172.21.15.106:0/1966857492 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fac84016020 con 0x7fac9c0c7e30 2024-04-25T10:32:09.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.427+0000 7fac93fff640 1 --2- 172.21.15.106:0/1966857492 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fac7c074490 0x7fac7c0769a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:09.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.427+0000 7fac93fff640 1 -- 172.21.15.106:0/1966857492 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fac84094e40 con 0x7fac9c0c7e30 2024-04-25T10:32:09.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.427+0000 7fac9affd640 1 --2- 172.21.15.106:0/1966857492 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fac7c074490 0x7fac7c0769a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:09.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.427+0000 7fac9affd640 1 --2- 172.21.15.106:0/1966857492 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fac7c074490 0x7fac7c0769a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fac94033000 tx=0x7fac94009000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:09.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.431+0000 7fac93fff640 1 -- 172.21.15.106:0/1966857492 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fac8401d020 con 0x7fac9c0c7e30 2024-04-25T10:32:09.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.699+0000 7faca1a49640 1 -- 172.21.15.106:0/1966857492 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fac9c06a3d0 con 0x7fac9c0c7e30 2024-04-25T10:32:09.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.699+0000 7fac93fff640 1 -- 172.21.15.106:0/1966857492 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1569 (secure 0 0 0) 0x7fac8405f560 con 0x7fac9c0c7e30 2024-04-25T10:32:09.702 INFO:teuthology.orchestra.run.smithi106.stdout:{"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":20,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"3ee77985-e8e4-41cf-81e9-7c774cf7891f","modified":"2024-04-25T10:29:33.446228Z","created":"2024-04-25T10:29:33.446228Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:3300","nonce":0},{"type":"v1","addr":"172.21.15.106:6789","nonce":0}]},"addr":"172.21.15.106:6789/0","public_addr":"172.21.15.106:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T10:32:09.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.701+0000 7faca1a49640 1 -- 172.21.15.106:0/1966857492 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fac7c074490 msgr2=0x7fac7c0769a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:09.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.701+0000 7faca1a49640 1 --2- 172.21.15.106:0/1966857492 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fac7c074490 0x7fac7c0769a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fac94033000 tx=0x7fac94009000 comp rx=0 tx=0).stop 2024-04-25T10:32:09.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.701+0000 7faca1a49640 1 -- 172.21.15.106:0/1966857492 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fac9c0c7e30 msgr2=0x7fac9c0d11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:09.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.701+0000 7faca1a49640 1 --2- 172.21.15.106:0/1966857492 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fac9c0c7e30 0x7fac9c0d11a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fac8400b530 tx=0x7fac8400b9f0 comp rx=0 tx=0).stop 2024-04-25T10:32:09.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.701+0000 7faca1a49640 1 -- 172.21.15.106:0/1966857492 shutdown_connections 2024-04-25T10:32:09.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.701+0000 7faca1a49640 1 --2- 172.21.15.106:0/1966857492 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fac7c074490 0x7fac7c0769a0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:09.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.701+0000 7faca1a49640 1 --2- 172.21.15.106:0/1966857492 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fac9c156ef0 0x7fac9c148840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:09.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.701+0000 7faca1a49640 1 --2- 172.21.15.106:0/1966857492 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fac9c0c7e30 0x7fac9c0d11a0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:09.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.701+0000 7faca1a49640 1 --2- 172.21.15.106:0/1966857492 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac9c06d000 0x7fac9c15a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:09.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.701+0000 7faca1a49640 1 -- 172.21.15.106:0/1966857492 >> 172.21.15.106:0/1966857492 conn(0x7fac9c068b10 msgr2=0x7fac9c1e91f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:09.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.702+0000 7faca1a49640 1 -- 172.21.15.106:0/1966857492 shutdown_connections 2024-04-25T10:32:09.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:09.702+0000 7faca1a49640 1 -- 172.21.15.106:0/1966857492 wait complete. 2024-04-25T10:32:12.716 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T10:32:12.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:12.836+0000 7f3fd1b72640 1 Processor -- start 2024-04-25T10:32:12.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:12.836+0000 7f3fd1b72640 1 -- start start 2024-04-25T10:32:12.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:12.837+0000 7f3fd1b72640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3fcc0c7e30 0x7f3fcc0c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:12.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:12.837+0000 7f3fd1b72640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3fcc0b4ce0 0x7f3fcc15ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:12.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:12.837+0000 7f3fd1b72640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3fcc0ce630 con 0x7f3fcc156ef0 2024-04-25T10:32:12.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:12.837+0000 7f3fd1b72640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f3fcc0b50f0 con 0x7f3fcc0c7e30 2024-04-25T10:32:12.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:12.837+0000 7f3fd1b72640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f3fcc0c7c60 con 0x7f3fcc0b4ce0 2024-04-25T10:32:12.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:12.837+0000 7f3fcbfff640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3fcc0b4ce0 msgr2=0x7f3fcc15ce30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-04-25T10:32:12.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:12.837+0000 7f3fcbfff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3fcc0b4ce0 0x7f3fcc15ce30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:32:12.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:12.837+0000 7f3fcb7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3fcc0c7e30 0x7f3fcc0c8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:12.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:12.837+0000 7f3fcb7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3fcc0c7e30 0x7f3fcc0c8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53148/0 (socket says 172.21.15.106:53148) 2024-04-25T10:32:12.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:12.837+0000 7f3fcb7fe640 1 -- 172.21.15.106:0/2983420107 learned_addr learned my addr 172.21.15.106:0/2983420107 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:32:12.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:12.837+0000 7f3fcb7fe640 1 -- 172.21.15.106:0/2983420107 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3fcc0b4ce0 msgr2=0x7f3fcc15ce30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:32:12.840 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:12.838+0000 7f3fcb7fe640 1 --2- 172.21.15.106:0/2983420107 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3fcc0b4ce0 0x7f3fcc15ce30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:12.838+0000 7f3fcb7fe640 1 -- 172.21.15.106:0/2983420107 >> v1:172.21.15.134:6789/0 conn(0x7f3fcc156ef0 legacy=0x7f3fcc06d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:12.841 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:12.838+0000 7f3fcb7fe640 1 -- 172.21.15.106:0/2983420107 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fcc0c7980 con 0x7f3fcc0c7e30 2024-04-25T10:32:12.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:12.838+0000 7f3fcb7fe640 1 --2- 172.21.15.106:0/2983420107 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3fcc0c7e30 0x7f3fcc0c8d90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc0c7de0 tx=0x7f3fbc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b11877be365b99c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:12.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:12.838+0000 7f3fca7fc640 1 -- 172.21.15.106:0/2983420107 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fbc037070 con 0x7f3fcc0c7e30 2024-04-25T10:32:12.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:12.838+0000 7f3fca7fc640 1 -- 172.21.15.106:0/2983420107 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fbc032440 con 0x7f3fcc0c7e30 2024-04-25T10:32:12.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:12.838+0000 7f3fca7fc640 1 -- 172.21.15.106:0/2983420107 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fbc032810 con 0x7f3fcc0c7e30 2024-04-25T10:32:12.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:12.839+0000 7f3fd1b72640 1 -- 172.21.15.106:0/2983420107 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3fcc0c7e30 msgr2=0x7f3fcc0c8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:12.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:12.839+0000 7f3fd1b72640 1 --2- 172.21.15.106:0/2983420107 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3fcc0c7e30 0x7f3fcc0c8d90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc0c7de0 tx=0x7f3fbc02a1e0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:12.839+0000 7f3fd1b72640 1 -- 172.21.15.106:0/2983420107 shutdown_connections 2024-04-25T10:32:12.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:12.839+0000 7f3fd1b72640 1 --2- 172.21.15.106:0/2983420107 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3fcc0b4ce0 0x7f3fcc15ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:12.839+0000 7f3fd1b72640 1 --2- 172.21.15.106:0/2983420107 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3fcc0c7e30 0x7f3fcc0c8d90 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:12.839+0000 7f3fd1b72640 1 -- 172.21.15.106:0/2983420107 >> 172.21.15.106:0/2983420107 conn(0x7f3fcc068b10 msgr2=0x7f3fcc0d1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:12.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:12.840+0000 7f3fd1b72640 1 -- 172.21.15.106:0/2983420107 shutdown_connections 2024-04-25T10:32:12.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:12.840+0000 7f3fd1b72640 1 -- 172.21.15.106:0/2983420107 wait complete. 2024-04-25T10:32:12.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:12.841+0000 7f3fd1b72640 1 Processor -- start 2024-04-25T10:32:12.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:12.841+0000 7f3fd1b72640 1 -- start start 2024-04-25T10:32:12.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:12.843+0000 7f3fd1b72640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3fcc0b4ce0 0x7f3fcc152b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:12.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:12.843+0000 7f3fd1b72640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3fcc0c7e30 0x7f3fcc155dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:12.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:12.843+0000 7f3fd1b72640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3fcc156ef0 0x7f3fcc14a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:12.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:12.843+0000 7f3fcaffd640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3fcc0c7e30 msgr2=0x7f3fcc155dd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-04-25T10:32:12.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:12.843+0000 7f3fcbfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3fcc156ef0 0x7f3fcc14a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:12.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:12.843+0000 7f3fcaffd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3fcc0c7e30 0x7f3fcc155dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:32:12.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:12.843+0000 7f3fd1b72640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3fcc06d710 con 0x7f3fcc156ef0 2024-04-25T10:32:12.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:12.843+0000 7f3fd1b72640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3fcc0c72f0 con 0x7f3fcc0b4ce0 2024-04-25T10:32:12.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:12.843+0000 7f3fd1b72640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f3fcc06d890 con 0x7f3fcc0c7e30 2024-04-25T10:32:12.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:12.843+0000 7f3fcbfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3fcc156ef0 0x7f3fcc14a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53164/0 (socket says 172.21.15.106:53164) 2024-04-25T10:32:12.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:12.843+0000 7f3fcbfff640 1 -- 172.21.15.106:0/3611347793 learned_addr learned my addr 172.21.15.106:0/3611347793 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:32:12.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:12.843+0000 7f3fcb7fe640 1 --2- 172.21.15.106:0/3611347793 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3fcc0b4ce0 0x7f3fcc152b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:12.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:12.843+0000 7f3fcbfff640 1 -- 172.21.15.106:0/3611347793 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3fcc0c7e30 msgr2=0x7f3fcc155dd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:32:12.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:12.843+0000 7f3fcbfff640 1 --2- 172.21.15.106:0/3611347793 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3fcc0c7e30 0x7f3fcc155dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:12.843+0000 7f3fcbfff640 1 -- 172.21.15.106:0/3611347793 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3fcc0b4ce0 msgr2=0x7f3fcc152b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:12.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:12.843+0000 7f3fcbfff640 1 --2- 172.21.15.106:0/3611347793 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3fcc0b4ce0 0x7f3fcc152b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:12.843+0000 7f3fcbfff640 1 -- 172.21.15.106:0/3611347793 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fcc0cc230 con 0x7f3fcc156ef0 2024-04-25T10:32:12.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:12.843+0000 7f3fcbfff640 1 --2- 172.21.15.106:0/3611347793 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3fcc156ef0 0x7f3fcc14a870 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3fc400b9c0 tx=0x7f3fc400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:12.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:12.844+0000 7f3fb3fff640 1 -- 172.21.15.106:0/3611347793 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc400cb20 con 0x7f3fcc156ef0 2024-04-25T10:32:12.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:12.844+0000 7f3fb3fff640 1 -- 172.21.15.106:0/3611347793 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fc4010070 con 0x7f3fcc156ef0 2024-04-25T10:32:12.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:12.844+0000 7f3fb3fff640 1 -- 172.21.15.106:0/3611347793 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc4015470 con 0x7f3fcc156ef0 2024-04-25T10:32:12.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:12.844+0000 7f3fd1b72640 1 -- 172.21.15.106:0/3611347793 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fcc1f36b0 con 0x7f3fcc156ef0 2024-04-25T10:32:12.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:12.844+0000 7f3fd1b72640 1 -- 172.21.15.106:0/3611347793 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fcc1f3c70 con 0x7f3fcc156ef0 2024-04-25T10:32:12.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:12.845+0000 7f3fd1b72640 1 -- 172.21.15.106:0/3611347793 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fcc0cbf90 con 0x7f3fcc156ef0 2024-04-25T10:32:12.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:12.846+0000 7f3fb3fff640 1 -- 172.21.15.106:0/3611347793 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3fc401e3a0 con 0x7f3fcc156ef0 2024-04-25T10:32:12.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:12.846+0000 7f3fb3fff640 1 --2- 172.21.15.106:0/3611347793 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f3fb8074460 0x7f3fb8076970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:12.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:12.846+0000 7f3fcb7fe640 1 --2- 172.21.15.106:0/3611347793 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f3fb8074460 0x7f3fb8076970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:12.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:12.846+0000 7f3fb3fff640 1 -- 172.21.15.106:0/3611347793 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f3fc4095340 con 0x7f3fcc156ef0 2024-04-25T10:32:12.849 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:12.847+0000 7f3fcb7fe640 1 --2- 172.21.15.106:0/3611347793 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f3fb8074460 0x7f3fb8076970 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc034000 tx=0x7f3fbc02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:12.853 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:12.851+0000 7f3fb3fff640 1 -- 172.21.15.106:0/3611347793 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3fcc0cbf90 con 0x7f3fcc156ef0 2024-04-25T10:32:13.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:13.154+0000 7f3fd1b72640 1 -- 172.21.15.106:0/3611347793 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f3fcc1f34a0 con 0x7f3fcc156ef0 2024-04-25T10:32:13.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:13.155+0000 7f3fb3fff640 1 -- 172.21.15.106:0/3611347793 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1569 (secure 0 0 0) 0x7f3fc405fae0 con 0x7f3fcc156ef0 2024-04-25T10:32:13.157 INFO:teuthology.orchestra.run.smithi106.stdout:{"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":23,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"3ee77985-e8e4-41cf-81e9-7c774cf7891f","modified":"2024-04-25T10:29:33.446228Z","created":"2024-04-25T10:29:33.446228Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:3300","nonce":0},{"type":"v1","addr":"172.21.15.106:6789","nonce":0}]},"addr":"172.21.15.106:6789/0","public_addr":"172.21.15.106:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T10:32:13.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:13.157+0000 7f3fd1b72640 1 -- 172.21.15.106:0/3611347793 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f3fb8074460 msgr2=0x7f3fb8076970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:13.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:13.157+0000 7f3fd1b72640 1 --2- 172.21.15.106:0/3611347793 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f3fb8074460 0x7f3fb8076970 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc034000 tx=0x7f3fbc02e000 comp rx=0 tx=0).stop 2024-04-25T10:32:13.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:13.157+0000 7f3fd1b72640 1 -- 172.21.15.106:0/3611347793 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3fcc156ef0 msgr2=0x7f3fcc14a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:13.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:13.157+0000 7f3fd1b72640 1 --2- 172.21.15.106:0/3611347793 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3fcc156ef0 0x7f3fcc14a870 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3fc400b9c0 tx=0x7f3fc400be80 comp rx=0 tx=0).stop 2024-04-25T10:32:13.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:13.158+0000 7f3fd1b72640 1 -- 172.21.15.106:0/3611347793 shutdown_connections 2024-04-25T10:32:13.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:13.158+0000 7f3fd1b72640 1 --2- 172.21.15.106:0/3611347793 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f3fb8074460 0x7f3fb8076970 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:13.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:13.158+0000 7f3fd1b72640 1 --2- 172.21.15.106:0/3611347793 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3fcc156ef0 0x7f3fcc14a870 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:13.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:13.158+0000 7f3fd1b72640 1 --2- 172.21.15.106:0/3611347793 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3fcc0c7e30 0x7f3fcc155dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:13.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:13.158+0000 7f3fd1b72640 1 --2- 172.21.15.106:0/3611347793 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3fcc0b4ce0 0x7f3fcc152b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:13.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:13.158+0000 7f3fd1b72640 1 -- 172.21.15.106:0/3611347793 >> 172.21.15.106:0/3611347793 conn(0x7f3fcc068b10 msgr2=0x7f3fcc14cbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:13.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:13.158+0000 7f3fd1b72640 1 -- 172.21.15.106:0/3611347793 shutdown_connections 2024-04-25T10:32:13.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:13.158+0000 7f3fd1b72640 1 -- 172.21.15.106:0/3611347793 wait complete. 2024-04-25T10:32:16.172 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T10:32:16.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:16.279+0000 7f9142ed8640 1 Processor -- start 2024-04-25T10:32:16.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:16.279+0000 7f9142ed8640 1 -- start start 2024-04-25T10:32:16.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:16.281+0000 7f9142ed8640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f913c0cc250 0x7f913c159a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:16.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:16.281+0000 7f9142ed8640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f913c0c6a80 0x7f913c0c8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:16.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:16.281+0000 7f9142ed8640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f913c0c9d40 con 0x7f913c0cc6b0 2024-04-25T10:32:16.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:16.281+0000 7f9142ed8640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f913c159fa0 con 0x7f913c0c6a80 2024-04-25T10:32:16.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:16.281+0000 7f9142ed8640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f913c0c64c0 con 0x7f913c0cc250 2024-04-25T10:32:16.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:16.281+0000 7f9140c74640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f913c0cc250 msgr2=0x7f913c159a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-04-25T10:32:16.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:16.281+0000 7f9141475640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f913c0c6a80 0x7f913c0c8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:16.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:16.281+0000 7f9140c74640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f913c0cc250 0x7f913c159a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:32:16.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:16.281+0000 7f9141475640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f913c0c6a80 0x7f913c0c8a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:52782/0 (socket says 172.21.15.106:52782) 2024-04-25T10:32:16.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:16.281+0000 7f9141475640 1 -- 172.21.15.106:0/2257222631 learned_addr learned my addr 172.21.15.106:0/2257222631 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:32:16.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:16.281+0000 7f9141475640 1 -- 172.21.15.106:0/2257222631 >> v2:172.21.15.106:3300/0 conn(0x7f913c0c6a80 msgr2=0x7f913c0c8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T10:32:16.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:16.281+0000 7f9141475640 1 -- 172.21.15.106:0/2257222631 >> v2:172.21.15.106:3300/0 conn(0x7f913c0c6a80 msgr2=0x7f913c0c8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:32:16.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:16.281+0000 7f9141475640 1 --2- 172.21.15.106:0/2257222631 >> v2:172.21.15.106:3300/0 conn(0x7f913c0c6a80 0x7f913c0c8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:32:16.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:16.281+0000 7f9141475640 1 --2- 172.21.15.106:0/2257222631 >> v2:172.21.15.106:3300/0 conn(0x7f913c0c6a80 0x7f913c0c8a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:32:16.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:16.482+0000 7f9140c74640 1 -- 172.21.15.106:0/2257222631 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f913c0cc250 msgr2=0x7f913c159a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-04-25T10:32:16.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:16.482+0000 7f9140c74640 1 --2- 172.21.15.106:0/2257222631 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f913c0cc250 0x7f913c159a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T10:32:16.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:16.482+0000 7f9141475640 1 --2- 172.21.15.106:0/2257222631 >> v2:172.21.15.106:3300/0 conn(0x7f913c0c6a80 0x7f913c0c8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:16.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:16.482+0000 7f9141475640 1 -- 172.21.15.106:0/2257222631 >> v2:172.21.15.106:3300/0 conn(0x7f913c0c6a80 msgr2=0x7f913c0c8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T10:32:16.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:16.482+0000 7f9141475640 1 -- 172.21.15.106:0/2257222631 >> v2:172.21.15.106:3300/0 conn(0x7f913c0c6a80 msgr2=0x7f913c0c8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:32:16.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:16.482+0000 7f9141475640 1 --2- 172.21.15.106:0/2257222631 >> v2:172.21.15.106:3300/0 conn(0x7f913c0c6a80 0x7f913c0c8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:32:16.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:16.482+0000 7f9141475640 1 --2- 172.21.15.106:0/2257222631 >> v2:172.21.15.106:3300/0 conn(0x7f913c0c6a80 0x7f913c0c8a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T10:32:16.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:16.883+0000 7f9141475640 1 --2- 172.21.15.106:0/2257222631 >> v2:172.21.15.106:3300/0 conn(0x7f913c0c6a80 0x7f913c0c8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:16.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:16.883+0000 7f9140c74640 1 -- 172.21.15.106:0/2257222631 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f913c0cc250 msgr2=0x7f913c159a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-04-25T10:32:16.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:16.883+0000 7f9140c74640 1 --2- 172.21.15.106:0/2257222631 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f913c0cc250 0x7f913c159a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T10:32:16.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:16.883+0000 7f9141475640 1 -- 172.21.15.106:0/2257222631 >> v2:172.21.15.106:3300/0 conn(0x7f913c0c6a80 msgr2=0x7f913c0c8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T10:32:16.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:16.883+0000 7f9141475640 1 -- 172.21.15.106:0/2257222631 >> v2:172.21.15.106:3300/0 conn(0x7f913c0c6a80 msgr2=0x7f913c0c8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:32:16.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:16.883+0000 7f9141475640 1 --2- 172.21.15.106:0/2257222631 >> v2:172.21.15.106:3300/0 conn(0x7f913c0c6a80 0x7f913c0c8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:32:16.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:16.883+0000 7f9141475640 1 --2- 172.21.15.106:0/2257222631 >> v2:172.21.15.106:3300/0 conn(0x7f913c0c6a80 0x7f913c0c8a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T10:32:17.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:17.684+0000 7f9141475640 1 --2- 172.21.15.106:0/2257222631 >> v2:172.21.15.106:3300/0 conn(0x7f913c0c6a80 0x7f913c0c8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:17.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:17.684+0000 7f9140c74640 1 -- 172.21.15.106:0/2257222631 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f913c0cc250 msgr2=0x7f913c159a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-04-25T10:32:17.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:17.684+0000 7f9140c74640 1 --2- 172.21.15.106:0/2257222631 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f913c0cc250 0x7f913c159a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T10:32:17.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:17.684+0000 7f9141475640 1 -- 172.21.15.106:0/2257222631 >> v2:172.21.15.106:3300/0 conn(0x7f913c0c6a80 msgr2=0x7f913c0c8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T10:32:17.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:17.684+0000 7f9141475640 1 -- 172.21.15.106:0/2257222631 >> v2:172.21.15.106:3300/0 conn(0x7f913c0c6a80 msgr2=0x7f913c0c8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:32:17.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:17.684+0000 7f9141475640 1 --2- 172.21.15.106:0/2257222631 >> v2:172.21.15.106:3300/0 conn(0x7f913c0c6a80 0x7f913c0c8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:32:17.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:17.684+0000 7f9141475640 1 --2- 172.21.15.106:0/2257222631 >> v2:172.21.15.106:3300/0 conn(0x7f913c0c6a80 0x7f913c0c8a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T10:32:19.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:19.281+0000 7f913a7fc640 1 -- 172.21.15.106:0/2257222631 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f913c0cc250 msgr2=0x7f913c159a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:32:19.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:19.281+0000 7f913a7fc640 1 --2- 172.21.15.106:0/2257222631 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f913c0cc250 0x7f913c159a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:19.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:19.281+0000 7f913a7fc640 1 -- 172.21.15.106:0/2257222631 >> v2:172.21.15.106:3300/0 conn(0x7f913c0c6a80 msgr2=0x7f913c0c8a70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:32:19.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:19.281+0000 7f913a7fc640 1 --2- 172.21.15.106:0/2257222631 >> v2:172.21.15.106:3300/0 conn(0x7f913c0c6a80 0x7f913c0c8a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:19.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:19.281+0000 7f913a7fc640 1 -- 172.21.15.106:0/2257222631 >> v1:172.21.15.134:6789/0 conn(0x7f913c0cc6b0 legacy=0x7f913c153c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:19.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:19.281+0000 7f913a7fc640 1 --2- 172.21.15.106:0/2257222631 >> v2:172.21.15.134:3300/0 conn(0x7f91200073c0 0x7f91200098b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:19.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:19.281+0000 7f913a7fc640 1 -- 172.21.15.106:0/2257222631 --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f913c0c67a0 con 0x7f91200036c0 2024-04-25T10:32:19.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:19.281+0000 7f913a7fc640 1 -- 172.21.15.106:0/2257222631 --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f913c159fa0 con 0x7f91200073c0 2024-04-25T10:32:19.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:19.281+0000 7f913bfff640 1 --2- 172.21.15.106:0/2257222631 >> v2:172.21.15.134:3300/0 conn(0x7f91200073c0 0x7f91200098b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:19.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:19.281+0000 7f913bfff640 1 -- 172.21.15.106:0/2257222631 >> v2:172.21.15.134:3300/0 conn(0x7f91200073c0 msgr2=0x7f91200098b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T10:32:19.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:19.281+0000 7f913bfff640 1 -- 172.21.15.106:0/2257222631 >> v2:172.21.15.134:3300/0 conn(0x7f91200073c0 msgr2=0x7f91200098b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:32:19.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:19.281+0000 7f913bfff640 1 --2- 172.21.15.106:0/2257222631 >> v2:172.21.15.134:3300/0 conn(0x7f91200073c0 0x7f91200098b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:32:19.285 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:19.281+0000 7f913bfff640 1 --2- 172.21.15.106:0/2257222631 >> v2:172.21.15.134:3300/0 conn(0x7f91200073c0 0x7f91200098b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:32:19.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:19.482+0000 7f913bfff640 1 --2- 172.21.15.106:0/2257222631 >> v2:172.21.15.134:3300/0 conn(0x7f91200073c0 0x7f91200098b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:19.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:19.482+0000 7f913bfff640 1 -- 172.21.15.106:0/2257222631 >> v2:172.21.15.134:3300/0 conn(0x7f91200073c0 msgr2=0x7f91200098b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T10:32:19.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:19.482+0000 7f913bfff640 1 -- 172.21.15.106:0/2257222631 >> v2:172.21.15.134:3300/0 conn(0x7f91200073c0 msgr2=0x7f91200098b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:32:19.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:19.482+0000 7f913bfff640 1 --2- 172.21.15.106:0/2257222631 >> v2:172.21.15.134:3300/0 conn(0x7f91200073c0 0x7f91200098b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:32:19.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:19.482+0000 7f913bfff640 1 --2- 172.21.15.106:0/2257222631 >> v2:172.21.15.134:3300/0 conn(0x7f91200073c0 0x7f91200098b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T10:32:19.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:19.883+0000 7f913bfff640 1 --2- 172.21.15.106:0/2257222631 >> v2:172.21.15.134:3300/0 conn(0x7f91200073c0 0x7f91200098b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:19.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:19.883+0000 7f913bfff640 1 -- 172.21.15.106:0/2257222631 >> v2:172.21.15.134:3300/0 conn(0x7f91200073c0 msgr2=0x7f91200098b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T10:32:19.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:19.883+0000 7f913bfff640 1 -- 172.21.15.106:0/2257222631 >> v2:172.21.15.134:3300/0 conn(0x7f91200073c0 msgr2=0x7f91200098b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:32:19.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:19.883+0000 7f913bfff640 1 --2- 172.21.15.106:0/2257222631 >> v2:172.21.15.134:3300/0 conn(0x7f91200073c0 0x7f91200098b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:32:19.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:19.883+0000 7f913bfff640 1 --2- 172.21.15.106:0/2257222631 >> v2:172.21.15.134:3300/0 conn(0x7f91200073c0 0x7f91200098b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T10:32:20.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.374+0000 7f913b7fe640 1 -- 172.21.15.106:0/2257222631 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3750727887 0 0) 0x7f913c0c67a0 con 0x7f91200036c0 2024-04-25T10:32:20.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.374+0000 7f913b7fe640 1 -- 172.21.15.106:0/2257222631 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f912c003520 con 0x7f91200036c0 2024-04-25T10:32:20.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.375+0000 7f913b7fe640 1 -- 172.21.15.106:0/2257222631 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 443887442 0 0) 0x7f912c003520 con 0x7f91200036c0 2024-04-25T10:32:20.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.375+0000 7f913b7fe640 1 -- 172.21.15.106:0/2257222631 >> v2:172.21.15.134:3300/0 conn(0x7f91200073c0 msgr2=0x7f91200098b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:32:20.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.375+0000 7f913b7fe640 1 --2- 172.21.15.106:0/2257222631 >> v2:172.21.15.134:3300/0 conn(0x7f91200073c0 0x7f91200098b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:20.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.375+0000 7f913b7fe640 1 -- 172.21.15.106:0/2257222631 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f912000b010 con 0x7f91200036c0 2024-04-25T10:32:20.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.375+0000 7f913b7fe640 1 -- 172.21.15.106:0/2257222631 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3776459350 0 0) 0x7f9130004340 con 0x7f91200036c0 2024-04-25T10:32:20.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.375+0000 7f913b7fe640 1 -- 172.21.15.106:0/2257222631 >> v1:172.21.15.106:6789/0 conn(0x7f91200036c0 legacy=0x7f9120005b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:20.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.375+0000 7f9141475640 1 -- 172.21.15.106:0/2257222631 reap_dead start 2024-04-25T10:32:20.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.375+0000 7f913b7fe640 1 --2- 172.21.15.106:0/2257222631 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f912c0039b0 0x7f912c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:20.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.375+0000 7f913b7fe640 1 --2- 172.21.15.106:0/2257222631 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f912c004800 0x7f912c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:20.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.375+0000 7f913b7fe640 1 --2- 172.21.15.106:0/2257222631 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f912c005150 0x7f912c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:20.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.375+0000 7f913b7fe640 1 -- 172.21.15.106:0/2257222631 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f913c159fa0 con 0x7f912c004800 2024-04-25T10:32:20.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.375+0000 7f913b7fe640 1 -- 172.21.15.106:0/2257222631 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f912c00c090 con 0x7f912c005150 2024-04-25T10:32:20.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.375+0000 7f913b7fe640 1 -- 172.21.15.106:0/2257222631 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f912c00c290 con 0x7f912c0039b0 2024-04-25T10:32:20.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.376+0000 7f913bfff640 1 --2- 172.21.15.106:0/2257222631 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f912c004800 0x7f912c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:20.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.376+0000 7f9140c74640 1 -- 172.21.15.106:0/2257222631 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f912c0039b0 msgr2=0x7f912c003dc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-04-25T10:32:20.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.376+0000 7f9140c74640 1 --2- 172.21.15.106:0/2257222631 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f912c0039b0 0x7f912c003dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:32:20.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.376+0000 7f9141475640 1 --2- 172.21.15.106:0/2257222631 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f912c005150 0x7f912c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:20.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.376+0000 7f913bfff640 1 -- 172.21.15.106:0/2257222631 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f912c0039b0 msgr2=0x7f912c003dc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:32:20.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.376+0000 7f913bfff640 1 --2- 172.21.15.106:0/2257222631 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f912c0039b0 0x7f912c003dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:20.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.376+0000 7f913bfff640 1 -- 172.21.15.106:0/2257222631 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f912c005150 msgr2=0x7f912c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:20.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.376+0000 7f913bfff640 1 --2- 172.21.15.106:0/2257222631 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f912c005150 0x7f912c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:20.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.376+0000 7f913bfff640 1 -- 172.21.15.106:0/2257222631 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f912c003520 con 0x7f912c004800 2024-04-25T10:32:20.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.377+0000 7f913bfff640 1 --2- 172.21.15.106:0/2257222631 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f912c004800 0x7f912c004c10 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9128007c30 tx=0x7f912c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10d2e22146acd518 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:20.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.380+0000 7f913b7fe640 1 -- 172.21.15.106:0/2257222631 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9128036070 con 0x7f912c004800 2024-04-25T10:32:20.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.380+0000 7f913b7fe640 1 -- 172.21.15.106:0/2257222631 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91280050d0 con 0x7f912c004800 2024-04-25T10:32:20.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.380+0000 7f913b7fe640 1 -- 172.21.15.106:0/2257222631 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9128001430 con 0x7f912c004800 2024-04-25T10:32:20.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.381+0000 7f9142ed8640 1 -- 172.21.15.106:0/2257222631 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f912c004800 msgr2=0x7f912c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:20.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.381+0000 7f9142ed8640 1 --2- 172.21.15.106:0/2257222631 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f912c004800 0x7f912c004c10 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9128007c30 tx=0x7f912c005830 comp rx=0 tx=0).stop 2024-04-25T10:32:20.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.381+0000 7f9142ed8640 1 -- 172.21.15.106:0/2257222631 shutdown_connections 2024-04-25T10:32:20.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.381+0000 7f9142ed8640 1 --2- 172.21.15.106:0/2257222631 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f912c005150 0x7f912c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:20.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.381+0000 7f9142ed8640 1 --2- 172.21.15.106:0/2257222631 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f912c0039b0 0x7f912c003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:20.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.381+0000 7f9142ed8640 1 --2- 172.21.15.106:0/2257222631 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f912c004800 0x7f912c004c10 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:20.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.381+0000 7f9142ed8640 1 -- 172.21.15.106:0/2257222631 >> 172.21.15.106:0/2257222631 conn(0x7f913c06d6d0 msgr2=0x7f913c06dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:20.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.381+0000 7f9142ed8640 1 -- 172.21.15.106:0/2257222631 shutdown_connections 2024-04-25T10:32:20.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.381+0000 7f9142ed8640 1 -- 172.21.15.106:0/2257222631 wait complete. 2024-04-25T10:32:20.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.383+0000 7f9142ed8640 1 Processor -- start 2024-04-25T10:32:20.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.383+0000 7f9142ed8640 1 -- start start 2024-04-25T10:32:20.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.384+0000 7f9142ed8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f912c0039b0 0x7f913c1f2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:20.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.384+0000 7f9142ed8640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f912c004800 0x7f913c149a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:20.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.384+0000 7f9142ed8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f912c005150 0x7f913c0ce180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:20.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.384+0000 7f9142ed8640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f913c0c9920 con 0x7f912c005150 2024-04-25T10:32:20.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.384+0000 7f9142ed8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f913c0c7450 con 0x7f912c0039b0 2024-04-25T10:32:20.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.384+0000 7f9142ed8640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f913c0c9aa0 con 0x7f912c004800 2024-04-25T10:32:20.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.384+0000 7f913bfff640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f912c004800 msgr2=0x7f913c149a50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-04-25T10:32:20.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.384+0000 7f913bfff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f912c004800 0x7f913c149a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:32:20.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.384+0000 7f9141475640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f912c005150 0x7f913c0ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:20.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.384+0000 7f9141475640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f912c005150 0x7f913c0ce180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:52838/0 (socket says 172.21.15.106:52838) 2024-04-25T10:32:20.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.384+0000 7f9141475640 1 -- 172.21.15.106:0/1828507074 learned_addr learned my addr 172.21.15.106:0/1828507074 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:32:20.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.384+0000 7f9140c74640 1 --2- 172.21.15.106:0/1828507074 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f912c0039b0 0x7f913c1f2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:20.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.385+0000 7f9141475640 1 -- 172.21.15.106:0/1828507074 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f912c004800 msgr2=0x7f913c149a50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:32:20.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.385+0000 7f9141475640 1 --2- 172.21.15.106:0/1828507074 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f912c004800 0x7f913c149a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:20.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.385+0000 7f9141475640 1 -- 172.21.15.106:0/1828507074 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f912c0039b0 msgr2=0x7f913c1f2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:20.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.385+0000 7f9141475640 1 --2- 172.21.15.106:0/1828507074 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f912c0039b0 0x7f913c1f2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:20.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.385+0000 7f9141475640 1 -- 172.21.15.106:0/1828507074 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f913c165390 con 0x7f912c005150 2024-04-25T10:32:20.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.385+0000 7f9141475640 1 --2- 172.21.15.106:0/1828507074 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f912c005150 0x7f913c0ce180 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9120007330 tx=0x7f9134014e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:20.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.385+0000 7f91397fa640 1 -- 172.21.15.106:0/1828507074 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f913401b030 con 0x7f912c005150 2024-04-25T10:32:20.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.385+0000 7f91397fa640 1 -- 172.21.15.106:0/1828507074 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9134012ac0 con 0x7f912c005150 2024-04-25T10:32:20.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.385+0000 7f9142ed8640 1 -- 172.21.15.106:0/1828507074 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f913c165910 con 0x7f912c005150 2024-04-25T10:32:20.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.385+0000 7f9142ed8640 1 -- 172.21.15.106:0/1828507074 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f913c1f5ba0 con 0x7f912c005150 2024-04-25T10:32:20.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.385+0000 7f91397fa640 1 -- 172.21.15.106:0/1828507074 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9134012cb0 con 0x7f912c005150 2024-04-25T10:32:20.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.387+0000 7f9142ed8640 1 -- 172.21.15.106:0/1828507074 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f911c005290 con 0x7f912c005150 2024-04-25T10:32:20.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.388+0000 7f91397fa640 1 -- 172.21.15.106:0/1828507074 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9134015040 con 0x7f912c005150 2024-04-25T10:32:20.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.388+0000 7f91397fa640 1 --2- 172.21.15.106:0/1828507074 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f91200751b0 0x7f91200776c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:20.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.388+0000 7f91397fa640 1 -- 172.21.15.106:0/1828507074 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f91340958f0 con 0x7f912c005150 2024-04-25T10:32:20.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.388+0000 7f9140c74640 1 --2- 172.21.15.106:0/1828507074 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f91200751b0 0x7f91200776c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:20.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.389+0000 7f9140c74640 1 --2- 172.21.15.106:0/1828507074 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f91200751b0 0x7f91200776c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f91300038c0 tx=0x7f91300037b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:20.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.397+0000 7f91397fa640 1 -- 172.21.15.106:0/1828507074 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f913405f9b0 con 0x7f912c005150 2024-04-25T10:32:20.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.652+0000 7f9142ed8640 1 -- 172.21.15.106:0/1828507074 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f911c005850 con 0x7f912c005150 2024-04-25T10:32:20.655 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.653+0000 7f91397fa640 1 -- 172.21.15.106:0/1828507074 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1562 (secure 0 0 0) 0x7f9134063cb0 con 0x7f912c005150 2024-04-25T10:32:20.655 INFO:teuthology.orchestra.run.smithi106.stdout:{"election_epoch":14,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"3ee77985-e8e4-41cf-81e9-7c774cf7891f","modified":"2024-04-25T10:29:33.446228Z","created":"2024-04-25T10:29:33.446228Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:3300","nonce":0},{"type":"v1","addr":"172.21.15.106:6789","nonce":0}]},"addr":"172.21.15.106:6789/0","public_addr":"172.21.15.106:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T10:32:20.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.655+0000 7f9142ed8640 1 -- 172.21.15.106:0/1828507074 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f91200751b0 msgr2=0x7f91200776c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:20.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.655+0000 7f9142ed8640 1 --2- 172.21.15.106:0/1828507074 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f91200751b0 0x7f91200776c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f91300038c0 tx=0x7f91300037b0 comp rx=0 tx=0).stop 2024-04-25T10:32:20.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.655+0000 7f9142ed8640 1 -- 172.21.15.106:0/1828507074 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f912c005150 msgr2=0x7f913c0ce180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:20.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.655+0000 7f9142ed8640 1 --2- 172.21.15.106:0/1828507074 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f912c005150 0x7f913c0ce180 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9120007330 tx=0x7f9134014e80 comp rx=0 tx=0).stop 2024-04-25T10:32:20.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.656+0000 7f9142ed8640 1 -- 172.21.15.106:0/1828507074 shutdown_connections 2024-04-25T10:32:20.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.656+0000 7f9142ed8640 1 --2- 172.21.15.106:0/1828507074 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f91200751b0 0x7f91200776c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:20.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.656+0000 7f9142ed8640 1 --2- 172.21.15.106:0/1828507074 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f912c005150 0x7f913c0ce180 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:20.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.656+0000 7f9142ed8640 1 --2- 172.21.15.106:0/1828507074 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f912c004800 0x7f913c149a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:20.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.656+0000 7f9142ed8640 1 --2- 172.21.15.106:0/1828507074 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f912c0039b0 0x7f913c1f2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:20.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.656+0000 7f9142ed8640 1 -- 172.21.15.106:0/1828507074 >> 172.21.15.106:0/1828507074 conn(0x7f913c06d6d0 msgr2=0x7f913c0cab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:20.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.656+0000 7f9142ed8640 1 -- 172.21.15.106:0/1828507074 shutdown_connections 2024-04-25T10:32:20.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.656+0000 7f9142ed8640 1 -- 172.21.15.106:0/1828507074 wait complete. 2024-04-25T10:32:20.669 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-04-25T10:32:20.669 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-04-25T10:32:20.669 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-25T10:32:20.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.777+0000 7fea18c96640 1 Processor -- start 2024-04-25T10:32:20.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.777+0000 7fea18c96640 1 -- start start 2024-04-25T10:32:20.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.779+0000 7fea18c96640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fea140c8d90 0x7fea14156ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:20.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.779+0000 7fea18c96640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fea140b4ce0 0x7fea1415ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:20.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.780+0000 7fea18c96640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea140ce630 con 0x7fea140c7e30 2024-04-25T10:32:20.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.780+0000 7fea18c96640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fea140b50f0 con 0x7fea140b4ce0 2024-04-25T10:32:20.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.780+0000 7fea18c96640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fea140c7c60 con 0x7fea140c8d90 2024-04-25T10:32:20.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.780+0000 7fea1259c640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fea140c7e30 0x7fea1406d000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:39384/0 (socket says 172.21.15.106:39384) 2024-04-25T10:32:20.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.780+0000 7fea1259c640 1 -- 172.21.15.106:0/1440390609 learned_addr learned my addr 172.21.15.106:0/1440390609 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:32:20.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.780+0000 7fea12d9d640 1 --2- 172.21.15.106:0/1440390609 >> v2:172.21.15.106:3300/0 conn(0x7fea140b4ce0 0x7fea1415ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:20.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.780+0000 7fea11d9b640 1 --2- 172.21.15.106:0/1440390609 >> v2:172.21.15.134:3300/0 conn(0x7fea140c8d90 0x7fea14156ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:20.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.780+0000 7fea12d9d640 1 -- 172.21.15.106:0/1440390609 >> v2:172.21.15.134:3300/0 conn(0x7fea140c8d90 msgr2=0x7fea14156ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:20.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.780+0000 7fea12d9d640 1 --2- 172.21.15.106:0/1440390609 >> v2:172.21.15.134:3300/0 conn(0x7fea140c8d90 0x7fea14156ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:20.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.780+0000 7fea12d9d640 1 -- 172.21.15.106:0/1440390609 >> v1:172.21.15.106:6789/0 conn(0x7fea140c7e30 legacy=0x7fea1406d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:20.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.780+0000 7fea12d9d640 1 -- 172.21.15.106:0/1440390609 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea140c7980 con 0x7fea140b4ce0 2024-04-25T10:32:20.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.781+0000 7fea12d9d640 1 --2- 172.21.15.106:0/1440390609 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea140b4ce0 0x7fea1415ce30 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fea1408ffe0 tx=0x7fea0c02a4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0573803dae3a35a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:20.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.781+0000 7fea1159a640 1 -- 172.21.15.106:0/1440390609 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0c007740 con 0x7fea140b4ce0 2024-04-25T10:32:20.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.781+0000 7fea1159a640 1 -- 172.21.15.106:0/1440390609 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea0c0078e0 con 0x7fea140b4ce0 2024-04-25T10:32:20.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.782+0000 7fea1159a640 1 -- 172.21.15.106:0/1440390609 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0c007cb0 con 0x7fea140b4ce0 2024-04-25T10:32:20.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.782+0000 7fea18c96640 1 -- 172.21.15.106:0/1440390609 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea140b4ce0 msgr2=0x7fea1415ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:20.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.782+0000 7fea18c96640 1 --2- 172.21.15.106:0/1440390609 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea140b4ce0 0x7fea1415ce30 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fea1408ffe0 tx=0x7fea0c02a4e0 comp rx=0 tx=0).stop 2024-04-25T10:32:20.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.783+0000 7fea18c96640 1 -- 172.21.15.106:0/1440390609 shutdown_connections 2024-04-25T10:32:20.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.783+0000 7fea18c96640 1 --2- 172.21.15.106:0/1440390609 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea140b4ce0 0x7fea1415ce30 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:20.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.783+0000 7fea18c96640 1 --2- 172.21.15.106:0/1440390609 >> v2:172.21.15.134:3300/0 conn(0x7fea140c8d90 0x7fea14156ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:20.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.783+0000 7fea18c96640 1 -- 172.21.15.106:0/1440390609 >> 172.21.15.106:0/1440390609 conn(0x7fea14068b10 msgr2=0x7fea140d1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:20.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.783+0000 7fea18c96640 1 -- 172.21.15.106:0/1440390609 shutdown_connections 2024-04-25T10:32:20.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.783+0000 7fea18c96640 1 -- 172.21.15.106:0/1440390609 wait complete. 2024-04-25T10:32:20.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.785+0000 7fea18c96640 1 Processor -- start 2024-04-25T10:32:20.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.785+0000 7fea18c96640 1 -- start start 2024-04-25T10:32:20.791 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.786+0000 7fea18c96640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fea140b4ce0 0x7fea14152aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:20.791 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.786+0000 7fea18c96640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea140c7e30 0x7fea14151150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:20.791 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.786+0000 7fea18c96640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea140c8d90 0x7fea1415e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:20.791 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.786+0000 7fea18c96640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fea1406d710 con 0x7fea140c7e30 2024-04-25T10:32:20.791 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.786+0000 7fea18c96640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fea140c72f0 con 0x7fea140c8d90 2024-04-25T10:32:20.791 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.786+0000 7fea18c96640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fea1406d890 con 0x7fea140b4ce0 2024-04-25T10:32:20.791 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.786+0000 7fea11d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea140c7e30 0x7fea14151150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:20.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.786+0000 7fea11d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea140c7e30 0x7fea14151150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:52846/0 (socket says 172.21.15.106:52846) 2024-04-25T10:32:20.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.786+0000 7fea11d9b640 1 -- 172.21.15.106:0/3053976272 learned_addr learned my addr 172.21.15.106:0/3053976272 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:32:20.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.786+0000 7fea1259c640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fea140b4ce0 msgr2=0x7fea14152aa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-04-25T10:32:20.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.786+0000 7fea1259c640 1 --2- 172.21.15.106:0/3053976272 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fea140b4ce0 0x7fea14152aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:32:20.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.786+0000 7fea12d9d640 1 --2- 172.21.15.106:0/3053976272 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea140c8d90 0x7fea1415e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:20.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.786+0000 7fea11d9b640 1 -- 172.21.15.106:0/3053976272 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fea140b4ce0 msgr2=0x7fea14152aa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:32:20.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.786+0000 7fea11d9b640 1 --2- 172.21.15.106:0/3053976272 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fea140b4ce0 0x7fea14152aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:20.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.786+0000 7fea11d9b640 1 -- 172.21.15.106:0/3053976272 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea140c8d90 msgr2=0x7fea1415e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:20.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.786+0000 7fea11d9b640 1 --2- 172.21.15.106:0/3053976272 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea140c8d90 0x7fea1415e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:20.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.786+0000 7fea11d9b640 1 -- 172.21.15.106:0/3053976272 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea1415f1d0 con 0x7fea140c7e30 2024-04-25T10:32:20.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.787+0000 7fea11d9b640 1 --2- 172.21.15.106:0/3053976272 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea140c7e30 0x7fea14151150 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fea0800e870 tx=0x7fea0800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:20.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.787+0000 7fea02ffd640 1 -- 172.21.15.106:0/3053976272 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea08017070 con 0x7fea140c7e30 2024-04-25T10:32:20.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.787+0000 7fea02ffd640 1 -- 172.21.15.106:0/3053976272 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea0800c940 con 0x7fea140c7e30 2024-04-25T10:32:20.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.787+0000 7fea02ffd640 1 -- 172.21.15.106:0/3053976272 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0800ccf0 con 0x7fea140c7e30 2024-04-25T10:32:20.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.787+0000 7fea18c96640 1 -- 172.21.15.106:0/3053976272 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea14151690 con 0x7fea140c7e30 2024-04-25T10:32:20.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.787+0000 7fea18c96640 1 -- 172.21.15.106:0/3053976272 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea141537a0 con 0x7fea140c7e30 2024-04-25T10:32:20.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.788+0000 7fea18c96640 1 --2- 172.21.15.106:0/3053976272 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea141539a0 0x7fea1415f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:20.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.788+0000 7fea18c96640 1 -- 172.21.15.106:0/3053976272 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fea14151b60 con 0x7fea141539a0 2024-04-25T10:32:20.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.788+0000 7fea1259c640 1 --2- 172.21.15.106:0/3053976272 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea141539a0 0x7fea1415f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:20.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.789+0000 7fea1259c640 1 -- 172.21.15.106:0/3053976272 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fea14153270 con 0x7fea141539a0 2024-04-25T10:32:20.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.789+0000 7fea1259c640 1 --2- 172.21.15.106:0/3053976272 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea141539a0 0x7fea1415f4a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fea04005880 tx=0x7fea0400cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:20.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.789+0000 7fea02ffd640 1 -- 172.21.15.106:0/3053976272 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fea0801a070 con 0x7fea140c7e30 2024-04-25T10:32:20.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.790+0000 7fea02ffd640 1 --2- 172.21.15.106:0/3053976272 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fe9f4074460 0x7fe9f4076970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:20.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.790+0000 7fea02ffd640 1 -- 172.21.15.106:0/3053976272 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fea08095570 con 0x7fea140c7e30 2024-04-25T10:32:20.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.790+0000 7fea12d9d640 1 --2- 172.21.15.106:0/3053976272 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fe9f4074460 0x7fe9f4076970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:20.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.790+0000 7fea02ffd640 1 -- 172.21.15.106:0/3053976272 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea04005ab0 con 0x7fea141539a0 2024-04-25T10:32:20.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.790+0000 7fea02ffd640 1 -- 172.21.15.106:0/3053976272 <== mon.0 v2:172.21.15.106:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fea0400ce50 con 0x7fea141539a0 2024-04-25T10:32:20.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.790+0000 7fea02ffd640 1 -- 172.21.15.106:0/3053976272 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea141539a0 msgr2=0x7fea1415f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:20.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.790+0000 7fea02ffd640 1 --2- 172.21.15.106:0/3053976272 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea141539a0 0x7fea1415f4a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fea04005880 tx=0x7fea0400cb10 comp rx=0 tx=0).stop 2024-04-25T10:32:20.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.790+0000 7fea02ffd640 1 -- 172.21.15.106:0/3053976272 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea141539a0 msgr2=0x7fea1415f4a0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T10:32:20.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.790+0000 7fea02ffd640 1 --2- 172.21.15.106:0/3053976272 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea141539a0 0x7fea1415f4a0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:20.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.791+0000 7fea12d9d640 1 --2- 172.21.15.106:0/3053976272 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fe9f4074460 0x7fe9f4076970 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fea0c034000 tx=0x7fea0c02f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:20.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.797+0000 7fea18c96640 1 --2- 172.21.15.106:0/3053976272 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea1415e160 0x7fea1415f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:20.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.797+0000 7fea18c96640 1 -- 172.21.15.106:0/3053976272 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fea141f77a0 con 0x7fea1415e160 2024-04-25T10:32:20.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.797+0000 7fea1259c640 1 --2- 172.21.15.106:0/3053976272 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea1415e160 0x7fea1415f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:20.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.797+0000 7fea1259c640 1 -- 172.21.15.106:0/3053976272 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fea1406a3d0 con 0x7fea1415e160 2024-04-25T10:32:20.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.798+0000 7fea1259c640 1 --2- 172.21.15.106:0/3053976272 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea1415e160 0x7fea1415f9e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fea0400a4a0 tx=0x7fea040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:20.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.798+0000 7fea02ffd640 1 -- 172.21.15.106:0/3053976272 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0401b970 con 0x7fea1415e160 2024-04-25T10:32:20.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.798+0000 7fea02ffd640 1 -- 172.21.15.106:0/3053976272 <== mon.0 v2:172.21.15.106:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5060 (secure 0 0 0) 0x7fea0401b970 con 0x7fea1415e160 2024-04-25T10:32:20.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.798+0000 7fea02ffd640 1 -- 172.21.15.106:0/3053976272 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea1415e160 msgr2=0x7fea1415f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:20.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.798+0000 7fea02ffd640 1 --2- 172.21.15.106:0/3053976272 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea1415e160 0x7fea1415f9e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fea0400a4a0 tx=0x7fea040057f0 comp rx=0 tx=0).stop 2024-04-25T10:32:20.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.798+0000 7fea02ffd640 1 -- 172.21.15.106:0/3053976272 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea1415e160 msgr2=0x7fea1415f9e0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T10:32:20.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.798+0000 7fea02ffd640 1 --2- 172.21.15.106:0/3053976272 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea1415e160 0x7fea1415f9e0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:20.801 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-25T10:32:20.801 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "a", 2024-04-25T10:32:20.801 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 0, 2024-04-25T10:32:20.802 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "leader", 2024-04-25T10:32:20.802 INFO:teuthology.orchestra.run.smithi106.stdout: "election_epoch": 14, 2024-04-25T10:32:20.802 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 133036, 2024-04-25T10:32:20.802 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum": [ 2024-04-25T10:32:20.802 INFO:teuthology.orchestra.run.smithi106.stdout: 0, 2024-04-25T10:32:20.802 INFO:teuthology.orchestra.run.smithi106.stdout: 1 2024-04-25T10:32:20.802 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:32:20.802 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_age": 0, 2024-04-25T10:32:20.802 INFO:teuthology.orchestra.run.smithi106.stdout: "features": { 2024-04-25T10:32:20.802 INFO:teuthology.orchestra.run.smithi106.stdout: "required_con": "2450521708007866372", 2024-04-25T10:32:20.802 INFO:teuthology.orchestra.run.smithi106.stdout: "required_mon": [ 2024-04-25T10:32:20.803 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:32:20.803 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:32:20.803 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:32:20.803 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:32:20.803 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:32:20.803 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:32:20.803 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:32:20.803 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:32:20.803 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:32:20.803 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:32:20.803 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:32:20.803 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:32:20.804 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_con": "4540701547738038271", 2024-04-25T10:32:20.804 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_mon": [ 2024-04-25T10:32:20.804 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:32:20.804 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:32:20.804 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:32:20.804 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:32:20.804 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:32:20.804 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:32:20.804 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:32:20.804 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:32:20.804 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:32:20.804 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:32:20.805 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:32:20.805 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:32:20.805 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:20.805 INFO:teuthology.orchestra.run.smithi106.stdout: "outside_quorum": [], 2024-04-25T10:32:20.805 INFO:teuthology.orchestra.run.smithi106.stdout: "extra_probe_peers": [], 2024-04-25T10:32:20.805 INFO:teuthology.orchestra.run.smithi106.stdout: "sync_provider": [], 2024-04-25T10:32:20.805 INFO:teuthology.orchestra.run.smithi106.stdout: "monmap": { 2024-04-25T10:32:20.805 INFO:teuthology.orchestra.run.smithi106.stdout: "epoch": 1, 2024-04-25T10:32:20.805 INFO:teuthology.orchestra.run.smithi106.stdout: "fsid": "3ee77985-e8e4-41cf-81e9-7c774cf7891f", 2024-04-25T10:32:20.805 INFO:teuthology.orchestra.run.smithi106.stdout: "modified": "2024-04-25T10:29:33.446228Z", 2024-04-25T10:32:20.805 INFO:teuthology.orchestra.run.smithi106.stdout: "created": "2024-04-25T10:29:33.446228Z", 2024-04-25T10:32:20.805 INFO:teuthology.orchestra.run.smithi106.stdout: "min_mon_release": 19, 2024-04-25T10:32:20.805 INFO:teuthology.orchestra.run.smithi106.stdout: "min_mon_release_name": "squid", 2024-04-25T10:32:20.806 INFO:teuthology.orchestra.run.smithi106.stdout: "election_strategy": 1, 2024-04-25T10:32:20.806 INFO:teuthology.orchestra.run.smithi106.stdout: "disallowed_leaders: ": "", 2024-04-25T10:32:20.806 INFO:teuthology.orchestra.run.smithi106.stdout: "stretch_mode": false, 2024-04-25T10:32:20.806 INFO:teuthology.orchestra.run.smithi106.stdout: "tiebreaker_mon": "", 2024-04-25T10:32:20.806 INFO:teuthology.orchestra.run.smithi106.stdout: "removed_ranks: ": "", 2024-04-25T10:32:20.806 INFO:teuthology.orchestra.run.smithi106.stdout: "features": { 2024-04-25T10:32:20.806 INFO:teuthology.orchestra.run.smithi106.stdout: "persistent": [ 2024-04-25T10:32:20.806 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:32:20.806 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:32:20.806 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:32:20.806 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:32:20.806 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:32:20.807 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:32:20.807 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:32:20.807 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:32:20.807 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:32:20.807 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:32:20.807 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:32:20.807 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:32:20.807 INFO:teuthology.orchestra.run.smithi106.stdout: "optional": [] 2024-04-25T10:32:20.807 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:20.807 INFO:teuthology.orchestra.run.smithi106.stdout: "mons": [ 2024-04-25T10:32:20.807 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:20.807 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 0, 2024-04-25T10:32:20.808 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "a", 2024-04-25T10:32:20.808 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:32:20.808 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:32:20.808 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:20.808 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:32:20.808 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:3300", 2024-04-25T10:32:20.808 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:32:20.808 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:20.808 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:20.808 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:32:20.809 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:6789", 2024-04-25T10:32:20.809 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:32:20.809 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:20.809 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:32:20.809 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:20.809 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:6789/0", 2024-04-25T10:32:20.809 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.106:6789/0", 2024-04-25T10:32:20.809 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:32:20.809 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:32:20.809 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:32:20.810 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:20.810 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:20.810 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 1, 2024-04-25T10:32:20.810 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "b", 2024-04-25T10:32:20.810 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:32:20.810 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:32:20.810 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:20.810 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:32:20.810 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:3300", 2024-04-25T10:32:20.810 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:32:20.811 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:20.811 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:20.811 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:32:20.811 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6789", 2024-04-25T10:32:20.811 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:32:20.811 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:20.811 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:32:20.811 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:20.811 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6789/0", 2024-04-25T10:32:20.811 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.134:6789/0", 2024-04-25T10:32:20.811 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:32:20.811 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:32:20.812 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:32:20.812 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:20.812 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:20.812 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 2, 2024-04-25T10:32:20.812 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "c", 2024-04-25T10:32:20.812 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:32:20.812 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:32:20.812 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:20.812 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:32:20.812 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:3301", 2024-04-25T10:32:20.812 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:32:20.812 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:20.813 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:20.813 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:32:20.813 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6790", 2024-04-25T10:32:20.813 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:32:20.813 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:20.813 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:32:20.813 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:20.813 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6790/0", 2024-04-25T10:32:20.813 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.134:6790/0", 2024-04-25T10:32:20.813 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:32:20.813 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:32:20.813 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:32:20.813 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:20.814 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:32:20.814 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:20.814 INFO:teuthology.orchestra.run.smithi106.stdout: "feature_map": { 2024-04-25T10:32:20.814 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": [ 2024-04-25T10:32:20.814 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:20.814 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:32:20.814 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:32:20.814 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 1 2024-04-25T10:32:20.814 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:20.814 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:32:20.814 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": [ 2024-04-25T10:32:20.814 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:20.815 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:32:20.815 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:32:20.815 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 6 2024-04-25T10:32:20.815 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:20.815 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:32:20.815 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": [ 2024-04-25T10:32:20.815 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:20.815 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:32:20.815 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:32:20.815 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 4 2024-04-25T10:32:20.815 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:20.815 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:32:20.815 INFO:teuthology.orchestra.run.smithi106.stdout: "client": [ 2024-04-25T10:32:20.816 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:20.816 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:32:20.816 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:32:20.816 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 7 2024-04-25T10:32:20.816 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:20.816 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:32:20.816 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": [ 2024-04-25T10:32:20.816 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:20.816 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:32:20.816 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:32:20.816 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 2 2024-04-25T10:32:20.816 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:20.816 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:32:20.817 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:20.817 INFO:teuthology.orchestra.run.smithi106.stdout: "stretch_mode": false 2024-04-25T10:32:20.817 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-25T10:32:20.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.799+0000 7fea18c96640 1 -- 172.21.15.106:0/3053976272 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fe9f4074460 msgr2=0x7fe9f4076970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:20.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.799+0000 7fea18c96640 1 --2- 172.21.15.106:0/3053976272 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fe9f4074460 0x7fe9f4076970 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fea0c034000 tx=0x7fea0c02f000 comp rx=0 tx=0).stop 2024-04-25T10:32:20.819 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.799+0000 7fea12d9d640 1 -- 172.21.15.106:0/3053976272 reap_dead start 2024-04-25T10:32:20.819 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.799+0000 7fea18c96640 1 -- 172.21.15.106:0/3053976272 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea140c7e30 msgr2=0x7fea14151150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:20.819 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.799+0000 7fea18c96640 1 --2- 172.21.15.106:0/3053976272 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea140c7e30 0x7fea14151150 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fea0800e870 tx=0x7fea0800ed30 comp rx=0 tx=0).stop 2024-04-25T10:32:20.819 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.799+0000 7fea18c96640 1 -- 172.21.15.106:0/3053976272 shutdown_connections 2024-04-25T10:32:20.819 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.799+0000 7fea18c96640 1 --2- 172.21.15.106:0/3053976272 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea140c7e30 0x7fea14151150 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:20.819 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.799+0000 7fea18c96640 1 -- 172.21.15.106:0/3053976272 >> 172.21.15.106:0/3053976272 conn(0x7fea14068b10 msgr2=0x7fea1406d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:20.819 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.799+0000 7fea18c96640 1 -- 172.21.15.106:0/3053976272 shutdown_connections 2024-04-25T10:32:20.820 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.799+0000 7fea18c96640 1 -- 172.21.15.106:0/3053976272 wait complete. 2024-04-25T10:32:20.820 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-04-25T10:32:20.820 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-04-25T10:32:20.820 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-25T10:32:20.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.930+0000 7fcd754af640 1 Processor -- start 2024-04-25T10:32:20.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.930+0000 7fcd754af640 1 -- start start 2024-04-25T10:32:20.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.932+0000 7fcd754af640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fcd70156f70 0x7fcd7006ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:20.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.932+0000 7fcd754af640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd7015d6d0 con 0x7fcd700c7ad0 2024-04-25T10:32:20.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.932+0000 7fcd754af640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd7015d9b0 con 0x7fcd700c8120 2024-04-25T10:32:20.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.932+0000 7fcd754af640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fcd700c7150 con 0x7fcd70156f70 2024-04-25T10:32:20.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.932+0000 7fcd6e7fc640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fcd70156f70 0x7fcd7006ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:20.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.932+0000 7fcd6e7fc640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fcd70156f70 0x7fcd7006ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:52874/0 (socket says 172.21.15.106:52874) 2024-04-25T10:32:20.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.932+0000 7fcd6e7fc640 1 -- 172.21.15.106:0/3882655375 learned_addr learned my addr 172.21.15.106:0/3882655375 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:32:20.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.932+0000 7fcd6e7fc640 1 -- 172.21.15.106:0/3882655375 >> v1:172.21.15.134:6789/0 conn(0x7fcd700c8120 legacy=0x7fcd700c8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:20.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.932+0000 7fcd6dffb640 1 -- 172.21.15.106:0/3882655375 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4286953333 0 0) 0x7fcd7015d9b0 con 0x7fcd700c8120 2024-04-25T10:32:20.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.932+0000 7fcd6e7fc640 1 -- 172.21.15.106:0/3882655375 >> v1:172.21.15.106:6789/0 conn(0x7fcd700c7ad0 legacy=0x7fcd7006d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:20.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.932+0000 7fcd6e7fc640 1 -- 172.21.15.106:0/3882655375 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd700c7430 con 0x7fcd70156f70 2024-04-25T10:32:20.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.933+0000 7fcd6e7fc640 1 --2- 172.21.15.106:0/3882655375 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd70156f70 0x7fcd7006ea60 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fcd64009870 tx=0x7fcd6402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1026badc7d0a197 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:20.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.933+0000 7fcd6dffb640 1 -- 172.21.15.106:0/3882655375 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd64037070 con 0x7fcd70156f70 2024-04-25T10:32:20.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.933+0000 7fcd6dffb640 1 -- 172.21.15.106:0/3882655375 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd6402ae50 con 0x7fcd70156f70 2024-04-25T10:32:20.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.933+0000 7fcd6dffb640 1 -- 172.21.15.106:0/3882655375 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd64032860 con 0x7fcd70156f70 2024-04-25T10:32:20.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.934+0000 7fcd754af640 1 -- 172.21.15.106:0/3882655375 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd70156f70 msgr2=0x7fcd7006ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:20.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.934+0000 7fcd754af640 1 --2- 172.21.15.106:0/3882655375 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd70156f70 0x7fcd7006ea60 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fcd64009870 tx=0x7fcd6402a300 comp rx=0 tx=0).stop 2024-04-25T10:32:20.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.934+0000 7fcd754af640 1 -- 172.21.15.106:0/3882655375 shutdown_connections 2024-04-25T10:32:20.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.934+0000 7fcd754af640 1 --2- 172.21.15.106:0/3882655375 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd70156f70 0x7fcd7006ea60 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:20.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.934+0000 7fcd754af640 1 -- 172.21.15.106:0/3882655375 >> 172.21.15.106:0/3882655375 conn(0x7fcd70068ed0 msgr2=0x7fcd700d20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:20.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.934+0000 7fcd754af640 1 -- 172.21.15.106:0/3882655375 shutdown_connections 2024-04-25T10:32:20.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.934+0000 7fcd754af640 1 -- 172.21.15.106:0/3882655375 wait complete. 2024-04-25T10:32:20.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.936+0000 7fcd754af640 1 Processor -- start 2024-04-25T10:32:20.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.936+0000 7fcd754af640 1 -- start start 2024-04-25T10:32:20.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.937+0000 7fcd754af640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd700c7ad0 0x7fcd7015f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:20.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.937+0000 7fcd754af640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd700c8120 0x7fcd7015bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:20.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.937+0000 7fcd754af640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcd70156f70 0x7fcd700ca630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:20.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.937+0000 7fcd754af640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fcd700ce6c0 con 0x7fcd700c8120 2024-04-25T10:32:20.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.937+0000 7fcd754af640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcd700ce360 con 0x7fcd700c7ad0 2024-04-25T10:32:20.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.937+0000 7fcd754af640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fcd7006dc70 con 0x7fcd70156f70 2024-04-25T10:32:20.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.937+0000 7fcd6f7fe640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcd70156f70 msgr2=0x7fcd700ca630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-04-25T10:32:20.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.937+0000 7fcd6f7fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcd70156f70 0x7fcd700ca630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:32:20.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.937+0000 7fcd6e7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd700c8120 0x7fcd7015bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:20.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.937+0000 7fcd6e7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd700c8120 0x7fcd7015bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:52880/0 (socket says 172.21.15.106:52880) 2024-04-25T10:32:20.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.937+0000 7fcd6e7fc640 1 -- 172.21.15.106:0/1925703225 learned_addr learned my addr 172.21.15.106:0/1925703225 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:32:20.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.937+0000 7fcd6effd640 1 --2- 172.21.15.106:0/1925703225 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd700c7ad0 0x7fcd7015f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:20.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.938+0000 7fcd6e7fc640 1 -- 172.21.15.106:0/1925703225 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcd70156f70 msgr2=0x7fcd700ca630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:32:20.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.938+0000 7fcd6e7fc640 1 --2- 172.21.15.106:0/1925703225 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcd70156f70 0x7fcd700ca630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:20.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.938+0000 7fcd6e7fc640 1 -- 172.21.15.106:0/1925703225 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd700c7ad0 msgr2=0x7fcd7015f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:20.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.938+0000 7fcd6e7fc640 1 --2- 172.21.15.106:0/1925703225 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd700c7ad0 0x7fcd7015f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:20.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.938+0000 7fcd6e7fc640 1 -- 172.21.15.106:0/1925703225 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd700cb410 con 0x7fcd700c8120 2024-04-25T10:32:20.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.938+0000 7fcd6e7fc640 1 --2- 172.21.15.106:0/1925703225 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd700c8120 0x7fcd7015bb60 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fcd64000c00 tx=0x7fcd640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:20.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.938+0000 7fcd577fe640 1 -- 172.21.15.106:0/1925703225 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd64041070 con 0x7fcd700c8120 2024-04-25T10:32:20.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.938+0000 7fcd577fe640 1 -- 172.21.15.106:0/1925703225 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd6403c870 con 0x7fcd700c8120 2024-04-25T10:32:20.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.938+0000 7fcd754af640 1 -- 172.21.15.106:0/1925703225 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd700cb6f0 con 0x7fcd700c8120 2024-04-25T10:32:20.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.938+0000 7fcd754af640 1 -- 172.21.15.106:0/1925703225 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd700cbcb0 con 0x7fcd700c8120 2024-04-25T10:32:20.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.939+0000 7fcd577fe640 1 -- 172.21.15.106:0/1925703225 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd64037040 con 0x7fcd700c8120 2024-04-25T10:32:20.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.940+0000 7fcd754af640 1 --2- 172.21.15.106:0/1925703225 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd50002a70 0x7fcd50004f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:20.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.940+0000 7fcd754af640 1 -- 172.21.15.106:0/1925703225 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcd50007f10 con 0x7fcd50002a70 2024-04-25T10:32:20.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.940+0000 7fcd6effd640 1 --2- 172.21.15.106:0/1925703225 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd50002a70 0x7fcd50004f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:20.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.940+0000 7fcd6effd640 1 -- 172.21.15.106:0/1925703225 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcd50008110 con 0x7fcd50002a70 2024-04-25T10:32:20.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.940+0000 7fcd6effd640 1 --2- 172.21.15.106:0/1925703225 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd50002a70 0x7fcd50004f80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcd6000f470 tx=0x7fcd6000f930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:20.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.941+0000 7fcd577fe640 1 -- 172.21.15.106:0/1925703225 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd60019070 con 0x7fcd50002a70 2024-04-25T10:32:20.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.941+0000 7fcd577fe640 1 -- 172.21.15.106:0/1925703225 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fcd64044070 con 0x7fcd700c8120 2024-04-25T10:32:20.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.941+0000 7fcd577fe640 1 --2- 172.21.15.106:0/1925703225 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fcd5c0743e0 0x7fcd5c0768f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:20.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.941+0000 7fcd577fe640 1 -- 172.21.15.106:0/1925703225 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fcd640b5860 con 0x7fcd700c8120 2024-04-25T10:32:20.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.941+0000 7fcd6f7fe640 1 --2- 172.21.15.106:0/1925703225 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fcd5c0743e0 0x7fcd5c0768f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:20.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.942+0000 7fcd577fe640 1 -- 172.21.15.106:0/1925703225 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fcd60005320 con 0x7fcd50002a70 2024-04-25T10:32:20.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.942+0000 7fcd577fe640 1 -- 172.21.15.106:0/1925703225 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd50002a70 msgr2=0x7fcd50004f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:20.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.942+0000 7fcd577fe640 1 --2- 172.21.15.106:0/1925703225 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd50002a70 0x7fcd50004f80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcd6000f470 tx=0x7fcd6000f930 comp rx=0 tx=0).stop 2024-04-25T10:32:20.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.942+0000 7fcd577fe640 1 -- 172.21.15.106:0/1925703225 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd50002a70 msgr2=0x7fcd50004f80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T10:32:20.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.942+0000 7fcd577fe640 1 --2- 172.21.15.106:0/1925703225 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd50002a70 0x7fcd50004f80 secure :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcd6000f470 tx=0x7fcd6000f930 comp rx=0 tx=0).stop 2024-04-25T10:32:20.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.942+0000 7fcd6f7fe640 1 --2- 172.21.15.106:0/1925703225 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fcd5c0743e0 0x7fcd5c0768f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fcd700c9170 tx=0x7fcd68007d10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:20.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.948+0000 7fcd754af640 1 --2- 172.21.15.106:0/1925703225 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd50006fa0 0x7fcd50007480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:20.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.948+0000 7fcd754af640 1 -- 172.21.15.106:0/1925703225 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcd5000a4f0 con 0x7fcd50006fa0 2024-04-25T10:32:20.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.948+0000 7fcd6effd640 1 --2- 172.21.15.106:0/1925703225 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd50006fa0 0x7fcd50007480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:20.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.949+0000 7fcd6effd640 1 -- 172.21.15.106:0/1925703225 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fcd500079c0 con 0x7fcd50006fa0 2024-04-25T10:32:20.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.949+0000 7fcd6effd640 1 --2- 172.21.15.106:0/1925703225 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd50006fa0 0x7fcd50007480 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fcd6000f470 tx=0x7fcd60000ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:20.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.949+0000 7fcd577fe640 1 -- 172.21.15.106:0/1925703225 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6001c2a0 con 0x7fcd50006fa0 2024-04-25T10:32:20.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.950+0000 7fcd577fe640 1 -- 172.21.15.106:0/1925703225 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4718 (secure 0 0 0) 0x7fcd6001c2a0 con 0x7fcd50006fa0 2024-04-25T10:32:20.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.950+0000 7fcd577fe640 1 -- 172.21.15.106:0/1925703225 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd50006fa0 msgr2=0x7fcd50007480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:20.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.950+0000 7fcd577fe640 1 --2- 172.21.15.106:0/1925703225 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd50006fa0 0x7fcd50007480 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fcd6000f470 tx=0x7fcd60000ed0 comp rx=0 tx=0).stop 2024-04-25T10:32:20.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.950+0000 7fcd577fe640 1 -- 172.21.15.106:0/1925703225 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd50006fa0 msgr2=0x7fcd50007480 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T10:32:20.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.950+0000 7fcd577fe640 1 --2- 172.21.15.106:0/1925703225 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd50006fa0 0x7fcd50007480 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:20.953 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-25T10:32:20.953 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "b", 2024-04-25T10:32:20.953 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 1, 2024-04-25T10:32:20.953 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "peon", 2024-04-25T10:32:20.954 INFO:teuthology.orchestra.run.smithi106.stdout: "election_epoch": 14, 2024-04-25T10:32:20.954 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 133145, 2024-04-25T10:32:20.954 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum": [ 2024-04-25T10:32:20.954 INFO:teuthology.orchestra.run.smithi106.stdout: 0, 2024-04-25T10:32:20.954 INFO:teuthology.orchestra.run.smithi106.stdout: 1 2024-04-25T10:32:20.954 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:32:20.954 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_age": 0, 2024-04-25T10:32:20.954 INFO:teuthology.orchestra.run.smithi106.stdout: "features": { 2024-04-25T10:32:20.954 INFO:teuthology.orchestra.run.smithi106.stdout: "required_con": "2450521708007866372", 2024-04-25T10:32:20.954 INFO:teuthology.orchestra.run.smithi106.stdout: "required_mon": [ 2024-04-25T10:32:20.954 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:32:20.954 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:32:20.955 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:32:20.955 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:32:20.955 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:32:20.955 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:32:20.955 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:32:20.955 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:32:20.955 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:32:20.956 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:32:20.956 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:32:20.956 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:32:20.956 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_con": "4540701547738038271", 2024-04-25T10:32:20.956 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_mon": [ 2024-04-25T10:32:20.956 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:32:20.956 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:32:20.956 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:32:20.956 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:32:20.956 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:32:20.956 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:32:20.956 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:32:20.957 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:32:20.957 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:32:20.957 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:32:20.957 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:32:20.957 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:32:20.957 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:20.957 INFO:teuthology.orchestra.run.smithi106.stdout: "outside_quorum": [], 2024-04-25T10:32:20.957 INFO:teuthology.orchestra.run.smithi106.stdout: "extra_probe_peers": [], 2024-04-25T10:32:20.957 INFO:teuthology.orchestra.run.smithi106.stdout: "sync_provider": [], 2024-04-25T10:32:20.957 INFO:teuthology.orchestra.run.smithi106.stdout: "monmap": { 2024-04-25T10:32:20.957 INFO:teuthology.orchestra.run.smithi106.stdout: "epoch": 1, 2024-04-25T10:32:20.958 INFO:teuthology.orchestra.run.smithi106.stdout: "fsid": "3ee77985-e8e4-41cf-81e9-7c774cf7891f", 2024-04-25T10:32:20.958 INFO:teuthology.orchestra.run.smithi106.stdout: "modified": "2024-04-25T10:29:33.446228Z", 2024-04-25T10:32:20.958 INFO:teuthology.orchestra.run.smithi106.stdout: "created": "2024-04-25T10:29:33.446228Z", 2024-04-25T10:32:20.958 INFO:teuthology.orchestra.run.smithi106.stdout: "min_mon_release": 19, 2024-04-25T10:32:20.958 INFO:teuthology.orchestra.run.smithi106.stdout: "min_mon_release_name": "squid", 2024-04-25T10:32:20.958 INFO:teuthology.orchestra.run.smithi106.stdout: "election_strategy": 1, 2024-04-25T10:32:20.958 INFO:teuthology.orchestra.run.smithi106.stdout: "disallowed_leaders: ": "", 2024-04-25T10:32:20.958 INFO:teuthology.orchestra.run.smithi106.stdout: "stretch_mode": false, 2024-04-25T10:32:20.958 INFO:teuthology.orchestra.run.smithi106.stdout: "tiebreaker_mon": "", 2024-04-25T10:32:20.958 INFO:teuthology.orchestra.run.smithi106.stdout: "removed_ranks: ": "", 2024-04-25T10:32:20.958 INFO:teuthology.orchestra.run.smithi106.stdout: "features": { 2024-04-25T10:32:20.958 INFO:teuthology.orchestra.run.smithi106.stdout: "persistent": [ 2024-04-25T10:32:20.959 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:32:20.959 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:32:20.959 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:32:20.959 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:32:20.959 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:32:20.959 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:32:20.959 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:32:20.959 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:32:20.959 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:32:20.959 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:32:20.959 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:32:20.959 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:32:20.959 INFO:teuthology.orchestra.run.smithi106.stdout: "optional": [] 2024-04-25T10:32:20.960 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:20.960 INFO:teuthology.orchestra.run.smithi106.stdout: "mons": [ 2024-04-25T10:32:20.960 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:20.960 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 0, 2024-04-25T10:32:20.960 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "a", 2024-04-25T10:32:20.960 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:32:20.960 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:32:20.960 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:20.960 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:32:20.960 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:3300", 2024-04-25T10:32:20.960 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:32:20.960 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:20.960 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:20.961 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:32:20.961 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:6789", 2024-04-25T10:32:20.961 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:32:20.961 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:20.961 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:32:20.961 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:20.961 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:6789/0", 2024-04-25T10:32:20.961 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.106:6789/0", 2024-04-25T10:32:20.962 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:32:20.962 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:32:20.962 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:32:20.962 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:20.962 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:20.962 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 1, 2024-04-25T10:32:20.962 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "b", 2024-04-25T10:32:20.962 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:32:20.962 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:32:20.962 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:20.962 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:32:20.963 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:3300", 2024-04-25T10:32:20.963 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:32:20.963 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:20.963 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:20.963 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:32:20.963 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6789", 2024-04-25T10:32:20.963 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:32:20.963 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:20.963 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:32:20.963 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:20.963 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6789/0", 2024-04-25T10:32:20.964 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.134:6789/0", 2024-04-25T10:32:20.964 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:32:20.964 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:32:20.964 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:32:20.964 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:20.964 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:20.964 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 2, 2024-04-25T10:32:20.964 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "c", 2024-04-25T10:32:20.964 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:32:20.964 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:32:20.964 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:20.964 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:32:20.965 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:3301", 2024-04-25T10:32:20.965 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:32:20.965 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:20.965 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:20.965 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:32:20.965 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6790", 2024-04-25T10:32:20.965 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:32:20.965 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:20.965 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:32:20.966 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:20.966 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6790/0", 2024-04-25T10:32:20.966 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.134:6790/0", 2024-04-25T10:32:20.966 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:32:20.966 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:32:20.966 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:32:20.966 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:20.967 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:32:20.967 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:20.967 INFO:teuthology.orchestra.run.smithi106.stdout: "feature_map": { 2024-04-25T10:32:20.967 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": [ 2024-04-25T10:32:20.967 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:20.967 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:32:20.967 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:32:20.967 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 1 2024-04-25T10:32:20.967 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:20.967 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:32:20.967 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": [ 2024-04-25T10:32:20.967 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:20.967 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:32:20.968 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:32:20.968 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 4 2024-04-25T10:32:20.968 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:20.968 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:32:20.968 INFO:teuthology.orchestra.run.smithi106.stdout: "client": [ 2024-04-25T10:32:20.968 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:20.968 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:32:20.968 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:32:20.968 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 1 2024-04-25T10:32:20.968 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:20.969 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:32:20.969 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:20.969 INFO:teuthology.orchestra.run.smithi106.stdout: "stretch_mode": false 2024-04-25T10:32:20.969 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-25T10:32:20.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.950+0000 7fcd754af640 1 -- 172.21.15.106:0/1925703225 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fcd5c0743e0 msgr2=0x7fcd5c0768f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:20.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.950+0000 7fcd754af640 1 --2- 172.21.15.106:0/1925703225 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fcd5c0743e0 0x7fcd5c0768f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fcd700c9170 tx=0x7fcd68007d10 comp rx=0 tx=0).stop 2024-04-25T10:32:20.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.950+0000 7fcd754af640 1 -- 172.21.15.106:0/1925703225 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd700c8120 msgr2=0x7fcd7015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:20.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.950+0000 7fcd754af640 1 --2- 172.21.15.106:0/1925703225 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd700c8120 0x7fcd7015bb60 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fcd64000c00 tx=0x7fcd640046c0 comp rx=0 tx=0).stop 2024-04-25T10:32:20.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.950+0000 7fcd6f7fe640 1 -- 172.21.15.106:0/1925703225 reap_dead start 2024-04-25T10:32:20.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.950+0000 7fcd6f7fe640 1 -- 172.21.15.106:0/1925703225 reap_dead start 2024-04-25T10:32:20.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.950+0000 7fcd754af640 1 -- 172.21.15.106:0/1925703225 shutdown_connections 2024-04-25T10:32:20.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.950+0000 7fcd754af640 1 -- 172.21.15.106:0/1925703225 >> 172.21.15.106:0/1925703225 conn(0x7fcd70068ed0 msgr2=0x7fcd700c8530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:20.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.950+0000 7fcd754af640 1 -- 172.21.15.106:0/1925703225 shutdown_connections 2024-04-25T10:32:20.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:20.950+0000 7fcd754af640 1 -- 172.21.15.106:0/1925703225 wait complete. 2024-04-25T10:32:20.971 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-04-25T10:32:20.972 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-04-25T10:32:32.209 DEBUG:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:32:32.211 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:32:40.973 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-04-25T10:32:40.973 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-04-25T10:32:40.973 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-25T10:32:40.974 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-25T10:32:40.976 INFO:tasks.ceph.mon.c:Started 2024-04-25T10:32:40.976 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-04-25T10:32:40.977 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T10:32:41.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.086+0000 7f972bbb2640 1 Processor -- start 2024-04-25T10:32:41.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.086+0000 7f972bbb2640 1 -- start start 2024-04-25T10:32:41.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.087+0000 7f972bbb2640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f97240c90c0 0x7f972415d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:41.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.087+0000 7f972bbb2640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f972415d9e0 0x7f97240caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:41.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.087+0000 7f972bbb2640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9724074140 con 0x7f9724068800 2024-04-25T10:32:41.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.087+0000 7f972bbb2640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f97240733d0 con 0x7f97240c90c0 2024-04-25T10:32:41.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.087+0000 7f972bbb2640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f972415d010 con 0x7f972415d9e0 2024-04-25T10:32:41.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.088+0000 7f972a14f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f972415d9e0 0x7f97240caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:41.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.088+0000 7f972914d640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f97240c90c0 0x7f972415d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:41.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.088+0000 7f972a14f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f972415d9e0 0x7f97240caff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.106:51916/0 (socket says 172.21.15.106:51916) 2024-04-25T10:32:41.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.088+0000 7f972a14f640 1 -- 172.21.15.106:0/3987751699 learned_addr learned my addr 172.21.15.106:0/3987751699 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:32:41.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.088+0000 7f972994e640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f9724068800 0x7f9724156f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.106:42776/0 (socket says 172.21.15.106:42776) 2024-04-25T10:32:41.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.088+0000 7f972a14f640 1 -- 172.21.15.106:0/3987751699 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f972415d9e0 msgr2=0x7f97240caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T10:32:41.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.088+0000 7f972a14f640 1 -- 172.21.15.106:0/3987751699 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f972415d9e0 msgr2=0x7f97240caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:32:41.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.088+0000 7f972a14f640 1 --2- 172.21.15.106:0/3987751699 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f972415d9e0 0x7f97240caff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:32:41.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.088+0000 7f972a14f640 1 --2- 172.21.15.106:0/3987751699 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f972415d9e0 0x7f97240caff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:32:41.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.088+0000 7f972914d640 1 -- 172.21.15.106:0/3987751699 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f972415d9e0 msgr2=0x7f97240caff0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:32:41.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.088+0000 7f972914d640 1 --2- 172.21.15.106:0/3987751699 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f972415d9e0 0x7f97240caff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:41.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.088+0000 7f972914d640 1 -- 172.21.15.106:0/3987751699 >> v1:172.21.15.134:6789/0 conn(0x7f9724068800 legacy=0x7f9724156f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:41.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.088+0000 7f972914d640 1 -- 172.21.15.106:0/3987751699 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9724074400 con 0x7f97240c90c0 2024-04-25T10:32:41.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.089+0000 7f972914d640 1 --2- 172.21.15.106:0/3987751699 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97240c90c0 0x7f972415d4a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f97140097d0 tx=0x7f971402a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=680fe9e1b949b2ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:41.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.089+0000 7f972894c640 1 -- 172.21.15.106:0/3987751699 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9714037070 con 0x7f97240c90c0 2024-04-25T10:32:41.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.089+0000 7f972894c640 1 -- 172.21.15.106:0/3987751699 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f971402ae40 con 0x7f97240c90c0 2024-04-25T10:32:41.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.089+0000 7f972894c640 1 -- 172.21.15.106:0/3987751699 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9714032660 con 0x7f97240c90c0 2024-04-25T10:32:41.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.090+0000 7f972bbb2640 1 -- 172.21.15.106:0/3987751699 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97240c90c0 msgr2=0x7f972415d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:41.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.090+0000 7f972bbb2640 1 --2- 172.21.15.106:0/3987751699 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97240c90c0 0x7f972415d4a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f97140097d0 tx=0x7f971402a130 comp rx=0 tx=0).stop 2024-04-25T10:32:41.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.090+0000 7f972bbb2640 1 -- 172.21.15.106:0/3987751699 shutdown_connections 2024-04-25T10:32:41.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.090+0000 7f972bbb2640 1 --2- 172.21.15.106:0/3987751699 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f972415d9e0 0x7f97240caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:41.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.090+0000 7f972bbb2640 1 --2- 172.21.15.106:0/3987751699 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97240c90c0 0x7f972415d4a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:41.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.090+0000 7f972bbb2640 1 -- 172.21.15.106:0/3987751699 >> 172.21.15.106:0/3987751699 conn(0x7f9724070b30 msgr2=0x7f9724070f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:41.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.090+0000 7f972bbb2640 1 -- 172.21.15.106:0/3987751699 shutdown_connections 2024-04-25T10:32:41.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.090+0000 7f972bbb2640 1 -- 172.21.15.106:0/3987751699 wait complete. 2024-04-25T10:32:41.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.092+0000 7f972bbb2640 1 Processor -- start 2024-04-25T10:32:41.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.092+0000 7f972bbb2640 1 -- start start 2024-04-25T10:32:41.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.093+0000 7f972bbb2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9724068800 0x7f97240ce4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:41.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.093+0000 7f972bbb2640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f97240c90c0 0x7f9724160ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:41.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.093+0000 7f972bbb2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f972415d9e0 0x7f972415ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:41.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.093+0000 7f972bbb2640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f97240c94d0 con 0x7f972415d9e0 2024-04-25T10:32:41.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.093+0000 7f972bbb2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f97240749c0 con 0x7f9724068800 2024-04-25T10:32:41.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.093+0000 7f972bbb2640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f97240c9650 con 0x7f97240c90c0 2024-04-25T10:32:41.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.093+0000 7f972a14f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f972415d9e0 0x7f972415ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:41.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.093+0000 7f972994e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9724068800 0x7f97240ce4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:41.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.093+0000 7f972a14f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f972415d9e0 0x7f972415ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53514/0 (socket says 172.21.15.106:53514) 2024-04-25T10:32:41.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.093+0000 7f972a14f640 1 -- 172.21.15.106:0/647411696 learned_addr learned my addr 172.21.15.106:0/647411696 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:32:41.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.093+0000 7f972914d640 1 --2- 172.21.15.106:0/647411696 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f97240c90c0 0x7f9724160ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:41.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.093+0000 7f972914d640 1 -- 172.21.15.106:0/647411696 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f97240c90c0 msgr2=0x7f9724160ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-04-25T10:32:41.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.093+0000 7f972914d640 1 -- 172.21.15.106:0/647411696 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f97240c90c0 msgr2=0x7f9724160ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T10:32:41.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.093+0000 7f972914d640 1 --2- 172.21.15.106:0/647411696 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f97240c90c0 0x7f9724160ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:32:41.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.093+0000 7f972914d640 1 --2- 172.21.15.106:0/647411696 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f97240c90c0 0x7f9724160ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:32:41.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.094+0000 7f972994e640 1 -- 172.21.15.106:0/647411696 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f97240c90c0 msgr2=0x7f9724160ac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:32:41.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.094+0000 7f972994e640 1 --2- 172.21.15.106:0/647411696 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f97240c90c0 0x7f9724160ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:41.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.094+0000 7f972994e640 1 -- 172.21.15.106:0/647411696 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f972415d9e0 msgr2=0x7f972415ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:41.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.094+0000 7f972994e640 1 --2- 172.21.15.106:0/647411696 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f972415d9e0 0x7f972415ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:41.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.094+0000 7f972994e640 1 -- 172.21.15.106:0/647411696 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97241f3690 con 0x7f9724068800 2024-04-25T10:32:41.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.094+0000 7f972a14f640 1 --2- 172.21.15.106:0/647411696 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f972415d9e0 0x7f972415ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:32:41.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.094+0000 7f972994e640 1 --2- 172.21.15.106:0/647411696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9724068800 0x7f97240ce4e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f972000c3c0 tx=0x7f972000c7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:41.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.094+0000 7f971a7fc640 1 -- 172.21.15.106:0/647411696 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97200074e0 con 0x7f9724068800 2024-04-25T10:32:41.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.094+0000 7f971a7fc640 1 -- 172.21.15.106:0/647411696 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9720007680 con 0x7f9724068800 2024-04-25T10:32:41.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.094+0000 7f972bbb2640 1 -- 172.21.15.106:0/647411696 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97240cea20 con 0x7f9724068800 2024-04-25T10:32:41.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.094+0000 7f971a7fc640 1 -- 172.21.15.106:0/647411696 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9720004a20 con 0x7f9724068800 2024-04-25T10:32:41.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.094+0000 7f972bbb2640 1 -- 172.21.15.106:0/647411696 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97240cdce0 con 0x7f9724068800 2024-04-25T10:32:41.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.095+0000 7f972bbb2640 1 -- 172.21.15.106:0/647411696 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f972406a3d0 con 0x7f9724068800 2024-04-25T10:32:41.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.096+0000 7f971a7fc640 1 -- 172.21.15.106:0/647411696 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f972001a020 con 0x7f9724068800 2024-04-25T10:32:41.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.096+0000 7f971a7fc640 1 --2- 172.21.15.106:0/647411696 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f97080744b0 0x7f97080769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:41.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.096+0000 7f971a7fc640 1 -- 172.21.15.106:0/647411696 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f9720094940 con 0x7f9724068800 2024-04-25T10:32:41.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.096+0000 7f972914d640 1 --2- 172.21.15.106:0/647411696 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f97080744b0 0x7f97080769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:41.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.097+0000 7f972914d640 1 --2- 172.21.15.106:0/647411696 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f97080744b0 0x7f97080769c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9714034000 tx=0x7f971402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:41.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.101+0000 7f971a7fc640 1 -- 172.21.15.106:0/647411696 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f97241f3690 con 0x7f9724068800 2024-04-25T10:32:41.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.353+0000 7f972bbb2640 1 -- 172.21.15.106:0/647411696 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f9724073590 con 0x7f9724068800 2024-04-25T10:32:41.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.353+0000 7f971a7fc640 1 -- 172.21.15.106:0/647411696 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1563 (secure 0 0 0) 0x7f972005f0c0 con 0x7f9724068800 2024-04-25T10:32:41.356 INFO:teuthology.orchestra.run.smithi106.stdout:{"election_epoch":14,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":20,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"3ee77985-e8e4-41cf-81e9-7c774cf7891f","modified":"2024-04-25T10:29:33.446228Z","created":"2024-04-25T10:29:33.446228Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:3300","nonce":0},{"type":"v1","addr":"172.21.15.106:6789","nonce":0}]},"addr":"172.21.15.106:6789/0","public_addr":"172.21.15.106:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T10:32:41.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.355+0000 7f972bbb2640 1 -- 172.21.15.106:0/647411696 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f97080744b0 msgr2=0x7f97080769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:41.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.355+0000 7f972bbb2640 1 --2- 172.21.15.106:0/647411696 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f97080744b0 0x7f97080769c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9714034000 tx=0x7f971402e000 comp rx=0 tx=0).stop 2024-04-25T10:32:41.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.355+0000 7f972bbb2640 1 -- 172.21.15.106:0/647411696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9724068800 msgr2=0x7f97240ce4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:41.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.355+0000 7f972bbb2640 1 --2- 172.21.15.106:0/647411696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9724068800 0x7f97240ce4e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f972000c3c0 tx=0x7f972000c7c0 comp rx=0 tx=0).stop 2024-04-25T10:32:41.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.355+0000 7f972bbb2640 1 -- 172.21.15.106:0/647411696 shutdown_connections 2024-04-25T10:32:41.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.355+0000 7f972bbb2640 1 --2- 172.21.15.106:0/647411696 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f97080744b0 0x7f97080769c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:41.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.355+0000 7f972bbb2640 1 --2- 172.21.15.106:0/647411696 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f972415d9e0 0x7f972415ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:41.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.355+0000 7f972bbb2640 1 --2- 172.21.15.106:0/647411696 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f97240c90c0 0x7f9724160ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:41.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.355+0000 7f972bbb2640 1 --2- 172.21.15.106:0/647411696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9724068800 0x7f97240ce4e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:41.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.355+0000 7f972bbb2640 1 -- 172.21.15.106:0/647411696 >> 172.21.15.106:0/647411696 conn(0x7f9724070b30 msgr2=0x7f97240b2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:41.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.355+0000 7f972bbb2640 1 -- 172.21.15.106:0/647411696 shutdown_connections 2024-04-25T10:32:41.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:41.355+0000 7f972bbb2640 1 -- 172.21.15.106:0/647411696 wait complete. 2024-04-25T10:32:43.135 INFO:tasks.workunit.client.0.smithi106.stderr:Note: switching to '994e2f20102a20fa9a27966c14b625cc01884df8'. 2024-04-25T10:32:43.135 INFO:tasks.workunit.client.0.smithi106.stderr: 2024-04-25T10:32:43.135 INFO:tasks.workunit.client.0.smithi106.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-25T10:32:43.136 INFO:tasks.workunit.client.0.smithi106.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-25T10:32:43.136 INFO:tasks.workunit.client.0.smithi106.stderr:state without impacting any branches by switching back to a branch. 2024-04-25T10:32:43.136 INFO:tasks.workunit.client.0.smithi106.stderr: 2024-04-25T10:32:43.136 INFO:tasks.workunit.client.0.smithi106.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-25T10:32:43.136 INFO:tasks.workunit.client.0.smithi106.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-25T10:32:43.136 INFO:tasks.workunit.client.0.smithi106.stderr: 2024-04-25T10:32:43.136 INFO:tasks.workunit.client.0.smithi106.stderr: git switch -c 2024-04-25T10:32:43.136 INFO:tasks.workunit.client.0.smithi106.stderr: 2024-04-25T10:32:43.136 INFO:tasks.workunit.client.0.smithi106.stderr:Or undo this operation with: 2024-04-25T10:32:43.136 INFO:tasks.workunit.client.0.smithi106.stderr: 2024-04-25T10:32:43.136 INFO:tasks.workunit.client.0.smithi106.stderr: git switch - 2024-04-25T10:32:43.136 INFO:tasks.workunit.client.0.smithi106.stderr: 2024-04-25T10:32:43.136 INFO:tasks.workunit.client.0.smithi106.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-25T10:32:43.136 INFO:tasks.workunit.client.0.smithi106.stderr: 2024-04-25T10:32:43.136 INFO:tasks.workunit.client.0.smithi106.stderr:HEAD is now at 994e2f20102 Merge PR #56810 into wip-pdonnell-testing-20240425.015853-debug 2024-04-25T10:32:43.141 DEBUG:teuthology.orchestra.run.smithi106:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-25T10:32:43.371 INFO:tasks.workunit.client.0.smithi106.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-25T10:32:43.373 INFO:tasks.workunit.client.0.smithi106.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-25T10:32:43.373 INFO:tasks.workunit.client.0.smithi106.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-25T10:32:43.876 INFO:tasks.workunit.client.0.smithi106.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-25T10:32:43.932 INFO:tasks.workunit.client.0.smithi106.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-25T10:32:43.973 INFO:tasks.workunit.client.0.smithi106.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-25T10:32:43.975 INFO:tasks.workunit.client.0.smithi106.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-25T10:32:43.975 INFO:tasks.workunit.client.0.smithi106.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-25T10:32:44.014 INFO:tasks.workunit.client.0.smithi106.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-25T10:32:44.017 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T10:32:44.017 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-25T10:32:44.032 INFO:tasks.workunit:Running workunits matching suites/iozone.sh on client.0... 2024-04-25T10:32:44.033 INFO:tasks.workunit:Running workunit suites/iozone.sh... 2024-04-25T10:32:44.034 DEBUG:teuthology.orchestra.run.smithi106:workunit test suites/iozone.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=994e2f20102a20fa9a27966c14b625cc01884df8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iozone.sh 2024-04-25T10:32:44.099 INFO:tasks.workunit.client.0.smithi106.stderr:+ iozone -c -e -s 1024M -r 16K -t 1 -F f1 -i 0 -i 1 2024-04-25T10:32:44.172 INFO:tasks.workunit.client.0.smithi106.stdout: Iozone: Performance Test of File I/O 2024-04-25T10:32:44.173 INFO:tasks.workunit.client.0.smithi106.stdout: Version $Revision: 3.494 $ 2024-04-25T10:32:44.173 INFO:tasks.workunit.client.0.smithi106.stdout: Compiled for 64 bit mode. 2024-04-25T10:32:44.173 INFO:tasks.workunit.client.0.smithi106.stdout: Build: linux-AMD64 2024-04-25T10:32:44.173 INFO:tasks.workunit.client.0.smithi106.stdout: 2024-04-25T10:32:44.173 INFO:tasks.workunit.client.0.smithi106.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-04-25T10:32:44.173 INFO:tasks.workunit.client.0.smithi106.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-04-25T10:32:44.173 INFO:tasks.workunit.client.0.smithi106.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-04-25T10:32:44.173 INFO:tasks.workunit.client.0.smithi106.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-04-25T10:32:44.173 INFO:tasks.workunit.client.0.smithi106.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-04-25T10:32:44.173 INFO:tasks.workunit.client.0.smithi106.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-04-25T10:32:44.174 INFO:tasks.workunit.client.0.smithi106.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-04-25T10:32:44.174 INFO:tasks.workunit.client.0.smithi106.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-04-25T10:32:44.174 INFO:tasks.workunit.client.0.smithi106.stdout: Alexey Skidanov, Sudhir Kumar. 2024-04-25T10:32:44.174 INFO:tasks.workunit.client.0.smithi106.stdout: 2024-04-25T10:32:44.174 INFO:tasks.workunit.client.0.smithi106.stdout: Run began: Thu Apr 25 10:32:44 2024 2024-04-25T10:32:44.174 INFO:tasks.workunit.client.0.smithi106.stdout: 2024-04-25T10:32:44.174 INFO:tasks.workunit.client.0.smithi106.stdout: Include close in write timing 2024-04-25T10:32:44.174 INFO:tasks.workunit.client.0.smithi106.stdout: Include fsync in write timing 2024-04-25T10:32:44.174 INFO:tasks.workunit.client.0.smithi106.stdout: File size set to 1048576 kB 2024-04-25T10:32:44.174 INFO:tasks.workunit.client.0.smithi106.stdout: Record Size 16 kB 2024-04-25T10:32:44.174 INFO:tasks.workunit.client.0.smithi106.stdout: Command line used: iozone -c -e -s 1024M -r 16K -t 1 -F f1 -i 0 -i 1 2024-04-25T10:32:44.174 INFO:tasks.workunit.client.0.smithi106.stdout: Output is in kBytes/sec 2024-04-25T10:32:44.174 INFO:tasks.workunit.client.0.smithi106.stdout: Time Resolution = 0.000001 seconds. 2024-04-25T10:32:44.175 INFO:tasks.workunit.client.0.smithi106.stdout: Processor cache size set to 1024 kBytes. 2024-04-25T10:32:44.175 INFO:tasks.workunit.client.0.smithi106.stdout: Processor cache line size set to 32 bytes. 2024-04-25T10:32:44.175 INFO:tasks.workunit.client.0.smithi106.stdout: File stride size set to 17 * record size. 2024-04-25T10:32:44.175 INFO:tasks.workunit.client.0.smithi106.stdout: Throughput test with 1 process 2024-04-25T10:32:44.175 INFO:tasks.workunit.client.0.smithi106.stdout: Each process writes a 1048576 kByte file in 16 kByte records 2024-04-25T10:32:44.369 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T10:32:44.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.485+0000 7f6b7af6c640 1 Processor -- start 2024-04-25T10:32:44.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.485+0000 7f6b7af6c640 1 -- start start 2024-04-25T10:32:44.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.487+0000 7f6b7af6c640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f6b740c53f0 0x7f6b741531e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:44.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.487+0000 7f6b7af6c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b7415c650 0x7f6b7415ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:44.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.487+0000 7f6b7af6c640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b74073c20 con 0x7f6b74073670 2024-04-25T10:32:44.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.487+0000 7f6b7af6c640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f6b74159d10 con 0x7f6b740c53f0 2024-04-25T10:32:44.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.487+0000 7f6b7af6c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6b740c5800 con 0x7f6b7415c650 2024-04-25T10:32:44.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.487+0000 7f6b73fff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f6b740c53f0 0x7f6b741531e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:44.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.487+0000 7f6b79509640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b7415c650 0x7f6b7415ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:44.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.487+0000 7f6b73fff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f6b740c53f0 0x7f6b741531e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.106:60948/0 (socket says 172.21.15.106:60948) 2024-04-25T10:32:44.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.487+0000 7f6b73fff640 1 -- 172.21.15.106:0/3162927310 learned_addr learned my addr 172.21.15.106:0/3162927310 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:32:44.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.487+0000 7f6b78d08640 1 --1- 172.21.15.106:0/3162927310 >> v1:172.21.15.134:6789/0 conn(0x7f6b74073670 0x7f6b740c8fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.106:59598/0 (socket says 172.21.15.106:59598) 2024-04-25T10:32:44.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.487+0000 7f6b79509640 1 -- 172.21.15.106:0/3162927310 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b7415c650 msgr2=0x7f6b7415ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T10:32:44.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.487+0000 7f6b79509640 1 -- 172.21.15.106:0/3162927310 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b7415c650 msgr2=0x7f6b7415ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:32:44.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.487+0000 7f6b79509640 1 --2- 172.21.15.106:0/3162927310 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b7415c650 0x7f6b7415ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:32:44.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.487+0000 7f6b79509640 1 --2- 172.21.15.106:0/3162927310 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b7415c650 0x7f6b7415ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:32:44.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.487+0000 7f6b73fff640 1 -- 172.21.15.106:0/3162927310 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b7415c650 msgr2=0x7f6b7415ca60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:32:44.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.487+0000 7f6b73fff640 1 --2- 172.21.15.106:0/3162927310 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b7415c650 0x7f6b7415ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:44.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.487+0000 7f6b73fff640 1 -- 172.21.15.106:0/3162927310 >> v1:172.21.15.134:6789/0 conn(0x7f6b74073670 legacy=0x7f6b740c8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:44.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.487+0000 7f6b73fff640 1 -- 172.21.15.106:0/3162927310 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b74159100 con 0x7f6b740c53f0 2024-04-25T10:32:44.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.488+0000 7f6b73fff640 1 --2- 172.21.15.106:0/3162927310 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b740c53f0 0x7f6b741531e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6b60009870 tx=0x7f6b6002a300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6fd9031c6185f69 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:44.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.488+0000 7f6b737fe640 1 -- 172.21.15.106:0/3162927310 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b60037070 con 0x7f6b740c53f0 2024-04-25T10:32:44.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.488+0000 7f6b737fe640 1 -- 172.21.15.106:0/3162927310 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b6002ae50 con 0x7f6b740c53f0 2024-04-25T10:32:44.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.488+0000 7f6b737fe640 1 -- 172.21.15.106:0/3162927310 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b60032860 con 0x7f6b740c53f0 2024-04-25T10:32:44.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.489+0000 7f6b7af6c640 1 -- 172.21.15.106:0/3162927310 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b740c53f0 msgr2=0x7f6b741531e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:44.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.489+0000 7f6b7af6c640 1 --2- 172.21.15.106:0/3162927310 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b740c53f0 0x7f6b741531e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6b60009870 tx=0x7f6b6002a300 comp rx=0 tx=0).stop 2024-04-25T10:32:44.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.489+0000 7f6b7af6c640 1 -- 172.21.15.106:0/3162927310 shutdown_connections 2024-04-25T10:32:44.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.489+0000 7f6b7af6c640 1 --2- 172.21.15.106:0/3162927310 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b7415c650 0x7f6b7415ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:44.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.489+0000 7f6b7af6c640 1 --2- 172.21.15.106:0/3162927310 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b740c53f0 0x7f6b741531e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:44.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.489+0000 7f6b7af6c640 1 -- 172.21.15.106:0/3162927310 >> 172.21.15.106:0/3162927310 conn(0x7f6b7406c5c0 msgr2=0x7f6b7406c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:44.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.489+0000 7f6b7af6c640 1 -- 172.21.15.106:0/3162927310 shutdown_connections 2024-04-25T10:32:44.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.489+0000 7f6b7af6c640 1 -- 172.21.15.106:0/3162927310 wait complete. 2024-04-25T10:32:44.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.491+0000 7f6b7af6c640 1 Processor -- start 2024-04-25T10:32:44.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.491+0000 7f6b7af6c640 1 -- start start 2024-04-25T10:32:44.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.492+0000 7f6b7af6c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b74073670 0x7f6b7414fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:44.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.492+0000 7f6b7af6c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b740c53f0 0x7f6b7415b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:44.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.492+0000 7f6b7af6c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b7415c650 0x7f6b74144fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:44.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.492+0000 7f6b7af6c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6b741599a0 con 0x7f6b740c53f0 2024-04-25T10:32:44.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.492+0000 7f6b7af6c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6b741596c0 con 0x7f6b7415c650 2024-04-25T10:32:44.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.492+0000 7f6b7af6c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6b7415cfa0 con 0x7f6b74073670 2024-04-25T10:32:44.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.492+0000 7f6b73fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b740c53f0 0x7f6b7415b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:44.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.492+0000 7f6b78d08640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b74073670 0x7f6b7414fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:44.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.492+0000 7f6b73fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b740c53f0 0x7f6b7415b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60864/0 (socket says 172.21.15.106:60864) 2024-04-25T10:32:44.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.492+0000 7f6b73fff640 1 -- 172.21.15.106:0/1804661936 learned_addr learned my addr 172.21.15.106:0/1804661936 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:32:44.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.492+0000 7f6b79509640 1 --2- 172.21.15.106:0/1804661936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b7415c650 0x7f6b74144fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:44.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.492+0000 7f6b78d08640 1 -- 172.21.15.106:0/1804661936 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b74073670 msgr2=0x7f6b7414fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-04-25T10:32:44.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.492+0000 7f6b78d08640 1 -- 172.21.15.106:0/1804661936 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b74073670 msgr2=0x7f6b7414fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T10:32:44.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.492+0000 7f6b78d08640 1 --2- 172.21.15.106:0/1804661936 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b74073670 0x7f6b7414fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:32:44.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.492+0000 7f6b78d08640 1 --2- 172.21.15.106:0/1804661936 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b74073670 0x7f6b7414fc60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:32:44.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.493+0000 7f6b73fff640 1 -- 172.21.15.106:0/1804661936 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b74073670 msgr2=0x7f6b7414fc60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:32:44.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.493+0000 7f6b73fff640 1 --2- 172.21.15.106:0/1804661936 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b74073670 0x7f6b7414fc60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:44.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.493+0000 7f6b73fff640 1 -- 172.21.15.106:0/1804661936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b7415c650 msgr2=0x7f6b74144fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:44.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.493+0000 7f6b73fff640 1 --2- 172.21.15.106:0/1804661936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b7415c650 0x7f6b74144fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:44.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.493+0000 7f6b73fff640 1 -- 172.21.15.106:0/1804661936 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b741501a0 con 0x7f6b740c53f0 2024-04-25T10:32:44.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.493+0000 7f6b79509640 1 --2- 172.21.15.106:0/1804661936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b7415c650 0x7f6b74144fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:32:44.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.493+0000 7f6b73fff640 1 --2- 172.21.15.106:0/1804661936 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b740c53f0 0x7f6b7415b520 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6b60000c00 tx=0x7f6b600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:44.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.493+0000 7f6b717fa640 1 -- 172.21.15.106:0/1804661936 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b60041070 con 0x7f6b740c53f0 2024-04-25T10:32:44.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.493+0000 7f6b717fa640 1 -- 172.21.15.106:0/1804661936 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b6003c670 con 0x7f6b740c53f0 2024-04-25T10:32:44.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.493+0000 7f6b7af6c640 1 -- 172.21.15.106:0/1804661936 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b740cd600 con 0x7f6b740c53f0 2024-04-25T10:32:44.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.493+0000 7f6b717fa640 1 -- 172.21.15.106:0/1804661936 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b60037040 con 0x7f6b740c53f0 2024-04-25T10:32:44.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.493+0000 7f6b7af6c640 1 -- 172.21.15.106:0/1804661936 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b740cdb80 con 0x7f6b740c53f0 2024-04-25T10:32:44.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.494+0000 7f6b7af6c640 1 -- 172.21.15.106:0/1804661936 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b7406a3d0 con 0x7f6b740c53f0 2024-04-25T10:32:44.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.495+0000 7f6b717fa640 1 -- 172.21.15.106:0/1804661936 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6b60044070 con 0x7f6b740c53f0 2024-04-25T10:32:44.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.495+0000 7f6b717fa640 1 --2- 172.21.15.106:0/1804661936 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f6b580744b0 0x7f6b580769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:44.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.496+0000 7f6b717fa640 1 -- 172.21.15.106:0/1804661936 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f6b600b5ac0 con 0x7f6b740c53f0 2024-04-25T10:32:44.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.496+0000 7f6b78d08640 1 --2- 172.21.15.106:0/1804661936 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f6b580744b0 0x7f6b580769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:44.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.496+0000 7f6b78d08640 1 --2- 172.21.15.106:0/1804661936 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f6b580744b0 0x7f6b580769c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6b68000f60 tx=0x7f6b68007c20 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:44.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.502+0000 7f6b717fa640 1 -- 172.21.15.106:0/1804661936 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6b6007fb60 con 0x7f6b740c53f0 2024-04-25T10:32:44.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.804+0000 7f6b7af6c640 1 -- 172.21.15.106:0/1804661936 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f6b74156980 con 0x7f6b740c53f0 2024-04-25T10:32:44.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.805+0000 7f6b717fa640 1 -- 172.21.15.106:0/1804661936 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1568 (secure 0 0 0) 0x7f6b60083e60 con 0x7f6b740c53f0 2024-04-25T10:32:44.808 INFO:teuthology.orchestra.run.smithi106.stdout:{"election_epoch":16,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"3ee77985-e8e4-41cf-81e9-7c774cf7891f","modified":"2024-04-25T10:29:33.446228Z","created":"2024-04-25T10:29:33.446228Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:3300","nonce":0},{"type":"v1","addr":"172.21.15.106:6789","nonce":0}]},"addr":"172.21.15.106:6789/0","public_addr":"172.21.15.106:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T10:32:44.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.807+0000 7f6b7af6c640 1 -- 172.21.15.106:0/1804661936 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f6b580744b0 msgr2=0x7f6b580769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:44.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.807+0000 7f6b7af6c640 1 --2- 172.21.15.106:0/1804661936 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f6b580744b0 0x7f6b580769c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6b68000f60 tx=0x7f6b68007c20 comp rx=0 tx=0).stop 2024-04-25T10:32:44.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.807+0000 7f6b7af6c640 1 -- 172.21.15.106:0/1804661936 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b740c53f0 msgr2=0x7f6b7415b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:44.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.807+0000 7f6b7af6c640 1 --2- 172.21.15.106:0/1804661936 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b740c53f0 0x7f6b7415b520 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6b60000c00 tx=0x7f6b600046c0 comp rx=0 tx=0).stop 2024-04-25T10:32:44.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.807+0000 7f6b7af6c640 1 -- 172.21.15.106:0/1804661936 shutdown_connections 2024-04-25T10:32:44.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.807+0000 7f6b7af6c640 1 --2- 172.21.15.106:0/1804661936 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f6b580744b0 0x7f6b580769c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:44.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.807+0000 7f6b7af6c640 1 --2- 172.21.15.106:0/1804661936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b7415c650 0x7f6b74144fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:44.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.807+0000 7f6b7af6c640 1 --2- 172.21.15.106:0/1804661936 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b740c53f0 0x7f6b7415b520 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:44.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.807+0000 7f6b7af6c640 1 --2- 172.21.15.106:0/1804661936 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b74073670 0x7f6b7414fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:44.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.807+0000 7f6b7af6c640 1 -- 172.21.15.106:0/1804661936 >> 172.21.15.106:0/1804661936 conn(0x7f6b7406c5c0 msgr2=0x7f6b74157100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:44.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.807+0000 7f6b7af6c640 1 -- 172.21.15.106:0/1804661936 shutdown_connections 2024-04-25T10:32:44.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.807+0000 7f6b7af6c640 1 -- 172.21.15.106:0/1804661936 wait complete. 2024-04-25T10:32:44.821 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-04-25T10:32:44.821 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-04-25T10:32:44.821 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-25T10:32:44.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.944+0000 7f6655f14640 1 Processor -- start 2024-04-25T10:32:44.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.944+0000 7f6655f14640 1 -- start start 2024-04-25T10:32:44.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.945+0000 7f6655f14640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f66500711c0 0x7f665015e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:44.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.945+0000 7f6655f14640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6650154a00 0x7f66500cbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:44.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.945+0000 7f6655f14640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66500742a0 con 0x7f665015b780 2024-04-25T10:32:44.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.945+0000 7f6655f14640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f665015b2f0 con 0x7f6650154a00 2024-04-25T10:32:44.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.945+0000 7f6655f14640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f6650074820 con 0x7f66500711c0 2024-04-25T10:32:44.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.945+0000 7f664f7fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f665015b780 0x7f66500739a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:56722/0 (socket says 172.21.15.106:56722) 2024-04-25T10:32:44.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.945+0000 7f664ffff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6650154a00 0x7f66500cbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:44.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.946+0000 7f664f7fe640 1 -- 172.21.15.106:0/3628808397 learned_addr learned my addr 172.21.15.106:0/3628808397 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:32:44.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.946+0000 7f664effd640 1 --2- 172.21.15.106:0/3628808397 >> v2:172.21.15.134:3300/0 conn(0x7f66500711c0 0x7f665015e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:44.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.946+0000 7f664ffff640 1 -- 172.21.15.106:0/3628808397 >> v2:172.21.15.134:3300/0 conn(0x7f66500711c0 msgr2=0x7f665015e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:44.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.946+0000 7f664ffff640 1 --2- 172.21.15.106:0/3628808397 >> v2:172.21.15.134:3300/0 conn(0x7f66500711c0 0x7f665015e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:44.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.946+0000 7f664ffff640 1 -- 172.21.15.106:0/3628808397 >> v1:172.21.15.106:6789/0 conn(0x7f665015b780 legacy=0x7f66500739a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:44.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.946+0000 7f664ffff640 1 -- 172.21.15.106:0/3628808397 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6650074540 con 0x7f6650154a00 2024-04-25T10:32:44.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.946+0000 7f664e7fc640 1 -- 172.21.15.106:0/3628808397 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3605543204 0 0) 0x7f66500742a0 con 0x7f665015b780 2024-04-25T10:32:44.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.947+0000 7f664ffff640 1 --2- 172.21.15.106:0/3628808397 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6650154a00 0x7f66500cbdb0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f665015b470 tx=0x7f664802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82a992a90b13316c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:44.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.947+0000 7f664e7fc640 1 -- 172.21.15.106:0/3628808397 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6648007740 con 0x7f6650154a00 2024-04-25T10:32:44.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.947+0000 7f664e7fc640 1 -- 172.21.15.106:0/3628808397 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66480078e0 con 0x7f6650154a00 2024-04-25T10:32:44.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.947+0000 7f664e7fc640 1 -- 172.21.15.106:0/3628808397 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6648007cb0 con 0x7f6650154a00 2024-04-25T10:32:44.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.948+0000 7f6655f14640 1 -- 172.21.15.106:0/3628808397 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6650154a00 msgr2=0x7f66500cbdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:44.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.948+0000 7f6655f14640 1 --2- 172.21.15.106:0/3628808397 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6650154a00 0x7f66500cbdb0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f665015b470 tx=0x7f664802a660 comp rx=0 tx=0).stop 2024-04-25T10:32:44.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.948+0000 7f6655f14640 1 -- 172.21.15.106:0/3628808397 shutdown_connections 2024-04-25T10:32:44.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.948+0000 7f6655f14640 1 --2- 172.21.15.106:0/3628808397 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6650154a00 0x7f66500cbdb0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:44.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.948+0000 7f6655f14640 1 --2- 172.21.15.106:0/3628808397 >> v2:172.21.15.134:3300/0 conn(0x7f66500711c0 0x7f665015e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:44.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.948+0000 7f6655f14640 1 -- 172.21.15.106:0/3628808397 >> 172.21.15.106:0/3628808397 conn(0x7f665006daf0 msgr2=0x7f665006df00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:44.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.948+0000 7f6655f14640 1 -- 172.21.15.106:0/3628808397 shutdown_connections 2024-04-25T10:32:44.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.948+0000 7f6655f14640 1 -- 172.21.15.106:0/3628808397 wait complete. 2024-04-25T10:32:44.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.950+0000 7f6655f14640 1 Processor -- start 2024-04-25T10:32:44.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.950+0000 7f6655f14640 1 -- start start 2024-04-25T10:32:44.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.951+0000 7f6655f14640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66500711c0 0x7f66501593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:44.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.951+0000 7f6655f14640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6650154a00 0x7f66500c7b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:44.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.951+0000 7f6655f14640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f665015b780 0x7f66501e4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:44.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.951+0000 7f6655f14640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f66500722b0 con 0x7f6650154a00 2024-04-25T10:32:44.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.951+0000 7f6655f14640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f665015af80 con 0x7f66500711c0 2024-04-25T10:32:44.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.951+0000 7f6655f14640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6650154e10 con 0x7f665015b780 2024-04-25T10:32:44.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.951+0000 7f664effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6650154a00 0x7f66500c7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:44.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.951+0000 7f664effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6650154a00 0x7f66500c7b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60878/0 (socket says 172.21.15.106:60878) 2024-04-25T10:32:44.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.951+0000 7f664effd640 1 -- 172.21.15.106:0/1572840823 learned_addr learned my addr 172.21.15.106:0/1572840823 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:32:44.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.952+0000 7f664f7fe640 1 --2- 172.21.15.106:0/1572840823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66500711c0 0x7f66501593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:44.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.952+0000 7f664ffff640 1 --2- 172.21.15.106:0/1572840823 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f665015b780 0x7f66501e4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:44.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.952+0000 7f664effd640 1 -- 172.21.15.106:0/1572840823 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f665015b780 msgr2=0x7f66501e4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:44.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.952+0000 7f664ffff640 1 -- 172.21.15.106:0/1572840823 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f665015b780 msgr2=0x7f66501e4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-04-25T10:32:44.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.952+0000 7f664ffff640 1 -- 172.21.15.106:0/1572840823 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f665015b780 msgr2=0x7f66501e4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T10:32:44.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.952+0000 7f664ffff640 1 --2- 172.21.15.106:0/1572840823 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f665015b780 0x7f66501e4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:32:44.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.952+0000 7f664ffff640 1 --2- 172.21.15.106:0/1572840823 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f665015b780 0x7f66501e4a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:32:44.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.952+0000 7f664effd640 1 --2- 172.21.15.106:0/1572840823 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f665015b780 0x7f66501e4a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:44.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.952+0000 7f664effd640 1 -- 172.21.15.106:0/1572840823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66500711c0 msgr2=0x7f66501593b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:44.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.952+0000 7f664effd640 1 --2- 172.21.15.106:0/1572840823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66500711c0 0x7f66501593b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:44.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.952+0000 7f664effd640 1 -- 172.21.15.106:0/1572840823 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66501f6760 con 0x7f6650154a00 2024-04-25T10:32:44.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.952+0000 7f664f7fe640 1 --2- 172.21.15.106:0/1572840823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66500711c0 0x7f66501593b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:32:44.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.952+0000 7f664effd640 1 --2- 172.21.15.106:0/1572840823 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6650154a00 0x7f66500c7b50 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f664400e840 tx=0x7f664400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:44.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.952+0000 7f6637fff640 1 -- 172.21.15.106:0/1572840823 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6644017070 con 0x7f6650154a00 2024-04-25T10:32:44.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.952+0000 7f6637fff640 1 -- 172.21.15.106:0/1572840823 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f664400c940 con 0x7f6650154a00 2024-04-25T10:32:44.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.952+0000 7f6655f14640 1 -- 172.21.15.106:0/1572840823 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66501f6a40 con 0x7f6650154a00 2024-04-25T10:32:44.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.952+0000 7f6637fff640 1 -- 172.21.15.106:0/1572840823 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f664400cd10 con 0x7f6650154a00 2024-04-25T10:32:44.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.952+0000 7f6655f14640 1 -- 172.21.15.106:0/1572840823 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66501599f0 con 0x7f6650154a00 2024-04-25T10:32:44.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.953+0000 7f6655f14640 1 --2- 172.21.15.106:0/1572840823 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6650153890 0x7f66501e4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:44.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.953+0000 7f6655f14640 1 -- 172.21.15.106:0/1572840823 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6650159cb0 con 0x7f6650153890 2024-04-25T10:32:44.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.954+0000 7f664f7fe640 1 --2- 172.21.15.106:0/1572840823 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6650153890 0x7f66501e4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:44.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.954+0000 7f664f7fe640 1 -- 172.21.15.106:0/1572840823 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f66501f6510 con 0x7f6650153890 2024-04-25T10:32:44.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.954+0000 7f664f7fe640 1 --2- 172.21.15.106:0/1572840823 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6650153890 0x7f66501e4fd0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f6640001380 tx=0x7f6640001840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:44.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.954+0000 7f6637fff640 1 -- 172.21.15.106:0/1572840823 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6640012590 con 0x7f6650153890 2024-04-25T10:32:44.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.954+0000 7f6637fff640 1 -- 172.21.15.106:0/1572840823 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f664401a070 con 0x7f6650154a00 2024-04-25T10:32:44.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.955+0000 7f6637fff640 1 --2- 172.21.15.106:0/1572840823 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f663c0743e0 0x7f663c0768f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:44.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.955+0000 7f664ffff640 1 --2- 172.21.15.106:0/1572840823 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f663c0743e0 0x7f663c0768f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:44.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.955+0000 7f6637fff640 1 -- 172.21.15.106:0/1572840823 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f6644095560 con 0x7f6650154a00 2024-04-25T10:32:44.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.955+0000 7f6637fff640 1 -- 172.21.15.106:0/1572840823 <== mon.0 v2:172.21.15.106:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f6640018070 con 0x7f6650153890 2024-04-25T10:32:44.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.955+0000 7f6637fff640 1 -- 172.21.15.106:0/1572840823 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6650153890 msgr2=0x7f66501e4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:44.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.955+0000 7f6637fff640 1 --2- 172.21.15.106:0/1572840823 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6650153890 0x7f66501e4fd0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f6640001380 tx=0x7f6640001840 comp rx=0 tx=0).stop 2024-04-25T10:32:44.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.955+0000 7f6637fff640 1 -- 172.21.15.106:0/1572840823 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6650153890 msgr2=0x7f66501e4fd0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T10:32:44.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.955+0000 7f6637fff640 1 --2- 172.21.15.106:0/1572840823 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6650153890 0x7f66501e4fd0 secure :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f6640001380 tx=0x7f6640001840 comp rx=0 tx=0).stop 2024-04-25T10:32:44.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.955+0000 7f664ffff640 1 --2- 172.21.15.106:0/1572840823 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f663c0743e0 0x7f663c0768f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6648034000 tx=0x7f6648033040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:44.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.962+0000 7f6655f14640 1 --2- 172.21.15.106:0/1572840823 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6650153d20 0x7f66501f2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:44.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.962+0000 7f6655f14640 1 -- 172.21.15.106:0/1572840823 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f665015a990 con 0x7f6650153d20 2024-04-25T10:32:44.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.962+0000 7f664f7fe640 1 --2- 172.21.15.106:0/1572840823 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6650153d20 0x7f66501f2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:44.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.962+0000 7f664f7fe640 1 -- 172.21.15.106:0/1572840823 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f66501542b0 con 0x7f6650153d20 2024-04-25T10:32:44.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.962+0000 7f664f7fe640 1 --2- 172.21.15.106:0/1572840823 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6650153d20 0x7f66501f2fa0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f6640001380 tx=0x7f6640001080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:44.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.962+0000 7f6637fff640 1 -- 172.21.15.106:0/1572840823 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f664001d9c0 con 0x7f6650153d20 2024-04-25T10:32:44.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.963+0000 7f6637fff640 1 -- 172.21.15.106:0/1572840823 <== mon.0 v2:172.21.15.106:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5071 (secure 0 0 0) 0x7f6640027430 con 0x7f6650153d20 2024-04-25T10:32:44.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.963+0000 7f6637fff640 1 -- 172.21.15.106:0/1572840823 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6650153d20 msgr2=0x7f66501f2fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:44.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.963+0000 7f6637fff640 1 --2- 172.21.15.106:0/1572840823 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6650153d20 0x7f66501f2fa0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f6640001380 tx=0x7f6640001080 comp rx=0 tx=0).stop 2024-04-25T10:32:44.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.963+0000 7f6637fff640 1 -- 172.21.15.106:0/1572840823 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6650153d20 msgr2=0x7f66501f2fa0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T10:32:44.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.963+0000 7f6637fff640 1 --2- 172.21.15.106:0/1572840823 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6650153d20 0x7f66501f2fa0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:44.966 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-25T10:32:44.966 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "a", 2024-04-25T10:32:44.966 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 0, 2024-04-25T10:32:44.966 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "leader", 2024-04-25T10:32:44.966 INFO:teuthology.orchestra.run.smithi106.stdout: "election_epoch": 16, 2024-04-25T10:32:44.966 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 157202, 2024-04-25T10:32:44.966 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum": [ 2024-04-25T10:32:44.967 INFO:teuthology.orchestra.run.smithi106.stdout: 0, 2024-04-25T10:32:44.967 INFO:teuthology.orchestra.run.smithi106.stdout: 1, 2024-04-25T10:32:44.967 INFO:teuthology.orchestra.run.smithi106.stdout: 2 2024-04-25T10:32:44.967 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:32:44.967 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_age": 3, 2024-04-25T10:32:44.967 INFO:teuthology.orchestra.run.smithi106.stdout: "features": { 2024-04-25T10:32:44.967 INFO:teuthology.orchestra.run.smithi106.stdout: "required_con": "2450521708007866372", 2024-04-25T10:32:44.967 INFO:teuthology.orchestra.run.smithi106.stdout: "required_mon": [ 2024-04-25T10:32:44.967 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:32:44.967 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:32:44.967 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:32:44.967 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:32:44.967 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:32:44.967 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:32:44.967 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:32:44.968 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:32:44.968 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:32:44.968 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:32:44.968 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:32:44.968 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:32:44.968 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_con": "4540701547738038271", 2024-04-25T10:32:44.968 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_mon": [ 2024-04-25T10:32:44.968 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:32:44.968 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:32:44.968 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:32:44.968 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:32:44.968 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:32:44.968 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:32:44.968 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:32:44.969 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:32:44.969 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:32:44.969 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:32:44.969 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:32:44.969 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:32:44.969 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:44.969 INFO:teuthology.orchestra.run.smithi106.stdout: "outside_quorum": [], 2024-04-25T10:32:44.969 INFO:teuthology.orchestra.run.smithi106.stdout: "extra_probe_peers": [], 2024-04-25T10:32:44.969 INFO:teuthology.orchestra.run.smithi106.stdout: "sync_provider": [], 2024-04-25T10:32:44.969 INFO:teuthology.orchestra.run.smithi106.stdout: "monmap": { 2024-04-25T10:32:44.969 INFO:teuthology.orchestra.run.smithi106.stdout: "epoch": 1, 2024-04-25T10:32:44.970 INFO:teuthology.orchestra.run.smithi106.stdout: "fsid": "3ee77985-e8e4-41cf-81e9-7c774cf7891f", 2024-04-25T10:32:44.970 INFO:teuthology.orchestra.run.smithi106.stdout: "modified": "2024-04-25T10:29:33.446228Z", 2024-04-25T10:32:44.970 INFO:teuthology.orchestra.run.smithi106.stdout: "created": "2024-04-25T10:29:33.446228Z", 2024-04-25T10:32:44.970 INFO:teuthology.orchestra.run.smithi106.stdout: "min_mon_release": 19, 2024-04-25T10:32:44.970 INFO:teuthology.orchestra.run.smithi106.stdout: "min_mon_release_name": "squid", 2024-04-25T10:32:44.970 INFO:teuthology.orchestra.run.smithi106.stdout: "election_strategy": 1, 2024-04-25T10:32:44.970 INFO:teuthology.orchestra.run.smithi106.stdout: "disallowed_leaders: ": "", 2024-04-25T10:32:44.970 INFO:teuthology.orchestra.run.smithi106.stdout: "stretch_mode": false, 2024-04-25T10:32:44.970 INFO:teuthology.orchestra.run.smithi106.stdout: "tiebreaker_mon": "", 2024-04-25T10:32:44.970 INFO:teuthology.orchestra.run.smithi106.stdout: "removed_ranks: ": "", 2024-04-25T10:32:44.970 INFO:teuthology.orchestra.run.smithi106.stdout: "features": { 2024-04-25T10:32:44.970 INFO:teuthology.orchestra.run.smithi106.stdout: "persistent": [ 2024-04-25T10:32:44.970 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:32:44.970 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:32:44.971 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:32:44.971 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:32:44.971 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:32:44.971 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:32:44.971 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:32:44.971 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:32:44.971 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:32:44.971 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:32:44.971 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:32:44.971 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:32:44.971 INFO:teuthology.orchestra.run.smithi106.stdout: "optional": [] 2024-04-25T10:32:44.971 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:44.971 INFO:teuthology.orchestra.run.smithi106.stdout: "mons": [ 2024-04-25T10:32:44.971 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:44.971 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 0, 2024-04-25T10:32:44.972 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "a", 2024-04-25T10:32:44.972 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:32:44.972 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:32:44.972 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:44.972 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:32:44.972 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:3300", 2024-04-25T10:32:44.972 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:32:44.972 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:44.972 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:44.972 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:32:44.972 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:6789", 2024-04-25T10:32:44.972 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:32:44.972 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:44.972 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:32:44.972 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:44.973 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:6789/0", 2024-04-25T10:32:44.973 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.106:6789/0", 2024-04-25T10:32:44.973 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:32:44.973 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:32:44.973 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:32:44.973 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:44.973 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:44.973 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 1, 2024-04-25T10:32:44.973 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "b", 2024-04-25T10:32:44.973 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:32:44.973 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:32:44.974 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:44.974 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:32:44.974 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:3300", 2024-04-25T10:32:44.974 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:32:44.974 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:44.974 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:44.974 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:32:44.974 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6789", 2024-04-25T10:32:44.974 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:32:44.975 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:44.975 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:32:44.975 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:44.975 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6789/0", 2024-04-25T10:32:44.975 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.134:6789/0", 2024-04-25T10:32:44.975 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:32:44.975 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:32:44.975 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:32:44.975 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:44.975 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:44.975 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 2, 2024-04-25T10:32:44.975 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "c", 2024-04-25T10:32:44.975 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:32:44.975 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:32:44.976 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:44.976 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:32:44.976 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:3301", 2024-04-25T10:32:44.976 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:32:44.976 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:44.976 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:44.976 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:32:44.976 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6790", 2024-04-25T10:32:44.976 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:32:44.976 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:44.976 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:32:44.976 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:44.977 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6790/0", 2024-04-25T10:32:44.977 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.134:6790/0", 2024-04-25T10:32:44.977 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:32:44.977 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:32:44.977 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:32:44.977 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:44.977 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:32:44.977 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:44.977 INFO:teuthology.orchestra.run.smithi106.stdout: "feature_map": { 2024-04-25T10:32:44.977 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": [ 2024-04-25T10:32:44.977 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:44.977 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:32:44.977 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:32:44.978 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 1 2024-04-25T10:32:44.978 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:44.978 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:32:44.978 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": [ 2024-04-25T10:32:44.978 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:44.978 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:32:44.978 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:32:44.978 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 6 2024-04-25T10:32:44.978 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:44.978 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:32:44.978 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": [ 2024-04-25T10:32:44.978 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:44.978 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:32:44.978 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:32:44.979 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 4 2024-04-25T10:32:44.979 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:44.979 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:32:44.979 INFO:teuthology.orchestra.run.smithi106.stdout: "client": [ 2024-04-25T10:32:44.979 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:44.979 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:32:44.979 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:32:44.979 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 7 2024-04-25T10:32:44.979 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:44.979 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:32:44.979 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": [ 2024-04-25T10:32:44.980 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:44.980 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:32:44.980 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:32:44.980 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 2 2024-04-25T10:32:44.980 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:44.980 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:32:44.980 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:44.980 INFO:teuthology.orchestra.run.smithi106.stdout: "stretch_mode": false 2024-04-25T10:32:44.980 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-25T10:32:44.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.963+0000 7f6655f14640 1 -- 172.21.15.106:0/1572840823 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f663c0743e0 msgr2=0x7f663c0768f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:44.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.963+0000 7f6655f14640 1 --2- 172.21.15.106:0/1572840823 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f663c0743e0 0x7f663c0768f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6648034000 tx=0x7f6648033040 comp rx=0 tx=0).stop 2024-04-25T10:32:44.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.963+0000 7f6655f14640 1 -- 172.21.15.106:0/1572840823 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6650154a00 msgr2=0x7f66500c7b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:44.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.963+0000 7f6655f14640 1 --2- 172.21.15.106:0/1572840823 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6650154a00 0x7f66500c7b50 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f664400e840 tx=0x7f664400ed00 comp rx=0 tx=0).stop 2024-04-25T10:32:44.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.963+0000 7f664ffff640 1 -- 172.21.15.106:0/1572840823 reap_dead start 2024-04-25T10:32:44.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.963+0000 7f664ffff640 1 -- 172.21.15.106:0/1572840823 reap_dead start 2024-04-25T10:32:44.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.963+0000 7f6655f14640 1 -- 172.21.15.106:0/1572840823 shutdown_connections 2024-04-25T10:32:44.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.963+0000 7f6655f14640 1 -- 172.21.15.106:0/1572840823 >> 172.21.15.106:0/1572840823 conn(0x7f665006daf0 msgr2=0x7f66500c4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:44.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.963+0000 7f6655f14640 1 -- 172.21.15.106:0/1572840823 shutdown_connections 2024-04-25T10:32:44.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:44.963+0000 7f6655f14640 1 -- 172.21.15.106:0/1572840823 wait complete. 2024-04-25T10:32:44.982 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T10:32:44.982 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-04-25T10:32:44.983 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-25T10:32:45.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.097+0000 7f017a005640 1 Processor -- start 2024-04-25T10:32:45.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.097+0000 7f017a005640 1 -- start start 2024-04-25T10:32:45.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.099+0000 7f017a005640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f01740c8190 0x7f0174156c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:45.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.099+0000 7f017a005640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0174160340 con 0x7f0174074630 2024-04-25T10:32:45.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.099+0000 7f017a005640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0174160620 con 0x7f01740c9920 2024-04-25T10:32:45.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.099+0000 7f017a005640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f017415dbe0 con 0x7f01740c8190 2024-04-25T10:32:45.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.099+0000 7f0172ffd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f0174074630 0x7f01740c7a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:56734/0 (socket says 172.21.15.106:56734) 2024-04-25T10:32:45.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.099+0000 7f0172ffd640 1 -- 172.21.15.106:0/364405222 learned_addr learned my addr 172.21.15.106:0/364405222 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:32:45.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.099+0000 7f0173fff640 1 --2- 172.21.15.106:0/364405222 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f01740c8190 0x7f0174156c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:45.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.099+0000 7f01727fc640 1 -- 172.21.15.106:0/364405222 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1818062873 0 0) 0x7f0174160340 con 0x7f0174074630 2024-04-25T10:32:45.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.099+0000 7f01727fc640 1 -- 172.21.15.106:0/364405222 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f015c003520 con 0x7f0174074630 2024-04-25T10:32:45.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.099+0000 7f0173fff640 1 -- 172.21.15.106:0/364405222 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f01740c8190 msgr2=0x7f0174156c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T10:32:45.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.099+0000 7f0173fff640 1 -- 172.21.15.106:0/364405222 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f01740c8190 msgr2=0x7f0174156c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:32:45.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.099+0000 7f0173fff640 1 --2- 172.21.15.106:0/364405222 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f01740c8190 0x7f0174156c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:32:45.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.099+0000 7f0173fff640 1 --2- 172.21.15.106:0/364405222 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f01740c8190 0x7f0174156c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:32:45.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.099+0000 7f01727fc640 1 -- 172.21.15.106:0/364405222 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2573573184 0 0) 0x7f0174160620 con 0x7f01740c9920 2024-04-25T10:32:45.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.099+0000 7f01727fc640 1 -- 172.21.15.106:0/364405222 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0174160340 con 0x7f01740c9920 2024-04-25T10:32:45.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.099+0000 7f01727fc640 1 -- 172.21.15.106:0/364405222 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3619742653 0 0) 0x7f015c003520 con 0x7f0174074630 2024-04-25T10:32:45.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.099+0000 7f01727fc640 1 -- 172.21.15.106:0/364405222 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f01740c8190 msgr2=0x7f0174156c50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:32:45.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.099+0000 7f01727fc640 1 --2- 172.21.15.106:0/364405222 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f01740c8190 0x7f0174156c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:45.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.099+0000 7f01727fc640 1 -- 172.21.15.106:0/364405222 >> v1:172.21.15.134:6789/0 conn(0x7f01740c9920 legacy=0x7f01740738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:45.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.100+0000 7f01727fc640 1 -- 172.21.15.106:0/364405222 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0174160900 con 0x7f0174074630 2024-04-25T10:32:45.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.100+0000 7f01727fc640 1 -- 172.21.15.106:0/364405222 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3776459350 0 0) 0x7f0168001670 con 0x7f0174074630 2024-04-25T10:32:45.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.100+0000 7f01727fc640 1 -- 172.21.15.106:0/364405222 >> v1:172.21.15.106:6789/0 conn(0x7f0174074630 legacy=0x7f01740c7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:45.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.100+0000 7f01727fc640 1 --2- 172.21.15.106:0/364405222 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f015c003b70 0x7f015c003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:45.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.100+0000 7f01727fc640 1 --2- 172.21.15.106:0/364405222 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f015c004800 0x7f015c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:45.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.100+0000 7f01727fc640 1 --2- 172.21.15.106:0/364405222 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f01740c8190 0x7f015c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:45.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.100+0000 7f01727fc640 1 -- 172.21.15.106:0/364405222 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f017415dbe0 con 0x7f015c004800 2024-04-25T10:32:45.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.100+0000 7f01727fc640 1 -- 172.21.15.106:0/364405222 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f015c005520 con 0x7f015c003b70 2024-04-25T10:32:45.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.100+0000 7f01727fc640 1 -- 172.21.15.106:0/364405222 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f015c00bf00 con 0x7f01740c8190 2024-04-25T10:32:45.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.100+0000 7f0172ffd640 1 --2- 172.21.15.106:0/364405222 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f015c004800 0x7f015c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:45.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.100+0000 7f01737fe640 1 --2- 172.21.15.106:0/364405222 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f015c003b70 0x7f015c003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:45.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.100+0000 7f0173fff640 1 --2- 172.21.15.106:0/364405222 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f01740c8190 0x7f015c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:45.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.100+0000 7f0172ffd640 1 -- 172.21.15.106:0/364405222 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f01740c8190 msgr2=0x7f015c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:45.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.100+0000 7f0172ffd640 1 --2- 172.21.15.106:0/364405222 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f01740c8190 0x7f015c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:45.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.100+0000 7f0172ffd640 1 -- 172.21.15.106:0/364405222 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f015c003b70 msgr2=0x7f015c003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:45.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.100+0000 7f0172ffd640 1 --2- 172.21.15.106:0/364405222 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f015c003b70 0x7f015c003fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:45.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.100+0000 7f0172ffd640 1 -- 172.21.15.106:0/364405222 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f015c003520 con 0x7f015c004800 2024-04-25T10:32:45.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.101+0000 7f0172ffd640 1 --2- 172.21.15.106:0/364405222 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f015c004800 0x7f015c004c10 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0168007fd0 tx=0x7f015c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d631d77debb1309f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:45.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.101+0000 7f01727fc640 1 -- 172.21.15.106:0/364405222 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0168007ca0 con 0x7f015c004800 2024-04-25T10:32:45.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.101+0000 7f01727fc640 1 -- 172.21.15.106:0/364405222 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0168005ad0 con 0x7f015c004800 2024-04-25T10:32:45.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.101+0000 7f01727fc640 1 -- 172.21.15.106:0/364405222 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0168007070 con 0x7f015c004800 2024-04-25T10:32:45.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.102+0000 7f017a005640 1 -- 172.21.15.106:0/364405222 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f015c004800 msgr2=0x7f015c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:45.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.102+0000 7f017a005640 1 --2- 172.21.15.106:0/364405222 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f015c004800 0x7f015c004c10 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0168007fd0 tx=0x7f015c005830 comp rx=0 tx=0).stop 2024-04-25T10:32:45.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.102+0000 7f0173fff640 1 -- 172.21.15.106:0/364405222 reap_dead start 2024-04-25T10:32:45.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.102+0000 7f017a005640 1 -- 172.21.15.106:0/364405222 shutdown_connections 2024-04-25T10:32:45.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.102+0000 7f017a005640 1 -- 172.21.15.106:0/364405222 >> 172.21.15.106:0/364405222 conn(0x7f017406d6d0 msgr2=0x7f017406dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:45.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.102+0000 7f017a005640 1 -- 172.21.15.106:0/364405222 shutdown_connections 2024-04-25T10:32:45.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.102+0000 7f017a005640 1 -- 172.21.15.106:0/364405222 wait complete. 2024-04-25T10:32:45.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.104+0000 7f017a005640 1 Processor -- start 2024-04-25T10:32:45.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.105+0000 7f017a005640 1 -- start start 2024-04-25T10:32:45.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.106+0000 7f017a005640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f015c004800 0x7f0174150a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:45.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.106+0000 7f017a005640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0174151e40 0x7f01741e9c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:45.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.106+0000 7f017a005640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f017415fda0 0x7f017414c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:45.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.106+0000 7f017a005640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0174073fe0 con 0x7f015c004800 2024-04-25T10:32:45.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.106+0000 7f017a005640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f017406e280 con 0x7f0174151e40 2024-04-25T10:32:45.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.106+0000 7f017a005640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f0174074160 con 0x7f017415fda0 2024-04-25T10:32:45.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.106+0000 7f01737fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f015c004800 0x7f0174150a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:45.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.106+0000 7f01737fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f015c004800 0x7f0174150a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60920/0 (socket says 172.21.15.106:60920) 2024-04-25T10:32:45.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.106+0000 7f01737fe640 1 -- 172.21.15.106:0/764490058 learned_addr learned my addr 172.21.15.106:0/764490058 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:32:45.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.106+0000 7f0173fff640 1 --2- 172.21.15.106:0/764490058 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f017415fda0 0x7f017414c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:45.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.106+0000 7f0172ffd640 1 --2- 172.21.15.106:0/764490058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0174151e40 0x7f01741e9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:45.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.107+0000 7f0173fff640 1 -- 172.21.15.106:0/764490058 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f017415fda0 msgr2=0x7f017414c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-04-25T10:32:45.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.107+0000 7f0173fff640 1 -- 172.21.15.106:0/764490058 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f017415fda0 msgr2=0x7f017414c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T10:32:45.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.107+0000 7f01737fe640 1 -- 172.21.15.106:0/764490058 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f017415fda0 msgr2=0x7f017414c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:45.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.107+0000 7f0173fff640 1 --2- 172.21.15.106:0/764490058 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f017415fda0 0x7f017414c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:32:45.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.107+0000 7f0173fff640 1 --2- 172.21.15.106:0/764490058 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f017415fda0 0x7f017414c6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:32:45.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.107+0000 7f01737fe640 1 --2- 172.21.15.106:0/764490058 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f017415fda0 0x7f017414c6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:45.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.107+0000 7f01737fe640 1 -- 172.21.15.106:0/764490058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0174151e40 msgr2=0x7f01741e9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:45.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.107+0000 7f01737fe640 1 --2- 172.21.15.106:0/764490058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0174151e40 0x7f01741e9c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:45.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.107+0000 7f01737fe640 1 -- 172.21.15.106:0/764490058 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01741526b0 con 0x7f015c004800 2024-04-25T10:32:45.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.107+0000 7f0172ffd640 1 --2- 172.21.15.106:0/764490058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0174151e40 0x7f01741e9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:32:45.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.107+0000 7f01737fe640 1 --2- 172.21.15.106:0/764490058 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f015c004800 0x7f0174150a10 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f016400f7d0 tx=0x7f016400fc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:45.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.107+0000 7f015bfff640 1 -- 172.21.15.106:0/764490058 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f016400fe80 con 0x7f015c004800 2024-04-25T10:32:45.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.107+0000 7f015bfff640 1 -- 172.21.15.106:0/764490058 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0164018070 con 0x7f015c004800 2024-04-25T10:32:45.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.107+0000 7f015bfff640 1 -- 172.21.15.106:0/764490058 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f016400d730 con 0x7f015c004800 2024-04-25T10:32:45.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.107+0000 7f017a005640 1 -- 172.21.15.106:0/764490058 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01741f2fc0 con 0x7f015c004800 2024-04-25T10:32:45.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.107+0000 7f017a005640 1 -- 172.21.15.106:0/764490058 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01741f2c70 con 0x7f015c004800 2024-04-25T10:32:45.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.108+0000 7f017a005640 1 --2- 172.21.15.106:0/764490058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f017415dde0 0x7f017415e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:45.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.108+0000 7f017a005640 1 -- 172.21.15.106:0/764490058 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0174153580 con 0x7f017415dde0 2024-04-25T10:32:45.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.108+0000 7f0172ffd640 1 --2- 172.21.15.106:0/764490058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f017415dde0 0x7f017415e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:45.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.109+0000 7f0172ffd640 1 -- 172.21.15.106:0/764490058 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f01741ea160 con 0x7f017415dde0 2024-04-25T10:32:45.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.109+0000 7f0172ffd640 1 --2- 172.21.15.106:0/764490058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f017415dde0 0x7f017415e1f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f01740704f0 tx=0x7f01740c6d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:45.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.109+0000 7f015bfff640 1 -- 172.21.15.106:0/764490058 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f016400d8d0 con 0x7f015c004800 2024-04-25T10:32:45.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.109+0000 7f015bfff640 1 --2- 172.21.15.106:0/764490058 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f01600744e0 0x7f01600769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:45.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.110+0000 7f015bfff640 1 -- 172.21.15.106:0/764490058 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f0164095260 con 0x7f015c004800 2024-04-25T10:32:45.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.110+0000 7f0173fff640 1 --2- 172.21.15.106:0/764490058 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f01600744e0 0x7f01600769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:45.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.110+0000 7f015bfff640 1 -- 172.21.15.106:0/764490058 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0168037040 con 0x7f017415dde0 2024-04-25T10:32:45.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.110+0000 7f015bfff640 1 -- 172.21.15.106:0/764490058 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f01680047e0 con 0x7f017415dde0 2024-04-25T10:32:45.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.110+0000 7f015bfff640 1 -- 172.21.15.106:0/764490058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f017415dde0 msgr2=0x7f017415e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:45.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.110+0000 7f015bfff640 1 --2- 172.21.15.106:0/764490058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f017415dde0 0x7f017415e1f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f01740704f0 tx=0x7f01740c6d50 comp rx=0 tx=0).stop 2024-04-25T10:32:45.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.110+0000 7f015bfff640 1 -- 172.21.15.106:0/764490058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f017415dde0 msgr2=0x7f017415e1f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T10:32:45.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.110+0000 7f015bfff640 1 --2- 172.21.15.106:0/764490058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f017415dde0 0x7f017415e1f0 secure :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f01740704f0 tx=0x7f01740c6d50 comp rx=0 tx=0).stop 2024-04-25T10:32:45.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.110+0000 7f0173fff640 1 --2- 172.21.15.106:0/764490058 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f01600744e0 0x7f01600769f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f017406d630 tx=0x7f016c009770 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:45.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.116+0000 7f017a005640 1 --2- 172.21.15.106:0/764490058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01741f61e0 0x7f01741f65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:45.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.116+0000 7f017a005640 1 -- 172.21.15.106:0/764490058 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f017406a3d0 con 0x7f01741f61e0 2024-04-25T10:32:45.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.117+0000 7f0172ffd640 1 --2- 172.21.15.106:0/764490058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01741f61e0 0x7f01741f65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:45.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.117+0000 7f0172ffd640 1 -- 172.21.15.106:0/764490058 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f01740cc5f0 con 0x7f01741f61e0 2024-04-25T10:32:45.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.117+0000 7f0172ffd640 1 --2- 172.21.15.106:0/764490058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01741f61e0 0x7f01741f65f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f01680018f0 tx=0x7f016802d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:45.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.117+0000 7f015bfff640 1 -- 172.21.15.106:0/764490058 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01680448b0 con 0x7f01741f61e0 2024-04-25T10:32:45.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.118+0000 7f015bfff640 1 -- 172.21.15.106:0/764490058 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4729 (secure 0 0 0) 0x7f0168044290 con 0x7f01741f61e0 2024-04-25T10:32:45.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.118+0000 7f015bfff640 1 -- 172.21.15.106:0/764490058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01741f61e0 msgr2=0x7f01741f65f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:45.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.118+0000 7f015bfff640 1 --2- 172.21.15.106:0/764490058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01741f61e0 0x7f01741f65f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f01680018f0 tx=0x7f016802d000 comp rx=0 tx=0).stop 2024-04-25T10:32:45.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.118+0000 7f015bfff640 1 -- 172.21.15.106:0/764490058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01741f61e0 msgr2=0x7f01741f65f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T10:32:45.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.118+0000 7f015bfff640 1 --2- 172.21.15.106:0/764490058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01741f61e0 0x7f01741f65f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:45.121 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-25T10:32:45.121 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "b", 2024-04-25T10:32:45.121 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 1, 2024-04-25T10:32:45.121 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "peon", 2024-04-25T10:32:45.121 INFO:teuthology.orchestra.run.smithi106.stdout: "election_epoch": 16, 2024-04-25T10:32:45.121 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 157314, 2024-04-25T10:32:45.121 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum": [ 2024-04-25T10:32:45.122 INFO:teuthology.orchestra.run.smithi106.stdout: 0, 2024-04-25T10:32:45.122 INFO:teuthology.orchestra.run.smithi106.stdout: 1, 2024-04-25T10:32:45.122 INFO:teuthology.orchestra.run.smithi106.stdout: 2 2024-04-25T10:32:45.122 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:32:45.122 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_age": 3, 2024-04-25T10:32:45.122 INFO:teuthology.orchestra.run.smithi106.stdout: "features": { 2024-04-25T10:32:45.122 INFO:teuthology.orchestra.run.smithi106.stdout: "required_con": "2450521708007866372", 2024-04-25T10:32:45.122 INFO:teuthology.orchestra.run.smithi106.stdout: "required_mon": [ 2024-04-25T10:32:45.122 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:32:45.122 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:32:45.122 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:32:45.122 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:32:45.122 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:32:45.122 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:32:45.123 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:32:45.123 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:32:45.123 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:32:45.123 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:32:45.123 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:32:45.123 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:32:45.123 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_con": "4540701547738038271", 2024-04-25T10:32:45.123 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_mon": [ 2024-04-25T10:32:45.123 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:32:45.123 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:32:45.123 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:32:45.123 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:32:45.123 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:32:45.123 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:32:45.123 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:32:45.123 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:32:45.124 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:32:45.124 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:32:45.124 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:32:45.124 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:32:45.124 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:45.124 INFO:teuthology.orchestra.run.smithi106.stdout: "outside_quorum": [], 2024-04-25T10:32:45.124 INFO:teuthology.orchestra.run.smithi106.stdout: "extra_probe_peers": [], 2024-04-25T10:32:45.124 INFO:teuthology.orchestra.run.smithi106.stdout: "sync_provider": [], 2024-04-25T10:32:45.124 INFO:teuthology.orchestra.run.smithi106.stdout: "monmap": { 2024-04-25T10:32:45.124 INFO:teuthology.orchestra.run.smithi106.stdout: "epoch": 1, 2024-04-25T10:32:45.124 INFO:teuthology.orchestra.run.smithi106.stdout: "fsid": "3ee77985-e8e4-41cf-81e9-7c774cf7891f", 2024-04-25T10:32:45.124 INFO:teuthology.orchestra.run.smithi106.stdout: "modified": "2024-04-25T10:29:33.446228Z", 2024-04-25T10:32:45.124 INFO:teuthology.orchestra.run.smithi106.stdout: "created": "2024-04-25T10:29:33.446228Z", 2024-04-25T10:32:45.124 INFO:teuthology.orchestra.run.smithi106.stdout: "min_mon_release": 19, 2024-04-25T10:32:45.125 INFO:teuthology.orchestra.run.smithi106.stdout: "min_mon_release_name": "squid", 2024-04-25T10:32:45.125 INFO:teuthology.orchestra.run.smithi106.stdout: "election_strategy": 1, 2024-04-25T10:32:45.125 INFO:teuthology.orchestra.run.smithi106.stdout: "disallowed_leaders: ": "", 2024-04-25T10:32:45.125 INFO:teuthology.orchestra.run.smithi106.stdout: "stretch_mode": false, 2024-04-25T10:32:45.125 INFO:teuthology.orchestra.run.smithi106.stdout: "tiebreaker_mon": "", 2024-04-25T10:32:45.125 INFO:teuthology.orchestra.run.smithi106.stdout: "removed_ranks: ": "", 2024-04-25T10:32:45.125 INFO:teuthology.orchestra.run.smithi106.stdout: "features": { 2024-04-25T10:32:45.125 INFO:teuthology.orchestra.run.smithi106.stdout: "persistent": [ 2024-04-25T10:32:45.125 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:32:45.125 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:32:45.125 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:32:45.125 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:32:45.125 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:32:45.125 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:32:45.126 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:32:45.126 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:32:45.126 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:32:45.126 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:32:45.126 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:32:45.126 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:32:45.126 INFO:teuthology.orchestra.run.smithi106.stdout: "optional": [] 2024-04-25T10:32:45.126 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:45.126 INFO:teuthology.orchestra.run.smithi106.stdout: "mons": [ 2024-04-25T10:32:45.126 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:45.126 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 0, 2024-04-25T10:32:45.127 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "a", 2024-04-25T10:32:45.127 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:32:45.127 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:32:45.127 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:45.127 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:32:45.127 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:3300", 2024-04-25T10:32:45.127 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:32:45.127 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:45.127 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:45.127 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:32:45.127 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:6789", 2024-04-25T10:32:45.127 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:32:45.127 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:45.127 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:32:45.128 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:45.128 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:6789/0", 2024-04-25T10:32:45.128 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.106:6789/0", 2024-04-25T10:32:45.128 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:32:45.128 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:32:45.128 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:32:45.128 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:45.128 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:45.128 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 1, 2024-04-25T10:32:45.128 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "b", 2024-04-25T10:32:45.129 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:32:45.129 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:32:45.129 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:45.129 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:32:45.129 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:3300", 2024-04-25T10:32:45.129 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:32:45.129 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:45.129 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:45.129 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:32:45.129 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6789", 2024-04-25T10:32:45.129 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:32:45.129 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:45.129 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:32:45.130 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:45.130 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6789/0", 2024-04-25T10:32:45.130 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.134:6789/0", 2024-04-25T10:32:45.130 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:32:45.130 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:32:45.130 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:32:45.130 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:45.130 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:45.130 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 2, 2024-04-25T10:32:45.130 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "c", 2024-04-25T10:32:45.131 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:32:45.131 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:32:45.131 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:45.131 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:32:45.131 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:3301", 2024-04-25T10:32:45.131 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:32:45.131 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:45.131 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:45.131 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:32:45.131 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6790", 2024-04-25T10:32:45.131 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:32:45.131 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:45.131 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:32:45.131 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:45.131 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6790/0", 2024-04-25T10:32:45.132 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.134:6790/0", 2024-04-25T10:32:45.132 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:32:45.132 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:32:45.132 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:32:45.132 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:45.132 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:32:45.132 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:45.132 INFO:teuthology.orchestra.run.smithi106.stdout: "feature_map": { 2024-04-25T10:32:45.132 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": [ 2024-04-25T10:32:45.132 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:45.132 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:32:45.132 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:32:45.132 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 1 2024-04-25T10:32:45.132 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:45.133 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:32:45.133 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": [ 2024-04-25T10:32:45.133 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:45.133 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:32:45.133 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:32:45.133 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 4 2024-04-25T10:32:45.133 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:45.133 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:32:45.133 INFO:teuthology.orchestra.run.smithi106.stdout: "client": [ 2024-04-25T10:32:45.133 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:45.133 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:32:45.133 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:32:45.133 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 1 2024-04-25T10:32:45.133 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:45.134 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:32:45.134 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:45.134 INFO:teuthology.orchestra.run.smithi106.stdout: "stretch_mode": false 2024-04-25T10:32:45.134 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-25T10:32:45.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.118+0000 7f017a005640 1 -- 172.21.15.106:0/764490058 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f01600744e0 msgr2=0x7f01600769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:45.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.118+0000 7f017a005640 1 --2- 172.21.15.106:0/764490058 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f01600744e0 0x7f01600769f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f017406d630 tx=0x7f016c009770 comp rx=0 tx=0).stop 2024-04-25T10:32:45.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.118+0000 7f017a005640 1 -- 172.21.15.106:0/764490058 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f015c004800 msgr2=0x7f0174150a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:45.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.118+0000 7f017a005640 1 --2- 172.21.15.106:0/764490058 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f015c004800 0x7f0174150a10 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f016400f7d0 tx=0x7f016400fc90 comp rx=0 tx=0).stop 2024-04-25T10:32:45.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.118+0000 7f0173fff640 1 -- 172.21.15.106:0/764490058 reap_dead start 2024-04-25T10:32:45.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.118+0000 7f017a005640 1 -- 172.21.15.106:0/764490058 shutdown_connections 2024-04-25T10:32:45.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.118+0000 7f017a005640 1 --2- 172.21.15.106:0/764490058 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f015c004800 0x7f0174150a10 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:45.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.118+0000 7f017a005640 1 -- 172.21.15.106:0/764490058 >> 172.21.15.106:0/764490058 conn(0x7f017406d6d0 msgr2=0x7f01740c79f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:45.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.118+0000 7f017a005640 1 -- 172.21.15.106:0/764490058 shutdown_connections 2024-04-25T10:32:45.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.118+0000 7f017a005640 1 -- 172.21.15.106:0/764490058 wait complete. 2024-04-25T10:32:45.136 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T10:32:45.136 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-04-25T10:32:45.136 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-25T10:32:45.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.262+0000 7fb8aa39d640 1 Processor -- start 2024-04-25T10:32:45.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.262+0000 7fb8aa39d640 1 -- start start 2024-04-25T10:32:45.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.264+0000 7fb8aa39d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb8a4156e60 0x7fb8a40ce350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:45.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.264+0000 7fb8aa39d640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fb8a40c75d0 0x7fb8a415d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:45.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.264+0000 7fb8aa39d640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8a40c7b50 con 0x7fb8a40ccd20 2024-04-25T10:32:45.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.264+0000 7fb8aa39d640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fb8a40c7df0 con 0x7fb8a40c75d0 2024-04-25T10:32:45.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.264+0000 7fb8aa39d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb8a40c80d0 con 0x7fb8a4156e60 2024-04-25T10:32:45.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.264+0000 7fb8a9b9c640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fb8a40ccd20 0x7fb8a40d1e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:56740/0 (socket says 172.21.15.106:56740) 2024-04-25T10:32:45.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.264+0000 7fb8a9b9c640 1 -- 172.21.15.106:0/61840075 learned_addr learned my addr 172.21.15.106:0/61840075 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:32:45.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.264+0000 7fb8a939b640 1 --2- 172.21.15.106:0/61840075 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb8a4156e60 0x7fb8a40ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:45.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.264+0000 7fb8a8b9a640 1 --2- 172.21.15.106:0/61840075 >> v2:172.21.15.134:3300/0 conn(0x7fb8a40c75d0 0x7fb8a415d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:45.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.265+0000 7fb8a939b640 1 -- 172.21.15.106:0/61840075 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb8a4156e60 msgr2=0x7fb8a40ce350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T10:32:45.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.265+0000 7fb8a939b640 1 -- 172.21.15.106:0/61840075 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb8a4156e60 msgr2=0x7fb8a40ce350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:32:45.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.265+0000 7fb8a939b640 1 --2- 172.21.15.106:0/61840075 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb8a4156e60 0x7fb8a40ce350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:32:45.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.265+0000 7fb8a939b640 1 --2- 172.21.15.106:0/61840075 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb8a4156e60 0x7fb8a40ce350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:32:45.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.265+0000 7fb89bfff640 1 -- 172.21.15.106:0/61840075 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3184231098 0 0) 0x7fb8a40c7b50 con 0x7fb8a40ccd20 2024-04-25T10:32:45.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.265+0000 7fb89bfff640 1 -- 172.21.15.106:0/61840075 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb890003520 con 0x7fb8a40ccd20 2024-04-25T10:32:45.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.265+0000 7fb8a8b9a640 1 -- 172.21.15.106:0/61840075 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb8a4156e60 msgr2=0x7fb8a40ce350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:32:45.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.265+0000 7fb8a8b9a640 1 --2- 172.21.15.106:0/61840075 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb8a4156e60 0x7fb8a40ce350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:45.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.265+0000 7fb8a8b9a640 1 -- 172.21.15.106:0/61840075 >> v1:172.21.15.106:6789/0 conn(0x7fb8a40ccd20 legacy=0x7fb8a40d1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:45.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.265+0000 7fb8a8b9a640 1 -- 172.21.15.106:0/61840075 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8a40c83b0 con 0x7fb8a40c75d0 2024-04-25T10:32:45.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.266+0000 7fb8a8b9a640 1 --2- 172.21.15.106:0/61840075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb8a40c75d0 0x7fb8a415d530 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb8940097a0 tx=0x7fb89402a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57045acee1bb8cb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:45.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.266+0000 7fb89bfff640 1 -- 172.21.15.106:0/61840075 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb894037070 con 0x7fb8a40c75d0 2024-04-25T10:32:45.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.266+0000 7fb89bfff640 1 -- 172.21.15.106:0/61840075 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb894032440 con 0x7fb8a40c75d0 2024-04-25T10:32:45.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.266+0000 7fb89bfff640 1 -- 172.21.15.106:0/61840075 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb894032810 con 0x7fb8a40c75d0 2024-04-25T10:32:45.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.266+0000 7fb8aa39d640 1 -- 172.21.15.106:0/61840075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb8a40c75d0 msgr2=0x7fb8a415d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:45.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.266+0000 7fb8aa39d640 1 --2- 172.21.15.106:0/61840075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb8a40c75d0 0x7fb8a415d530 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb8940097a0 tx=0x7fb89402a210 comp rx=0 tx=0).stop 2024-04-25T10:32:45.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.266+0000 7fb8aa39d640 1 -- 172.21.15.106:0/61840075 shutdown_connections 2024-04-25T10:32:45.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.266+0000 7fb8aa39d640 1 --2- 172.21.15.106:0/61840075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb8a40c75d0 0x7fb8a415d530 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:45.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.266+0000 7fb8aa39d640 1 --2- 172.21.15.106:0/61840075 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb8a4156e60 0x7fb8a40ce350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:45.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.267+0000 7fb8aa39d640 1 -- 172.21.15.106:0/61840075 >> 172.21.15.106:0/61840075 conn(0x7fb8a4073cf0 msgr2=0x7fb8a406d540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:45.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.267+0000 7fb8aa39d640 1 -- 172.21.15.106:0/61840075 shutdown_connections 2024-04-25T10:32:45.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.267+0000 7fb8aa39d640 1 -- 172.21.15.106:0/61840075 wait complete. 2024-04-25T10:32:45.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.269+0000 7fb8aa39d640 1 Processor -- start 2024-04-25T10:32:45.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.269+0000 7fb8aa39d640 1 -- start start 2024-04-25T10:32:45.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.270+0000 7fb8aa39d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb8a40c75d0 0x7fb8a41f3090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:45.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.271+0000 7fb8aa39d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb8a40ccd20 0x7fb8a4154e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:45.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.271+0000 7fb8aa39d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8a4156e60 0x7fb8a40d0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:45.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.271+0000 7fb8aa39d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a415c9e0 con 0x7fb8a4156e60 2024-04-25T10:32:45.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.271+0000 7fb8aa39d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a415d010 con 0x7fb8a40c75d0 2024-04-25T10:32:45.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.271+0000 7fb8aa39d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb8a40d2560 con 0x7fb8a40ccd20 2024-04-25T10:32:45.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.271+0000 7fb8a9b9c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8a4156e60 0x7fb8a40d0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:45.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.271+0000 7fb8a9b9c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8a4156e60 0x7fb8a40d0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60936/0 (socket says 172.21.15.106:60936) 2024-04-25T10:32:45.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.271+0000 7fb8a9b9c640 1 -- 172.21.15.106:0/829667674 learned_addr learned my addr 172.21.15.106:0/829667674 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:32:45.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.271+0000 7fb8a8b9a640 1 --2- 172.21.15.106:0/829667674 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb8a40ccd20 0x7fb8a4154e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:45.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.271+0000 7fb8a939b640 1 --2- 172.21.15.106:0/829667674 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb8a40c75d0 0x7fb8a41f3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:45.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.271+0000 7fb8a9b9c640 1 -- 172.21.15.106:0/829667674 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb8a40ccd20 msgr2=0x7fb8a4154e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:45.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.271+0000 7fb8a9b9c640 1 --2- 172.21.15.106:0/829667674 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb8a40ccd20 0x7fb8a4154e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:45.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.271+0000 7fb8a9b9c640 1 -- 172.21.15.106:0/829667674 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb8a40c75d0 msgr2=0x7fb8a41f3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:45.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.271+0000 7fb8a9b9c640 1 --2- 172.21.15.106:0/829667674 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb8a40c75d0 0x7fb8a41f3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:45.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.271+0000 7fb8a9b9c640 1 -- 172.21.15.106:0/829667674 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8a41553c0 con 0x7fb8a4156e60 2024-04-25T10:32:45.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.271+0000 7fb8a9b9c640 1 --2- 172.21.15.106:0/829667674 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8a4156e60 0x7fb8a40d0ed0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb89c00caa0 tx=0x7fb89c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:45.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.271+0000 7fb899ffb640 1 -- 172.21.15.106:0/829667674 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89c017070 con 0x7fb8a4156e60 2024-04-25T10:32:45.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.271+0000 7fb899ffb640 1 -- 172.21.15.106:0/829667674 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb89c00ac00 con 0x7fb8a4156e60 2024-04-25T10:32:45.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.271+0000 7fb8aa39d640 1 -- 172.21.15.106:0/829667674 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8a41561e0 con 0x7fb8a4156e60 2024-04-25T10:32:45.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.271+0000 7fb899ffb640 1 -- 172.21.15.106:0/829667674 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89c012400 con 0x7fb8a4156e60 2024-04-25T10:32:45.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.272+0000 7fb8aa39d640 1 -- 172.21.15.106:0/829667674 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8a4154680 con 0x7fb8a4156e60 2024-04-25T10:32:45.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.273+0000 7fb8aa39d640 1 --2- 172.21.15.106:0/829667674 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb8a40d1b10 0x7fb8a414cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:45.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.273+0000 7fb8aa39d640 1 -- 172.21.15.106:0/829667674 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb8a41544c0 con 0x7fb8a40d1b10 2024-04-25T10:32:45.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.273+0000 7fb8a939b640 1 --2- 172.21.15.106:0/829667674 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb8a40d1b10 0x7fb8a414cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:45.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.273+0000 7fb8a939b640 1 -- 172.21.15.106:0/829667674 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb8a40d1b10 msgr2=0x7fb8a414cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 20 2024-04-25T10:32:45.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.273+0000 7fb8a939b640 1 -- 172.21.15.106:0/829667674 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb8a40d1b10 msgr2=0x7fb8a414cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T10:32:45.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.273+0000 7fb8a939b640 1 --2- 172.21.15.106:0/829667674 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb8a40d1b10 0x7fb8a414cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:32:45.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.273+0000 7fb8a939b640 1 --2- 172.21.15.106:0/829667674 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb8a40d1b10 0x7fb8a414cbc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:32:45.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.273+0000 7fb899ffb640 1 -- 172.21.15.106:0/829667674 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb89c01a070 con 0x7fb8a4156e60 2024-04-25T10:32:45.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.274+0000 7fb899ffb640 1 --2- 172.21.15.106:0/829667674 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fb8880744b0 0x7fb8880769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:45.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.274+0000 7fb8a8b9a640 1 --2- 172.21.15.106:0/829667674 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fb8880744b0 0x7fb8880769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:45.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.274+0000 7fb899ffb640 1 -- 172.21.15.106:0/829667674 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fb89c094fc0 con 0x7fb8a4156e60 2024-04-25T10:32:45.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.274+0000 7fb8a8b9a640 1 --2- 172.21.15.106:0/829667674 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fb8880744b0 0x7fb8880769c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb894034000 tx=0x7fb89402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:45.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.474+0000 7fb8a939b640 1 --2- 172.21.15.106:0/829667674 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb8a40d1b10 0x7fb8a414cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:45.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.474+0000 7fb8a939b640 1 -- 172.21.15.106:0/829667674 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb8a40d1b10 msgr2=0x7fb8a414cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-04-25T10:32:45.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.474+0000 7fb8a939b640 1 -- 172.21.15.106:0/829667674 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb8a40d1b10 msgr2=0x7fb8a414cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T10:32:45.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.474+0000 7fb8a939b640 1 --2- 172.21.15.106:0/829667674 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb8a40d1b10 0x7fb8a414cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:32:45.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.474+0000 7fb8a939b640 1 --2- 172.21.15.106:0/829667674 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb8a40d1b10 0x7fb8a414cbc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T10:32:45.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.875+0000 7fb8a939b640 1 --2- 172.21.15.106:0/829667674 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb8a40d1b10 0x7fb8a414cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:45.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.875+0000 7fb8a939b640 1 -- 172.21.15.106:0/829667674 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb8a40d1b10 msgr2=0x7fb8a414cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-04-25T10:32:45.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.875+0000 7fb8a939b640 1 -- 172.21.15.106:0/829667674 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb8a40d1b10 msgr2=0x7fb8a414cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T10:32:45.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.875+0000 7fb8a939b640 1 --2- 172.21.15.106:0/829667674 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb8a40d1b10 0x7fb8a414cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:32:45.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:45.875+0000 7fb8a939b640 1 --2- 172.21.15.106:0/829667674 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb8a40d1b10 0x7fb8a414cbc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T10:32:46.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.676+0000 7fb8a939b640 1 --2- 172.21.15.106:0/829667674 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb8a40d1b10 0x7fb8a414cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:46.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.676+0000 7fb8a939b640 1 -- 172.21.15.106:0/829667674 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb8a40cc2f0 con 0x7fb8a40d1b10 2024-04-25T10:32:46.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.676+0000 7fb8a939b640 1 --2- 172.21.15.106:0/829667674 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb8a40d1b10 0x7fb8a414cbc0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0015900 tx=0x7fb8a0015dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:46.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.677+0000 7fb899ffb640 1 -- 172.21.15.106:0/829667674 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a0011b20 con 0x7fb8a40d1b10 2024-04-25T10:32:46.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.677+0000 7fb899ffb640 1 -- 172.21.15.106:0/829667674 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fb8a001d070 con 0x7fb8a40d1b10 2024-04-25T10:32:46.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.677+0000 7fb899ffb640 1 -- 172.21.15.106:0/829667674 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb8a40d1b10 msgr2=0x7fb8a414cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:46.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.677+0000 7fb899ffb640 1 --2- 172.21.15.106:0/829667674 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb8a40d1b10 0x7fb8a414cbc0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0015900 tx=0x7fb8a0015dc0 comp rx=0 tx=0).stop 2024-04-25T10:32:46.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.677+0000 7fb899ffb640 1 -- 172.21.15.106:0/829667674 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb8a40d1b10 msgr2=0x7fb8a414cbc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T10:32:46.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.677+0000 7fb899ffb640 1 --2- 172.21.15.106:0/829667674 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb8a40d1b10 0x7fb8a414cbc0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:46.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.684+0000 7fb8aa39d640 1 --2- 172.21.15.106:0/829667674 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb8a4155590 0x7fb8a414d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:46.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.684+0000 7fb8aa39d640 1 -- 172.21.15.106:0/829667674 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb8a40cd5f0 con 0x7fb8a4155590 2024-04-25T10:32:46.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.684+0000 7fb8a939b640 1 --2- 172.21.15.106:0/829667674 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb8a4155590 0x7fb8a414d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:46.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.684+0000 7fb8a939b640 1 -- 172.21.15.106:0/829667674 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fb8a4156700 con 0x7fb8a4155590 2024-04-25T10:32:46.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.685+0000 7fb8a939b640 1 --2- 172.21.15.106:0/829667674 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb8a4155590 0x7fb8a414d100 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0016010 tx=0x7fb8a0004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:46.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.685+0000 7fb899ffb640 1 -- 172.21.15.106:0/829667674 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a0030020 con 0x7fb8a4155590 2024-04-25T10:32:46.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.685+0000 7fb899ffb640 1 -- 172.21.15.106:0/829667674 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4557 (secure 0 0 0) 0x7fb8a0017640 con 0x7fb8a4155590 2024-04-25T10:32:46.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.685+0000 7fb899ffb640 1 -- 172.21.15.106:0/829667674 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb8a4155590 msgr2=0x7fb8a414d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:46.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.685+0000 7fb899ffb640 1 --2- 172.21.15.106:0/829667674 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb8a4155590 0x7fb8a414d100 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0016010 tx=0x7fb8a0004730 comp rx=0 tx=0).stop 2024-04-25T10:32:46.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.685+0000 7fb899ffb640 1 -- 172.21.15.106:0/829667674 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb8a4155590 msgr2=0x7fb8a414d100 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T10:32:46.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.685+0000 7fb899ffb640 1 --2- 172.21.15.106:0/829667674 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb8a4155590 0x7fb8a414d100 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:46.688 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-25T10:32:46.688 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "c", 2024-04-25T10:32:46.688 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 2, 2024-04-25T10:32:46.688 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "peon", 2024-04-25T10:32:46.689 INFO:teuthology.orchestra.run.smithi106.stdout: "election_epoch": 16, 2024-04-25T10:32:46.689 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 5608, 2024-04-25T10:32:46.689 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum": [ 2024-04-25T10:32:46.689 INFO:teuthology.orchestra.run.smithi106.stdout: 0, 2024-04-25T10:32:46.689 INFO:teuthology.orchestra.run.smithi106.stdout: 1, 2024-04-25T10:32:46.689 INFO:teuthology.orchestra.run.smithi106.stdout: 2 2024-04-25T10:32:46.689 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:32:46.689 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_age": 4, 2024-04-25T10:32:46.689 INFO:teuthology.orchestra.run.smithi106.stdout: "features": { 2024-04-25T10:32:46.689 INFO:teuthology.orchestra.run.smithi106.stdout: "required_con": "2450521708007866372", 2024-04-25T10:32:46.689 INFO:teuthology.orchestra.run.smithi106.stdout: "required_mon": [ 2024-04-25T10:32:46.689 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:32:46.690 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:32:46.690 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:32:46.690 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:32:46.690 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:32:46.690 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:32:46.690 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:32:46.690 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:32:46.690 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:32:46.690 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:32:46.690 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:32:46.690 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:32:46.690 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_con": "4540701547738038271", 2024-04-25T10:32:46.691 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_mon": [ 2024-04-25T10:32:46.691 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:32:46.691 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:32:46.691 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:32:46.691 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:32:46.691 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:32:46.691 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:32:46.691 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:32:46.691 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:32:46.692 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:32:46.692 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:32:46.692 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:32:46.692 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:32:46.692 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:46.692 INFO:teuthology.orchestra.run.smithi106.stdout: "outside_quorum": [], 2024-04-25T10:32:46.692 INFO:teuthology.orchestra.run.smithi106.stdout: "extra_probe_peers": [], 2024-04-25T10:32:46.692 INFO:teuthology.orchestra.run.smithi106.stdout: "sync_provider": [], 2024-04-25T10:32:46.692 INFO:teuthology.orchestra.run.smithi106.stdout: "monmap": { 2024-04-25T10:32:46.692 INFO:teuthology.orchestra.run.smithi106.stdout: "epoch": 1, 2024-04-25T10:32:46.692 INFO:teuthology.orchestra.run.smithi106.stdout: "fsid": "3ee77985-e8e4-41cf-81e9-7c774cf7891f", 2024-04-25T10:32:46.692 INFO:teuthology.orchestra.run.smithi106.stdout: "modified": "2024-04-25T10:29:33.446228Z", 2024-04-25T10:32:46.693 INFO:teuthology.orchestra.run.smithi106.stdout: "created": "2024-04-25T10:29:33.446228Z", 2024-04-25T10:32:46.693 INFO:teuthology.orchestra.run.smithi106.stdout: "min_mon_release": 19, 2024-04-25T10:32:46.693 INFO:teuthology.orchestra.run.smithi106.stdout: "min_mon_release_name": "squid", 2024-04-25T10:32:46.693 INFO:teuthology.orchestra.run.smithi106.stdout: "election_strategy": 1, 2024-04-25T10:32:46.693 INFO:teuthology.orchestra.run.smithi106.stdout: "disallowed_leaders: ": "", 2024-04-25T10:32:46.693 INFO:teuthology.orchestra.run.smithi106.stdout: "stretch_mode": false, 2024-04-25T10:32:46.693 INFO:teuthology.orchestra.run.smithi106.stdout: "tiebreaker_mon": "", 2024-04-25T10:32:46.693 INFO:teuthology.orchestra.run.smithi106.stdout: "removed_ranks: ": "", 2024-04-25T10:32:46.693 INFO:teuthology.orchestra.run.smithi106.stdout: "features": { 2024-04-25T10:32:46.693 INFO:teuthology.orchestra.run.smithi106.stdout: "persistent": [ 2024-04-25T10:32:46.693 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:32:46.693 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:32:46.694 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:32:46.694 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:32:46.694 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:32:46.694 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:32:46.694 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:32:46.694 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:32:46.694 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:32:46.694 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:32:46.694 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:32:46.694 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:32:46.694 INFO:teuthology.orchestra.run.smithi106.stdout: "optional": [] 2024-04-25T10:32:46.694 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:46.695 INFO:teuthology.orchestra.run.smithi106.stdout: "mons": [ 2024-04-25T10:32:46.695 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:46.695 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 0, 2024-04-25T10:32:46.695 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "a", 2024-04-25T10:32:46.695 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:32:46.695 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:32:46.695 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:46.695 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:32:46.695 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:3300", 2024-04-25T10:32:46.695 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:32:46.695 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:46.695 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:46.695 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:32:46.696 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:6789", 2024-04-25T10:32:46.696 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:32:46.696 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:46.696 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:32:46.696 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:46.696 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:6789/0", 2024-04-25T10:32:46.696 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.106:6789/0", 2024-04-25T10:32:46.696 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:32:46.696 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:32:46.696 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:32:46.696 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:46.696 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:46.697 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 1, 2024-04-25T10:32:46.697 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "b", 2024-04-25T10:32:46.697 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:32:46.697 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:32:46.697 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:46.697 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:32:46.697 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:3300", 2024-04-25T10:32:46.697 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:32:46.697 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:46.697 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:46.697 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:32:46.697 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6789", 2024-04-25T10:32:46.697 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:32:46.698 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:46.698 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:32:46.698 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:46.698 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6789/0", 2024-04-25T10:32:46.698 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.134:6789/0", 2024-04-25T10:32:46.698 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:32:46.698 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:32:46.698 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:32:46.698 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:46.698 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:46.698 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 2, 2024-04-25T10:32:46.698 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "c", 2024-04-25T10:32:46.699 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:32:46.699 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:32:46.699 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:46.699 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:32:46.699 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:3301", 2024-04-25T10:32:46.699 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:32:46.699 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:46.699 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:46.699 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:32:46.699 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6790", 2024-04-25T10:32:46.699 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:32:46.699 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:46.699 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:32:46.700 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:46.700 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6790/0", 2024-04-25T10:32:46.700 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.134:6790/0", 2024-04-25T10:32:46.700 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:32:46.700 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:32:46.700 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:32:46.700 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:46.700 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:32:46.700 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:46.700 INFO:teuthology.orchestra.run.smithi106.stdout: "feature_map": { 2024-04-25T10:32:46.700 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": [ 2024-04-25T10:32:46.700 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:46.700 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:32:46.701 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:32:46.701 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 1 2024-04-25T10:32:46.701 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:46.701 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:32:46.701 INFO:teuthology.orchestra.run.smithi106.stdout: "client": [ 2024-04-25T10:32:46.701 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:46.701 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:32:46.701 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:32:46.701 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 1 2024-04-25T10:32:46.701 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:46.701 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:32:46.701 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:46.702 INFO:teuthology.orchestra.run.smithi106.stdout: "stretch_mode": false 2024-04-25T10:32:46.702 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-25T10:32:46.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.685+0000 7fb8aa39d640 1 -- 172.21.15.106:0/829667674 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fb8880744b0 msgr2=0x7fb8880769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:46.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.685+0000 7fb8aa39d640 1 --2- 172.21.15.106:0/829667674 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fb8880744b0 0x7fb8880769c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb894034000 tx=0x7fb89402e000 comp rx=0 tx=0).stop 2024-04-25T10:32:46.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.685+0000 7fb8aa39d640 1 -- 172.21.15.106:0/829667674 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8a4156e60 msgr2=0x7fb8a40d0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:46.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.685+0000 7fb8aa39d640 1 --2- 172.21.15.106:0/829667674 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8a4156e60 0x7fb8a40d0ed0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb89c00caa0 tx=0x7fb89c00cf60 comp rx=0 tx=0).stop 2024-04-25T10:32:46.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.685+0000 7fb8a9b9c640 1 -- 172.21.15.106:0/829667674 reap_dead start 2024-04-25T10:32:46.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.685+0000 7fb8a9b9c640 1 -- 172.21.15.106:0/829667674 reap_dead start 2024-04-25T10:32:46.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.686+0000 7fb8aa39d640 1 -- 172.21.15.106:0/829667674 shutdown_connections 2024-04-25T10:32:46.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.686+0000 7fb8aa39d640 1 -- 172.21.15.106:0/829667674 >> 172.21.15.106:0/829667674 conn(0x7fb8a4073cf0 msgr2=0x7fb8a406d540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:46.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.686+0000 7fb8aa39d640 1 -- 172.21.15.106:0/829667674 shutdown_connections 2024-04-25T10:32:46.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.686+0000 7fb8aa39d640 1 -- 172.21.15.106:0/829667674 wait complete. 2024-04-25T10:32:46.704 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T10:32:46.704 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-04-25T10:32:46.704 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-04-25T10:32:46.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.816+0000 7f35d3a6e640 1 Processor -- start 2024-04-25T10:32:46.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.816+0000 7f35d3a6e640 1 -- start start 2024-04-25T10:32:46.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.818+0000 7f35d3a6e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35cc0738d0 0x7f35cc06c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:46.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.818+0000 7f35d3a6e640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f35cc154ca0 0x7f35cc0c8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:46.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.818+0000 7f35d3a6e640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35cc06cf10 con 0x7f35cc15b5e0 2024-04-25T10:32:46.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.818+0000 7f35d3a6e640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f35cc06cbb0 con 0x7f35cc154ca0 2024-04-25T10:32:46.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.818+0000 7f35d3a6e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f35cc0ca310 con 0x7f35cc0738d0 2024-04-25T10:32:46.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.819+0000 7f35d180a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35cc0738d0 0x7f35cc06c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:46.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.819+0000 7f35d1009640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f35cc15b5e0 0x7f35cc0cc280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.106:59612/0 (socket says 172.21.15.106:59612) 2024-04-25T10:32:46.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.819+0000 7f35d1009640 1 -- 172.21.15.106:0/1194578439 learned_addr learned my addr 172.21.15.106:0/1194578439 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:32:46.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.819+0000 7f35d0808640 1 -- 172.21.15.106:0/1194578439 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4164551783 0 0) 0x7f35cc06cf10 con 0x7f35cc15b5e0 2024-04-25T10:32:46.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.819+0000 7f35d200b640 1 --2- 172.21.15.106:0/1194578439 >> v2:172.21.15.134:3300/0 conn(0x7f35cc154ca0 0x7f35cc0c8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:46.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.819+0000 7f35d0808640 1 -- 172.21.15.106:0/1194578439 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f35b8003520 con 0x7f35cc15b5e0 2024-04-25T10:32:46.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.819+0000 7f35d180a640 1 -- 172.21.15.106:0/1194578439 >> v2:172.21.15.134:3300/0 conn(0x7f35cc154ca0 msgr2=0x7f35cc0c8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:46.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.819+0000 7f35d180a640 1 --2- 172.21.15.106:0/1194578439 >> v2:172.21.15.134:3300/0 conn(0x7f35cc154ca0 0x7f35cc0c8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:46.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.819+0000 7f35d180a640 1 -- 172.21.15.106:0/1194578439 >> v1:172.21.15.134:6789/0 conn(0x7f35cc15b5e0 legacy=0x7f35cc0cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:46.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.819+0000 7f35d180a640 1 -- 172.21.15.106:0/1194578439 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35cc0ca5d0 con 0x7f35cc0738d0 2024-04-25T10:32:46.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.820+0000 7f35d180a640 1 --2- 172.21.15.106:0/1194578439 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35cc0738d0 0x7f35cc06c670 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f35cc06cd30 tx=0x7f35c802a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c8149ff263eea50 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:46.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.820+0000 7f35d0808640 1 -- 172.21.15.106:0/1194578439 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c8037070 con 0x7f35cc0738d0 2024-04-25T10:32:46.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.821+0000 7f35d0808640 1 -- 172.21.15.106:0/1194578439 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35c8032440 con 0x7f35cc0738d0 2024-04-25T10:32:46.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.821+0000 7f35d0808640 1 -- 172.21.15.106:0/1194578439 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c8032810 con 0x7f35cc0738d0 2024-04-25T10:32:46.824 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.821+0000 7f35d3a6e640 1 -- 172.21.15.106:0/1194578439 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35cc0738d0 msgr2=0x7f35cc06c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:46.824 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.821+0000 7f35d3a6e640 1 --2- 172.21.15.106:0/1194578439 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35cc0738d0 0x7f35cc06c670 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f35cc06cd30 tx=0x7f35c802a1a0 comp rx=0 tx=0).stop 2024-04-25T10:32:46.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.822+0000 7f35d3a6e640 1 -- 172.21.15.106:0/1194578439 shutdown_connections 2024-04-25T10:32:46.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.822+0000 7f35d3a6e640 1 --2- 172.21.15.106:0/1194578439 >> v2:172.21.15.134:3300/0 conn(0x7f35cc154ca0 0x7f35cc0c8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:46.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.822+0000 7f35d3a6e640 1 --2- 172.21.15.106:0/1194578439 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35cc0738d0 0x7f35cc06c670 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:46.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.822+0000 7f35d3a6e640 1 -- 172.21.15.106:0/1194578439 >> 172.21.15.106:0/1194578439 conn(0x7f35cc06d6d0 msgr2=0x7f35cc06dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:46.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.822+0000 7f35d3a6e640 1 -- 172.21.15.106:0/1194578439 shutdown_connections 2024-04-25T10:32:46.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.822+0000 7f35d3a6e640 1 -- 172.21.15.106:0/1194578439 wait complete. 2024-04-25T10:32:46.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.824+0000 7f35d3a6e640 1 Processor -- start 2024-04-25T10:32:46.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.824+0000 7f35d3a6e640 1 -- start start 2024-04-25T10:32:46.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.825+0000 7f35d3a6e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35cc0738d0 0x7f35cc1f0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:46.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.825+0000 7f35d3a6e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35cc154ca0 0x7f35cc157800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:46.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.825+0000 7f35d3a6e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35cc15b5e0 0x7f35cc148660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:46.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.825+0000 7f35d3a6e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f35cc0ca8b0 con 0x7f35cc15b5e0 2024-04-25T10:32:46.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.825+0000 7f35d3a6e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f35cc0cba80 con 0x7f35cc154ca0 2024-04-25T10:32:46.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.825+0000 7f35d3a6e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f35cc15b9f0 con 0x7f35cc0738d0 2024-04-25T10:32:46.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.826+0000 7f35d200b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35cc15b5e0 0x7f35cc148660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:46.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.826+0000 7f35d200b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35cc15b5e0 0x7f35cc148660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60950/0 (socket says 172.21.15.106:60950) 2024-04-25T10:32:46.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.826+0000 7f35d200b640 1 -- 172.21.15.106:0/3718972721 learned_addr learned my addr 172.21.15.106:0/3718972721 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:32:46.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.826+0000 7f35d1009640 1 --2- 172.21.15.106:0/3718972721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35cc154ca0 0x7f35cc157800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:46.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.826+0000 7f35d180a640 1 --2- 172.21.15.106:0/3718972721 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35cc0738d0 0x7f35cc1f0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:46.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.826+0000 7f35d200b640 1 -- 172.21.15.106:0/3718972721 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35cc0738d0 msgr2=0x7f35cc1f0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:46.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.826+0000 7f35d200b640 1 --2- 172.21.15.106:0/3718972721 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35cc0738d0 0x7f35cc1f0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:46.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.826+0000 7f35d200b640 1 -- 172.21.15.106:0/3718972721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35cc154ca0 msgr2=0x7f35cc157800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:46.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.826+0000 7f35d200b640 1 --2- 172.21.15.106:0/3718972721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35cc154ca0 0x7f35cc157800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:46.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.826+0000 7f35d200b640 1 -- 172.21.15.106:0/3718972721 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35cc1f4190 con 0x7f35cc15b5e0 2024-04-25T10:32:46.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.826+0000 7f35d180a640 1 --2- 172.21.15.106:0/3718972721 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35cc0738d0 0x7f35cc1f0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:32:46.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.826+0000 7f35d1009640 1 --2- 172.21.15.106:0/3718972721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35cc154ca0 0x7f35cc157800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:32:46.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.826+0000 7f35d200b640 1 --2- 172.21.15.106:0/3718972721 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35cc15b5e0 0x7f35cc148660 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f35c4009fd0 tx=0x7f35c400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:46.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.827+0000 7f35c27fc640 1 -- 172.21.15.106:0/3718972721 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c4014420 con 0x7f35cc15b5e0 2024-04-25T10:32:46.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.827+0000 7f35c27fc640 1 -- 172.21.15.106:0/3718972721 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35c4019070 con 0x7f35cc15b5e0 2024-04-25T10:32:46.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.827+0000 7f35d3a6e640 1 -- 172.21.15.106:0/3718972721 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35cc1f4470 con 0x7f35cc15b5e0 2024-04-25T10:32:46.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.827+0000 7f35c27fc640 1 -- 172.21.15.106:0/3718972721 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c4014820 con 0x7f35cc15b5e0 2024-04-25T10:32:46.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.827+0000 7f35d3a6e640 1 -- 172.21.15.106:0/3718972721 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35cc1f4a30 con 0x7f35cc15b5e0 2024-04-25T10:32:46.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.828+0000 7f35d3a6e640 1 -- 172.21.15.106:0/3718972721 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35cc1502c0 con 0x7f35cc15b5e0 2024-04-25T10:32:46.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.828+0000 7f35c27fc640 1 -- 172.21.15.106:0/3718972721 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f35c40149c0 con 0x7f35cc15b5e0 2024-04-25T10:32:46.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.829+0000 7f35c27fc640 1 --2- 172.21.15.106:0/3718972721 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f35b00744b0 0x7f35b00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:46.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.829+0000 7f35c27fc640 1 -- 172.21.15.106:0/3718972721 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f35c4095910 con 0x7f35cc15b5e0 2024-04-25T10:32:46.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.829+0000 7f35d180a640 1 --2- 172.21.15.106:0/3718972721 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f35b00744b0 0x7f35b00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:46.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.830+0000 7f35d180a640 1 --2- 172.21.15.106:0/3718972721 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f35b00744b0 0x7f35b00769c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f35c8034000 tx=0x7f35c802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:46.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:46.834+0000 7f35c27fc640 1 -- 172.21.15.106:0/3718972721 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f35c405fa70 con 0x7f35cc15b5e0 2024-04-25T10:32:47.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:47.142+0000 7f35d3a6e640 1 -- 172.21.15.106:0/3718972721 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f35cc06a3d0 con 0x7f35cc15b5e0 2024-04-25T10:32:47.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:47.143+0000 7f35c27fc640 1 -- 172.21.15.106:0/3718972721 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7f35c4063d70 con 0x7f35cc15b5e0 2024-04-25T10:32:47.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:47.146+0000 7f35d3a6e640 1 -- 172.21.15.106:0/3718972721 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f35b00744b0 msgr2=0x7f35b00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:47.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:47.146+0000 7f35d3a6e640 1 --2- 172.21.15.106:0/3718972721 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f35b00744b0 0x7f35b00769c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f35c8034000 tx=0x7f35c802e000 comp rx=0 tx=0).stop 2024-04-25T10:32:47.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:47.146+0000 7f35d3a6e640 1 -- 172.21.15.106:0/3718972721 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35cc15b5e0 msgr2=0x7f35cc148660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:47.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:47.146+0000 7f35d3a6e640 1 --2- 172.21.15.106:0/3718972721 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35cc15b5e0 0x7f35cc148660 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f35c4009fd0 tx=0x7f35c400e4b0 comp rx=0 tx=0).stop 2024-04-25T10:32:47.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:47.146+0000 7f35d3a6e640 1 -- 172.21.15.106:0/3718972721 shutdown_connections 2024-04-25T10:32:47.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:47.146+0000 7f35d3a6e640 1 --2- 172.21.15.106:0/3718972721 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f35b00744b0 0x7f35b00769c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:47.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:47.146+0000 7f35d3a6e640 1 --2- 172.21.15.106:0/3718972721 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35cc15b5e0 0x7f35cc148660 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:47.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:47.146+0000 7f35d3a6e640 1 --2- 172.21.15.106:0/3718972721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35cc154ca0 0x7f35cc157800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:47.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:47.146+0000 7f35d3a6e640 1 --2- 172.21.15.106:0/3718972721 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35cc0738d0 0x7f35cc1f0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:47.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:47.146+0000 7f35d3a6e640 1 -- 172.21.15.106:0/3718972721 >> 172.21.15.106:0/3718972721 conn(0x7f35cc06d6d0 msgr2=0x7f35cc154210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:47.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:47.146+0000 7f35d3a6e640 1 -- 172.21.15.106:0/3718972721 shutdown_connections 2024-04-25T10:32:47.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:47.146+0000 7f35d3a6e640 1 -- 172.21.15.106:0/3718972721 wait complete. 2024-04-25T10:32:47.161 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-04-25T10:32:53.556 INFO:tasks.workunit.client.0.smithi106.stdout: 2024-04-25T10:32:53.557 INFO:tasks.workunit.client.0.smithi106.stdout: Children see throughput for 1 initial writers = 111777.80 kB/sec 2024-04-25T10:32:53.557 INFO:tasks.workunit.client.0.smithi106.stdout: Parent sees throughput for 1 initial writers = 111775.70 kB/sec 2024-04-25T10:32:53.557 INFO:tasks.workunit.client.0.smithi106.stdout: Min throughput per process = 111777.80 kB/sec 2024-04-25T10:32:53.557 INFO:tasks.workunit.client.0.smithi106.stdout: Max throughput per process = 111777.80 kB/sec 2024-04-25T10:32:53.557 INFO:tasks.workunit.client.0.smithi106.stdout: Avg throughput per process = 111777.80 kB/sec 2024-04-25T10:32:53.557 INFO:tasks.workunit.client.0.smithi106.stdout: Min xfer = 1048576.00 kB 2024-04-25T10:32:57.162 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-04-25T10:32:57.163 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T10:32:57.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.282+0000 7f6caeba8640 1 Processor -- start 2024-04-25T10:32:57.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.282+0000 7f6caeba8640 1 -- start start 2024-04-25T10:32:57.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.283+0000 7f6caeba8640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6ca8074910 0x7f6ca80c9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:57.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.283+0000 7f6caeba8640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f6ca8157080 0x7f6ca815cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:57.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.283+0000 7f6caeba8640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ca8070af0 con 0x7f6ca80684e0 2024-04-25T10:32:57.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.283+0000 7f6caeba8640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f6ca80733a0 con 0x7f6ca8074910 2024-04-25T10:32:57.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.283+0000 7f6caeba8640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f6ca80708a0 con 0x7f6ca8157080 2024-04-25T10:32:57.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.283+0000 7f6cac944640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6ca8074910 0x7f6ca80c9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:57.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.283+0000 7f6cad145640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f6ca80684e0 0x7f6ca80c9780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:44876/0 (socket says 172.21.15.106:44876) 2024-04-25T10:32:57.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.283+0000 7f6cad145640 1 -- 172.21.15.106:0/3272219992 learned_addr learned my addr 172.21.15.106:0/3272219992 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:32:57.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.284+0000 7f6ca7fff640 1 --2- 172.21.15.106:0/3272219992 >> v2:172.21.15.134:3300/0 conn(0x7f6ca8157080 0x7f6ca815cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:57.286 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.284+0000 7f6cac944640 1 -- 172.21.15.106:0/3272219992 >> v2:172.21.15.134:3300/0 conn(0x7f6ca8157080 msgr2=0x7f6ca815cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:57.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.284+0000 7f6cac944640 1 --2- 172.21.15.106:0/3272219992 >> v2:172.21.15.134:3300/0 conn(0x7f6ca8157080 0x7f6ca815cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:57.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.284+0000 7f6cac944640 1 -- 172.21.15.106:0/3272219992 >> v1:172.21.15.106:6789/0 conn(0x7f6ca80684e0 legacy=0x7f6ca80c9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:57.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.284+0000 7f6cac944640 1 -- 172.21.15.106:0/3272219992 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ca8070dd0 con 0x7f6ca8074910 2024-04-25T10:32:57.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.284+0000 7f6ca77fe640 1 -- 172.21.15.106:0/3272219992 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2777592496 0 0) 0x7f6ca8070af0 con 0x7f6ca80684e0 2024-04-25T10:32:57.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.284+0000 7f6cac944640 1 --2- 172.21.15.106:0/3272219992 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ca8074910 0x7f6ca80c9100 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6ca80cf3e0 tx=0x7f6c9c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93daf565c8e084b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:57.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.285+0000 7f6ca77fe640 1 -- 172.21.15.106:0/3272219992 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c037070 con 0x7f6ca8074910 2024-04-25T10:32:57.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.285+0000 7f6ca77fe640 1 -- 172.21.15.106:0/3272219992 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c9c032440 con 0x7f6ca8074910 2024-04-25T10:32:57.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.285+0000 7f6ca77fe640 1 -- 172.21.15.106:0/3272219992 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c032810 con 0x7f6ca8074910 2024-04-25T10:32:57.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.285+0000 7f6caeba8640 1 -- 172.21.15.106:0/3272219992 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ca8074910 msgr2=0x7f6ca80c9100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:57.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.285+0000 7f6caeba8640 1 --2- 172.21.15.106:0/3272219992 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ca8074910 0x7f6ca80c9100 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6ca80cf3e0 tx=0x7f6c9c02a1a0 comp rx=0 tx=0).stop 2024-04-25T10:32:57.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.285+0000 7f6caeba8640 1 -- 172.21.15.106:0/3272219992 shutdown_connections 2024-04-25T10:32:57.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.285+0000 7f6caeba8640 1 --2- 172.21.15.106:0/3272219992 >> v2:172.21.15.134:3300/0 conn(0x7f6ca8157080 0x7f6ca815cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:57.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.285+0000 7f6caeba8640 1 --2- 172.21.15.106:0/3272219992 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ca8074910 0x7f6ca80c9100 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:57.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.285+0000 7f6caeba8640 1 -- 172.21.15.106:0/3272219992 >> 172.21.15.106:0/3272219992 conn(0x7f6ca80ce570 msgr2=0x7f6ca80ce980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:57.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.285+0000 7f6caeba8640 1 -- 172.21.15.106:0/3272219992 shutdown_connections 2024-04-25T10:32:57.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.285+0000 7f6caeba8640 1 -- 172.21.15.106:0/3272219992 wait complete. 2024-04-25T10:32:57.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.287+0000 7f6caeba8640 1 Processor -- start 2024-04-25T10:32:57.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.287+0000 7f6caeba8640 1 -- start start 2024-04-25T10:32:57.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.289+0000 7f6caeba8640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ca80684e0 0x7f6ca8151da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:57.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.289+0000 7f6caeba8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ca8074910 0x7f6ca81f3340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:57.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.289+0000 7f6caeba8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ca8157080 0x7f6ca8157a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:57.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.289+0000 7f6caeba8640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca8074110 con 0x7f6ca8157080 2024-04-25T10:32:57.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.289+0000 7f6caeba8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca8073ec0 con 0x7f6ca8074910 2024-04-25T10:32:57.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.289+0000 7f6caeba8640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6ca80743f0 con 0x7f6ca80684e0 2024-04-25T10:32:57.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.289+0000 7f6cad145640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ca8157080 0x7f6ca8157a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:57.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.289+0000 7f6cad145640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ca8157080 0x7f6ca8157a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55678/0 (socket says 172.21.15.106:55678) 2024-04-25T10:32:57.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.289+0000 7f6cad145640 1 -- 172.21.15.106:0/2750765894 learned_addr learned my addr 172.21.15.106:0/2750765894 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:32:57.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.289+0000 7f6ca7fff640 1 --2- 172.21.15.106:0/2750765894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ca8074910 0x7f6ca81f3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:57.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.289+0000 7f6cac944640 1 --2- 172.21.15.106:0/2750765894 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ca80684e0 0x7f6ca8151da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:57.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.289+0000 7f6cad145640 1 -- 172.21.15.106:0/2750765894 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ca80684e0 msgr2=0x7f6ca8151da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:57.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.289+0000 7f6cad145640 1 --2- 172.21.15.106:0/2750765894 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ca80684e0 0x7f6ca8151da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:57.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.289+0000 7f6cad145640 1 -- 172.21.15.106:0/2750765894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ca8074910 msgr2=0x7f6ca81f3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:57.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.289+0000 7f6cad145640 1 --2- 172.21.15.106:0/2750765894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ca8074910 0x7f6ca81f3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:57.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.289+0000 7f6cad145640 1 -- 172.21.15.106:0/2750765894 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ca81f3880 con 0x7f6ca8157080 2024-04-25T10:32:57.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.289+0000 7f6cad145640 1 --2- 172.21.15.106:0/2750765894 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ca8157080 0x7f6ca8157a80 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6ca000c730 tx=0x7f6ca000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:57.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.289+0000 7f6ca57fa640 1 -- 172.21.15.106:0/2750765894 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca0018070 con 0x7f6ca8157080 2024-04-25T10:32:57.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.289+0000 7f6ca57fa640 1 -- 172.21.15.106:0/2750765894 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ca0004d60 con 0x7f6ca8157080 2024-04-25T10:32:57.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.289+0000 7f6caeba8640 1 -- 172.21.15.106:0/2750765894 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ca81f2b80 con 0x7f6ca8157080 2024-04-25T10:32:57.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.289+0000 7f6ca57fa640 1 -- 172.21.15.106:0/2750765894 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca000a820 con 0x7f6ca8157080 2024-04-25T10:32:57.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.290+0000 7f6caeba8640 1 -- 172.21.15.106:0/2750765894 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ca8160cb0 con 0x7f6ca8157080 2024-04-25T10:32:57.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.290+0000 7f6caeba8640 1 -- 172.21.15.106:0/2750765894 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ca81f2970 con 0x7f6ca8157080 2024-04-25T10:32:57.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.291+0000 7f6ca57fa640 1 -- 172.21.15.106:0/2750765894 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6ca001b070 con 0x7f6ca8157080 2024-04-25T10:32:57.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.292+0000 7f6ca57fa640 1 --2- 172.21.15.106:0/2750765894 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f6c8c0744b0 0x7f6c8c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:57.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.292+0000 7f6ca57fa640 1 -- 172.21.15.106:0/2750765894 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f6ca0094830 con 0x7f6ca8157080 2024-04-25T10:32:57.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.292+0000 7f6cac944640 1 --2- 172.21.15.106:0/2750765894 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f6c8c0744b0 0x7f6c8c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:57.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.292+0000 7f6cac944640 1 --2- 172.21.15.106:0/2750765894 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f6c8c0744b0 0x7f6c8c0769c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c034000 tx=0x7f6c9c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:57.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.296+0000 7f6ca57fa640 1 -- 172.21.15.106:0/2750765894 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6ca005f060 con 0x7f6ca8157080 2024-04-25T10:32:57.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.538+0000 7f6caeba8640 1 -- 172.21.15.106:0/2750765894 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f6ca806a3d0 con 0x7f6ca8157080 2024-04-25T10:32:57.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.539+0000 7f6ca57fa640 1 -- 172.21.15.106:0/2750765894 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1569 (secure 0 0 0) 0x7f6ca005e960 con 0x7f6ca8157080 2024-04-25T10:32:57.541 INFO:teuthology.orchestra.run.smithi106.stdout:{"election_epoch":16,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"3ee77985-e8e4-41cf-81e9-7c774cf7891f","modified":"2024-04-25T10:29:33.446228Z","created":"2024-04-25T10:29:33.446228Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:3300","nonce":0},{"type":"v1","addr":"172.21.15.106:6789","nonce":0}]},"addr":"172.21.15.106:6789/0","public_addr":"172.21.15.106:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T10:32:57.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.541+0000 7f6caeba8640 1 -- 172.21.15.106:0/2750765894 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f6c8c0744b0 msgr2=0x7f6c8c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:57.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.541+0000 7f6caeba8640 1 --2- 172.21.15.106:0/2750765894 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f6c8c0744b0 0x7f6c8c0769c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c034000 tx=0x7f6c9c02e000 comp rx=0 tx=0).stop 2024-04-25T10:32:57.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.541+0000 7f6caeba8640 1 -- 172.21.15.106:0/2750765894 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ca8157080 msgr2=0x7f6ca8157a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:57.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.541+0000 7f6caeba8640 1 --2- 172.21.15.106:0/2750765894 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ca8157080 0x7f6ca8157a80 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6ca000c730 tx=0x7f6ca000cbf0 comp rx=0 tx=0).stop 2024-04-25T10:32:57.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.541+0000 7f6caeba8640 1 -- 172.21.15.106:0/2750765894 shutdown_connections 2024-04-25T10:32:57.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.541+0000 7f6caeba8640 1 --2- 172.21.15.106:0/2750765894 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f6c8c0744b0 0x7f6c8c0769c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:57.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.541+0000 7f6caeba8640 1 --2- 172.21.15.106:0/2750765894 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ca8157080 0x7f6ca8157a80 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:57.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.541+0000 7f6caeba8640 1 --2- 172.21.15.106:0/2750765894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ca8074910 0x7f6ca81f3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:57.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.541+0000 7f6caeba8640 1 --2- 172.21.15.106:0/2750765894 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ca80684e0 0x7f6ca8151da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:57.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.541+0000 7f6caeba8640 1 -- 172.21.15.106:0/2750765894 >> 172.21.15.106:0/2750765894 conn(0x7f6ca80ce570 msgr2=0x7f6ca80cb7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:57.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.541+0000 7f6caeba8640 1 -- 172.21.15.106:0/2750765894 shutdown_connections 2024-04-25T10:32:57.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.541+0000 7f6caeba8640 1 -- 172.21.15.106:0/2750765894 wait complete. 2024-04-25T10:32:57.554 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-04-25T10:32:57.554 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-04-25T10:32:57.554 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-04-25T10:32:57.554 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-25T10:32:57.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.661+0000 7f1a2eca1640 1 Processor -- start 2024-04-25T10:32:57.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.661+0000 7f1a2eca1640 1 -- start start 2024-04-25T10:32:57.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.662+0000 7f1a2eca1640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f1a28073020 0x7f1a2806f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:57.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.662+0000 7f1a2eca1640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a28068ad0 con 0x7f1a28068600 2024-04-25T10:32:57.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.662+0000 7f1a2eca1640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a280c9010 con 0x7f1a28073db0 2024-04-25T10:32:57.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.662+0000 7f1a2eca1640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f1a28071190 con 0x7f1a28073020 2024-04-25T10:32:57.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.663+0000 7f1a2ca3d640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f1a28068600 0x7f1a28070a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:44892/0 (socket says 172.21.15.106:44892) 2024-04-25T10:32:57.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.663+0000 7f1a2ca3d640 1 -- 172.21.15.106:0/1338911124 learned_addr learned my addr 172.21.15.106:0/1338911124 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:32:57.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.663+0000 7f1a27fff640 1 --2- 172.21.15.106:0/1338911124 >> v2:172.21.15.134:3300/0 conn(0x7f1a28073020 0x7f1a2806f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:57.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.663+0000 7f1a277fe640 1 -- 172.21.15.106:0/1338911124 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2625953755 0 0) 0x7f1a28068ad0 con 0x7f1a28068600 2024-04-25T10:32:57.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.663+0000 7f1a277fe640 1 -- 172.21.15.106:0/1338911124 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1a10003520 con 0x7f1a28068600 2024-04-25T10:32:57.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.663+0000 7f1a277fe640 1 -- 172.21.15.106:0/1338911124 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 517937161 0 0) 0x7f1a10003520 con 0x7f1a28068600 2024-04-25T10:32:57.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.663+0000 7f1a277fe640 1 -- 172.21.15.106:0/1338911124 >> v2:172.21.15.134:3300/0 conn(0x7f1a28073020 msgr2=0x7f1a2806f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:57.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.663+0000 7f1a277fe640 1 --2- 172.21.15.106:0/1338911124 >> v2:172.21.15.134:3300/0 conn(0x7f1a28073020 0x7f1a2806f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:57.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.663+0000 7f1a277fe640 1 -- 172.21.15.106:0/1338911124 >> v1:172.21.15.134:6789/0 conn(0x7f1a28073db0 legacy=0x7f1a280741c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:57.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.663+0000 7f1a277fe640 1 -- 172.21.15.106:0/1338911124 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a280c92f0 con 0x7f1a28068600 2024-04-25T10:32:57.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.663+0000 7f1a277fe640 1 -- 172.21.15.106:0/1338911124 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3776459350 0 0) 0x7f1a18001670 con 0x7f1a28068600 2024-04-25T10:32:57.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.663+0000 7f1a277fe640 1 -- 172.21.15.106:0/1338911124 >> v1:172.21.15.106:6789/0 conn(0x7f1a28068600 legacy=0x7f1a28070a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:57.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.663+0000 7f1a277fe640 1 --2- 172.21.15.106:0/1338911124 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1a100039b0 0x7f1a10003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:57.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.663+0000 7f1a277fe640 1 --2- 172.21.15.106:0/1338911124 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1a10004800 0x7f1a10004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:57.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.663+0000 7f1a277fe640 1 --2- 172.21.15.106:0/1338911124 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a100051c0 0x7f1a1000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:57.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.663+0000 7f1a277fe640 1 -- 172.21.15.106:0/1338911124 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1a28071190 con 0x7f1a100051c0 2024-04-25T10:32:57.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.663+0000 7f1a277fe640 1 -- 172.21.15.106:0/1338911124 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1a1000c160 con 0x7f1a100039b0 2024-04-25T10:32:57.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.663+0000 7f1a277fe640 1 -- 172.21.15.106:0/1338911124 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f1a1000c360 con 0x7f1a10004800 2024-04-25T10:32:57.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.663+0000 7f1a2d23e640 1 --2- 172.21.15.106:0/1338911124 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a100051c0 0x7f1a1000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:57.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.663+0000 7f1a2ca3d640 1 --2- 172.21.15.106:0/1338911124 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1a100039b0 0x7f1a10003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:57.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.663+0000 7f1a27fff640 1 --2- 172.21.15.106:0/1338911124 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1a10004800 0x7f1a10004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:57.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.664+0000 7f1a2d23e640 1 -- 172.21.15.106:0/1338911124 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1a10004800 msgr2=0x7f1a10004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:57.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.664+0000 7f1a2d23e640 1 --2- 172.21.15.106:0/1338911124 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1a10004800 0x7f1a10004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:57.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.664+0000 7f1a2d23e640 1 -- 172.21.15.106:0/1338911124 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1a100039b0 msgr2=0x7f1a10003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:57.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.664+0000 7f1a2d23e640 1 --2- 172.21.15.106:0/1338911124 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1a100039b0 0x7f1a10003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:57.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.664+0000 7f1a2d23e640 1 -- 172.21.15.106:0/1338911124 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a10003520 con 0x7f1a100051c0 2024-04-25T10:32:57.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.664+0000 7f1a2d23e640 1 -- 172.21.15.106:0/1338911124 reap_dead start 2024-04-25T10:32:57.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.664+0000 7f1a2d23e640 1 --2- 172.21.15.106:0/1338911124 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a100051c0 0x7f1a1000b9c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1a20001e20 tx=0x7f1a10005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f00935fd994e1c21 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:57.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.664+0000 7f1a277fe640 1 -- 172.21.15.106:0/1338911124 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a20001a70 con 0x7f1a100051c0 2024-04-25T10:32:57.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.664+0000 7f1a277fe640 1 -- 172.21.15.106:0/1338911124 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a2002ad70 con 0x7f1a100051c0 2024-04-25T10:32:57.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.664+0000 7f1a277fe640 1 -- 172.21.15.106:0/1338911124 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a20003b90 con 0x7f1a100051c0 2024-04-25T10:32:57.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.665+0000 7f1a2eca1640 1 -- 172.21.15.106:0/1338911124 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a100051c0 msgr2=0x7f1a1000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:57.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.665+0000 7f1a2eca1640 1 --2- 172.21.15.106:0/1338911124 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a100051c0 0x7f1a1000b9c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1a20001e20 tx=0x7f1a10005830 comp rx=0 tx=0).stop 2024-04-25T10:32:57.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.665+0000 7f1a2eca1640 1 -- 172.21.15.106:0/1338911124 shutdown_connections 2024-04-25T10:32:57.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.665+0000 7f1a2eca1640 1 --2- 172.21.15.106:0/1338911124 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a100051c0 0x7f1a1000b9c0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:57.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.665+0000 7f1a2eca1640 1 -- 172.21.15.106:0/1338911124 >> 172.21.15.106:0/1338911124 conn(0x7f1a280ce570 msgr2=0x7f1a280ce980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:57.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.665+0000 7f1a2eca1640 1 -- 172.21.15.106:0/1338911124 shutdown_connections 2024-04-25T10:32:57.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.665+0000 7f1a2eca1640 1 -- 172.21.15.106:0/1338911124 wait complete. 2024-04-25T10:32:57.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.667+0000 7f1a2eca1640 1 Processor -- start 2024-04-25T10:32:57.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.667+0000 7f1a2eca1640 1 -- start start 2024-04-25T10:32:57.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.668+0000 7f1a2eca1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1a100051c0 0x7f1a280c66c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:57.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.668+0000 7f1a2eca1640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1a28155560 0x7f1a2814a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:57.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.668+0000 7f1a2eca1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a281f40a0 0x7f1a2814bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:57.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.668+0000 7f1a2eca1640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1a2806f780 con 0x7f1a281f40a0 2024-04-25T10:32:57.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.668+0000 7f1a2eca1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1a280c9ce0 con 0x7f1a100051c0 2024-04-25T10:32:57.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.668+0000 7f1a2eca1640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f1a2806f900 con 0x7f1a28155560 2024-04-25T10:32:57.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.668+0000 7f1a2d23e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a281f40a0 0x7f1a2814bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:57.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.668+0000 7f1a2d23e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a281f40a0 0x7f1a2814bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55698/0 (socket says 172.21.15.106:55698) 2024-04-25T10:32:57.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.668+0000 7f1a2d23e640 1 -- 172.21.15.106:0/1421762116 learned_addr learned my addr 172.21.15.106:0/1421762116 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:32:57.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.668+0000 7f1a2ca3d640 1 --2- 172.21.15.106:0/1421762116 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1a100051c0 0x7f1a280c66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:57.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.668+0000 7f1a27fff640 1 --2- 172.21.15.106:0/1421762116 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1a28155560 0x7f1a2814a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:57.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.669+0000 7f1a2d23e640 1 -- 172.21.15.106:0/1421762116 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1a28155560 msgr2=0x7f1a2814a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:57.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.669+0000 7f1a2d23e640 1 --2- 172.21.15.106:0/1421762116 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1a28155560 0x7f1a2814a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:57.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.669+0000 7f1a2d23e640 1 -- 172.21.15.106:0/1421762116 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1a100051c0 msgr2=0x7f1a280c66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:57.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.669+0000 7f1a2d23e640 1 --2- 172.21.15.106:0/1421762116 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1a100051c0 0x7f1a280c66c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:57.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.669+0000 7f1a2d23e640 1 -- 172.21.15.106:0/1421762116 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a280c6c00 con 0x7f1a281f40a0 2024-04-25T10:32:57.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.669+0000 7f1a2d23e640 1 --2- 172.21.15.106:0/1421762116 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a281f40a0 0x7f1a2814bba0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f1a20001e80 tx=0x7f1a200014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:57.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.669+0000 7f1a257fa640 1 -- 172.21.15.106:0/1421762116 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a20007db0 con 0x7f1a281f40a0 2024-04-25T10:32:57.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.669+0000 7f1a257fa640 1 -- 172.21.15.106:0/1421762116 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a2002ab70 con 0x7f1a281f40a0 2024-04-25T10:32:57.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.669+0000 7f1a257fa640 1 -- 172.21.15.106:0/1421762116 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a20005810 con 0x7f1a281f40a0 2024-04-25T10:32:57.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.669+0000 7f1a2eca1640 1 -- 172.21.15.106:0/1421762116 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a281f12c0 con 0x7f1a281f40a0 2024-04-25T10:32:57.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.669+0000 7f1a2eca1640 1 -- 172.21.15.106:0/1421762116 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a281f1880 con 0x7f1a281f40a0 2024-04-25T10:32:57.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.670+0000 7f1a2eca1640 1 --2- 172.21.15.106:0/1421762116 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a280c6dd0 0x7f1a2814c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:57.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.670+0000 7f1a2eca1640 1 -- 172.21.15.106:0/1421762116 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1a281f1010 con 0x7f1a280c6dd0 2024-04-25T10:32:57.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.670+0000 7f1a2ca3d640 1 --2- 172.21.15.106:0/1421762116 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a280c6dd0 0x7f1a2814c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:57.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.670+0000 7f1a2ca3d640 1 -- 172.21.15.106:0/1421762116 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1a2806a3d0 con 0x7f1a280c6dd0 2024-04-25T10:32:57.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.670+0000 7f1a2ca3d640 1 --2- 172.21.15.106:0/1421762116 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a280c6dd0 0x7f1a2814c0e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f1a180017e0 tx=0x7f1a18007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:57.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.670+0000 7f1a257fa640 1 -- 172.21.15.106:0/1421762116 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a18012a30 con 0x7f1a280c6dd0 2024-04-25T10:32:57.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.671+0000 7f1a257fa640 1 -- 172.21.15.106:0/1421762116 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1a20032440 con 0x7f1a281f40a0 2024-04-25T10:32:57.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.671+0000 7f1a257fa640 1 --2- 172.21.15.106:0/1421762116 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f1a140743e0 0x7f1a140768f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:57.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.671+0000 7f1a257fa640 1 -- 172.21.15.106:0/1421762116 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f1a200b4780 con 0x7f1a281f40a0 2024-04-25T10:32:57.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.671+0000 7f1a27fff640 1 --2- 172.21.15.106:0/1421762116 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f1a140743e0 0x7f1a140768f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:57.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.671+0000 7f1a257fa640 1 -- 172.21.15.106:0/1421762116 <== mon.0 v2:172.21.15.106:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f1a1801a070 con 0x7f1a280c6dd0 2024-04-25T10:32:57.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.671+0000 7f1a257fa640 1 -- 172.21.15.106:0/1421762116 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a280c6dd0 msgr2=0x7f1a2814c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:57.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.671+0000 7f1a257fa640 1 --2- 172.21.15.106:0/1421762116 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a280c6dd0 0x7f1a2814c0e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f1a180017e0 tx=0x7f1a18007780 comp rx=0 tx=0).stop 2024-04-25T10:32:57.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.671+0000 7f1a257fa640 1 -- 172.21.15.106:0/1421762116 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a280c6dd0 msgr2=0x7f1a2814c0e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T10:32:57.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.671+0000 7f1a257fa640 1 --2- 172.21.15.106:0/1421762116 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a280c6dd0 0x7f1a2814c0e0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:57.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.672+0000 7f1a27fff640 1 --2- 172.21.15.106:0/1421762116 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f1a140743e0 0x7f1a140768f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1a280c8180 tx=0x7f1a1c009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:57.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.678+0000 7f1a2eca1640 1 --2- 172.21.15.106:0/1421762116 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a280c6000 0x7f1a2814cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:57.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.678+0000 7f1a2eca1640 1 -- 172.21.15.106:0/1421762116 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1a281f1a50 con 0x7f1a280c6000 2024-04-25T10:32:57.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.678+0000 7f1a2ca3d640 1 --2- 172.21.15.106:0/1421762116 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a280c6000 0x7f1a2814cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:57.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.678+0000 7f1a2ca3d640 1 -- 172.21.15.106:0/1421762116 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f1a2815ee40 con 0x7f1a280c6000 2024-04-25T10:32:57.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.678+0000 7f1a2ca3d640 1 --2- 172.21.15.106:0/1421762116 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a280c6000 0x7f1a2814cab0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1a180017e0 tx=0x7f1a180078f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:57.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.678+0000 7f1a257fa640 1 -- 172.21.15.106:0/1421762116 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a18015920 con 0x7f1a280c6000 2024-04-25T10:32:57.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.679+0000 7f1a257fa640 1 -- 172.21.15.106:0/1421762116 <== mon.0 v2:172.21.15.106:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5072 (secure 0 0 0) 0x7f1a18015920 con 0x7f1a280c6000 2024-04-25T10:32:57.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.679+0000 7f1a257fa640 1 -- 172.21.15.106:0/1421762116 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a280c6000 msgr2=0x7f1a2814cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:57.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.679+0000 7f1a257fa640 1 --2- 172.21.15.106:0/1421762116 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a280c6000 0x7f1a2814cab0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1a180017e0 tx=0x7f1a180078f0 comp rx=0 tx=0).stop 2024-04-25T10:32:57.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.679+0000 7f1a257fa640 1 -- 172.21.15.106:0/1421762116 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a280c6000 msgr2=0x7f1a2814cab0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T10:32:57.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.679+0000 7f1a257fa640 1 --2- 172.21.15.106:0/1421762116 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a280c6000 0x7f1a2814cab0 secure :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1a180017e0 tx=0x7f1a180078f0 comp rx=0 tx=0).stop 2024-04-25T10:32:57.682 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-25T10:32:57.682 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "a", 2024-04-25T10:32:57.682 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 0, 2024-04-25T10:32:57.682 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "leader", 2024-04-25T10:32:57.682 INFO:teuthology.orchestra.run.smithi106.stdout: "election_epoch": 16, 2024-04-25T10:32:57.682 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 169918, 2024-04-25T10:32:57.683 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum": [ 2024-04-25T10:32:57.683 INFO:teuthology.orchestra.run.smithi106.stdout: 0, 2024-04-25T10:32:57.683 INFO:teuthology.orchestra.run.smithi106.stdout: 1, 2024-04-25T10:32:57.683 INFO:teuthology.orchestra.run.smithi106.stdout: 2 2024-04-25T10:32:57.683 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:32:57.683 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_age": 15, 2024-04-25T10:32:57.683 INFO:teuthology.orchestra.run.smithi106.stdout: "features": { 2024-04-25T10:32:57.683 INFO:teuthology.orchestra.run.smithi106.stdout: "required_con": "2450521708007866372", 2024-04-25T10:32:57.683 INFO:teuthology.orchestra.run.smithi106.stdout: "required_mon": [ 2024-04-25T10:32:57.683 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:32:57.683 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:32:57.683 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:32:57.683 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:32:57.683 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:32:57.684 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:32:57.684 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:32:57.684 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:32:57.684 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:32:57.684 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:32:57.684 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:32:57.684 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:32:57.684 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_con": "4540701547738038271", 2024-04-25T10:32:57.684 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_mon": [ 2024-04-25T10:32:57.684 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:32:57.684 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:32:57.684 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:32:57.684 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:32:57.684 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:32:57.684 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:32:57.684 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:32:57.685 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:32:57.685 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:32:57.685 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:32:57.685 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:32:57.685 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:32:57.685 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:57.685 INFO:teuthology.orchestra.run.smithi106.stdout: "outside_quorum": [], 2024-04-25T10:32:57.685 INFO:teuthology.orchestra.run.smithi106.stdout: "extra_probe_peers": [], 2024-04-25T10:32:57.685 INFO:teuthology.orchestra.run.smithi106.stdout: "sync_provider": [], 2024-04-25T10:32:57.685 INFO:teuthology.orchestra.run.smithi106.stdout: "monmap": { 2024-04-25T10:32:57.685 INFO:teuthology.orchestra.run.smithi106.stdout: "epoch": 1, 2024-04-25T10:32:57.685 INFO:teuthology.orchestra.run.smithi106.stdout: "fsid": "3ee77985-e8e4-41cf-81e9-7c774cf7891f", 2024-04-25T10:32:57.685 INFO:teuthology.orchestra.run.smithi106.stdout: "modified": "2024-04-25T10:29:33.446228Z", 2024-04-25T10:32:57.685 INFO:teuthology.orchestra.run.smithi106.stdout: "created": "2024-04-25T10:29:33.446228Z", 2024-04-25T10:32:57.686 INFO:teuthology.orchestra.run.smithi106.stdout: "min_mon_release": 19, 2024-04-25T10:32:57.686 INFO:teuthology.orchestra.run.smithi106.stdout: "min_mon_release_name": "squid", 2024-04-25T10:32:57.686 INFO:teuthology.orchestra.run.smithi106.stdout: "election_strategy": 1, 2024-04-25T10:32:57.686 INFO:teuthology.orchestra.run.smithi106.stdout: "disallowed_leaders: ": "", 2024-04-25T10:32:57.686 INFO:teuthology.orchestra.run.smithi106.stdout: "stretch_mode": false, 2024-04-25T10:32:57.686 INFO:teuthology.orchestra.run.smithi106.stdout: "tiebreaker_mon": "", 2024-04-25T10:32:57.686 INFO:teuthology.orchestra.run.smithi106.stdout: "removed_ranks: ": "", 2024-04-25T10:32:57.686 INFO:teuthology.orchestra.run.smithi106.stdout: "features": { 2024-04-25T10:32:57.687 INFO:teuthology.orchestra.run.smithi106.stdout: "persistent": [ 2024-04-25T10:32:57.687 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:32:57.687 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:32:57.687 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:32:57.687 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:32:57.687 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:32:57.687 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:32:57.687 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:32:57.687 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:32:57.687 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:32:57.687 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:32:57.687 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:32:57.687 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:32:57.687 INFO:teuthology.orchestra.run.smithi106.stdout: "optional": [] 2024-04-25T10:32:57.687 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:57.688 INFO:teuthology.orchestra.run.smithi106.stdout: "mons": [ 2024-04-25T10:32:57.688 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:57.688 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 0, 2024-04-25T10:32:57.688 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "a", 2024-04-25T10:32:57.688 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:32:57.688 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:32:57.688 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:57.688 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:32:57.688 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:3300", 2024-04-25T10:32:57.688 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:32:57.688 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:57.688 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:57.688 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:32:57.688 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:6789", 2024-04-25T10:32:57.689 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:32:57.689 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:57.689 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:32:57.689 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:57.689 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:6789/0", 2024-04-25T10:32:57.689 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.106:6789/0", 2024-04-25T10:32:57.689 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:32:57.689 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:32:57.689 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:32:57.689 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:57.689 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:57.689 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 1, 2024-04-25T10:32:57.689 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "b", 2024-04-25T10:32:57.689 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:32:57.690 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:32:57.690 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:57.690 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:32:57.690 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:3300", 2024-04-25T10:32:57.690 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:32:57.690 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:57.690 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:57.690 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:32:57.690 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6789", 2024-04-25T10:32:57.690 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:32:57.690 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:57.690 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:32:57.690 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:57.691 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6789/0", 2024-04-25T10:32:57.691 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.134:6789/0", 2024-04-25T10:32:57.691 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:32:57.691 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:32:57.691 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:32:57.691 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:57.691 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:57.691 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 2, 2024-04-25T10:32:57.691 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "c", 2024-04-25T10:32:57.691 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:32:57.691 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:32:57.691 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:57.691 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:32:57.692 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:3301", 2024-04-25T10:32:57.692 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:32:57.692 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:57.692 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:57.692 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:32:57.692 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6790", 2024-04-25T10:32:57.692 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:32:57.692 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:57.692 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:32:57.692 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:57.692 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6790/0", 2024-04-25T10:32:57.692 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.134:6790/0", 2024-04-25T10:32:57.692 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:32:57.693 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:32:57.693 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:32:57.693 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:57.693 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:32:57.693 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:57.693 INFO:teuthology.orchestra.run.smithi106.stdout: "feature_map": { 2024-04-25T10:32:57.693 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": [ 2024-04-25T10:32:57.693 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:57.693 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:32:57.693 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:32:57.693 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 1 2024-04-25T10:32:57.693 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:57.693 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:32:57.694 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": [ 2024-04-25T10:32:57.694 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:57.694 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:32:57.694 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:32:57.694 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 6 2024-04-25T10:32:57.694 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:57.694 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:32:57.694 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": [ 2024-04-25T10:32:57.694 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:57.694 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:32:57.694 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:32:57.694 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 4 2024-04-25T10:32:57.694 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:57.695 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:32:57.695 INFO:teuthology.orchestra.run.smithi106.stdout: "client": [ 2024-04-25T10:32:57.695 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:57.695 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:32:57.695 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:32:57.695 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 7 2024-04-25T10:32:57.695 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:57.695 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:32:57.695 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": [ 2024-04-25T10:32:57.695 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:57.695 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:32:57.695 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:32:57.695 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 2 2024-04-25T10:32:57.695 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:57.696 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:32:57.696 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:57.696 INFO:teuthology.orchestra.run.smithi106.stdout: "stretch_mode": false 2024-04-25T10:32:57.696 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-25T10:32:57.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.679+0000 7f1a2eca1640 1 -- 172.21.15.106:0/1421762116 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f1a140743e0 msgr2=0x7f1a140768f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:57.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.679+0000 7f1a2eca1640 1 --2- 172.21.15.106:0/1421762116 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f1a140743e0 0x7f1a140768f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1a280c8180 tx=0x7f1a1c009190 comp rx=0 tx=0).stop 2024-04-25T10:32:57.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.679+0000 7f1a2eca1640 1 -- 172.21.15.106:0/1421762116 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a281f40a0 msgr2=0x7f1a2814bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:57.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.679+0000 7f1a2d23e640 1 -- 172.21.15.106:0/1421762116 reap_dead start 2024-04-25T10:32:57.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.679+0000 7f1a2eca1640 1 --2- 172.21.15.106:0/1421762116 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a281f40a0 0x7f1a2814bba0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f1a20001e80 tx=0x7f1a200014a0 comp rx=0 tx=0).stop 2024-04-25T10:32:57.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.679+0000 7f1a2eca1640 1 -- 172.21.15.106:0/1421762116 shutdown_connections 2024-04-25T10:32:57.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.679+0000 7f1a2eca1640 1 --2- 172.21.15.106:0/1421762116 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a281f40a0 0x7f1a2814bba0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:57.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.679+0000 7f1a2eca1640 1 -- 172.21.15.106:0/1421762116 >> 172.21.15.106:0/1421762116 conn(0x7f1a280ce570 msgr2=0x7f1a28070a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:57.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.679+0000 7f1a2eca1640 1 -- 172.21.15.106:0/1421762116 shutdown_connections 2024-04-25T10:32:57.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.679+0000 7f1a2eca1640 1 -- 172.21.15.106:0/1421762116 wait complete. 2024-04-25T10:32:57.698 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T10:32:57.699 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-04-25T10:32:57.699 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-25T10:32:57.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.804+0000 7eff8f996640 1 Processor -- start 2024-04-25T10:32:57.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.804+0000 7eff8f996640 1 -- start start 2024-04-25T10:32:57.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.806+0000 7eff8f996640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7eff880cc600 0x7eff880c5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:57.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.806+0000 7eff8f996640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7eff880c58d0 0x7eff880cce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:57.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.806+0000 7eff8f996640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7eff880cb2b0 0x7eff880cd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:57.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.806+0000 7eff8f996640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7eff8815eea0 con 0x7eff880cb2b0 2024-04-25T10:32:57.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.806+0000 7eff8f996640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7eff8815b8c0 con 0x7eff880c58d0 2024-04-25T10:32:57.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.806+0000 7eff8f996640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7eff8815b5e0 con 0x7eff880cc600 2024-04-25T10:32:57.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.806+0000 7eff8df33640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7eff880cb2b0 0x7eff880cd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:57.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.806+0000 7eff8df33640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7eff880cb2b0 0x7eff880cd3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55718/0 (socket says 172.21.15.106:55718) 2024-04-25T10:32:57.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.806+0000 7eff8df33640 1 -- 172.21.15.106:0/2987841393 learned_addr learned my addr 172.21.15.106:0/2987841393 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:32:57.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.806+0000 7eff8df33640 1 -- 172.21.15.106:0/2987841393 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7eff880cc600 msgr2=0x7eff880c5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:57.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.806+0000 7eff8d732640 1 --2- 172.21.15.106:0/2987841393 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7eff880cc600 0x7eff880c5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:57.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.806+0000 7eff8cf31640 1 --2- 172.21.15.106:0/2987841393 >> v2:172.21.15.134:3300/0 conn(0x7eff880c58d0 0x7eff880cce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:57.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.806+0000 7eff8df33640 1 --2- 172.21.15.106:0/2987841393 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7eff880cc600 0x7eff880c5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:57.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.806+0000 7eff8df33640 1 -- 172.21.15.106:0/2987841393 >> v2:172.21.15.134:3300/0 conn(0x7eff880c58d0 msgr2=0x7eff880cce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:57.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.806+0000 7eff8df33640 1 --2- 172.21.15.106:0/2987841393 >> v2:172.21.15.134:3300/0 conn(0x7eff880c58d0 0x7eff880cce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:57.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.806+0000 7eff8df33640 1 -- 172.21.15.106:0/2987841393 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff8815e0e0 con 0x7eff880cb2b0 2024-04-25T10:32:57.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.807+0000 7eff8df33640 1 --2- 172.21.15.106:0/2987841393 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff880cb2b0 0x7eff880cd3d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7eff88073550 tx=0x7eff8002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a422df814a8e6ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:57.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.807+0000 7eff7ffff640 1 -- 172.21.15.106:0/2987841393 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff80037070 con 0x7eff880cb2b0 2024-04-25T10:32:57.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.807+0000 7eff7ffff640 1 -- 172.21.15.106:0/2987841393 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff80032440 con 0x7eff880cb2b0 2024-04-25T10:32:57.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.807+0000 7eff7ffff640 1 -- 172.21.15.106:0/2987841393 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff80032810 con 0x7eff880cb2b0 2024-04-25T10:32:57.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.808+0000 7eff8f996640 1 -- 172.21.15.106:0/2987841393 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff880cb2b0 msgr2=0x7eff880cd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:57.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.808+0000 7eff8f996640 1 --2- 172.21.15.106:0/2987841393 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff880cb2b0 0x7eff880cd3d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7eff88073550 tx=0x7eff8002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:32:57.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.808+0000 7eff8f996640 1 -- 172.21.15.106:0/2987841393 shutdown_connections 2024-04-25T10:32:57.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.808+0000 7eff8f996640 1 --2- 172.21.15.106:0/2987841393 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff880cb2b0 0x7eff880cd3d0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:57.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.808+0000 7eff8f996640 1 --2- 172.21.15.106:0/2987841393 >> v2:172.21.15.134:3300/0 conn(0x7eff880c58d0 0x7eff880cce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:57.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.808+0000 7eff8f996640 1 --2- 172.21.15.106:0/2987841393 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7eff880cc600 0x7eff880c5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:57.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.808+0000 7eff8f996640 1 -- 172.21.15.106:0/2987841393 >> 172.21.15.106:0/2987841393 conn(0x7eff880727d0 msgr2=0x7eff88072c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:57.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.808+0000 7eff8f996640 1 -- 172.21.15.106:0/2987841393 shutdown_connections 2024-04-25T10:32:57.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.808+0000 7eff8f996640 1 -- 172.21.15.106:0/2987841393 wait complete. 2024-04-25T10:32:57.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.809+0000 7eff8f996640 1 Processor -- start 2024-04-25T10:32:57.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.809+0000 7eff8f996640 1 -- start start 2024-04-25T10:32:57.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.811+0000 7eff8f996640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7eff880c58d0 0x7eff88157f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:57.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.811+0000 7eff8f996640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff880cb2b0 0x7eff88159160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:57.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.811+0000 7eff8f996640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff880cc600 0x7eff881f18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:57.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.811+0000 7eff8f996640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7eff8815e980 con 0x7eff880cc600 2024-04-25T10:32:57.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.811+0000 7eff8f996640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7eff8815e6a0 con 0x7eff880cb2b0 2024-04-25T10:32:57.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.811+0000 7eff8f996640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7eff880cbff0 con 0x7eff880c58d0 2024-04-25T10:32:57.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.811+0000 7eff8df33640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff880cc600 0x7eff881f18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:57.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.811+0000 7eff8df33640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff880cc600 0x7eff881f18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55722/0 (socket says 172.21.15.106:55722) 2024-04-25T10:32:57.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.811+0000 7eff8df33640 1 -- 172.21.15.106:0/2391788594 learned_addr learned my addr 172.21.15.106:0/2391788594 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:32:57.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.811+0000 7eff8d732640 1 --2- 172.21.15.106:0/2391788594 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7eff880c58d0 0x7eff88157f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:57.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.811+0000 7eff8cf31640 1 --2- 172.21.15.106:0/2391788594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff880cb2b0 0x7eff88159160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:57.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.811+0000 7eff8df33640 1 -- 172.21.15.106:0/2391788594 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7eff880c58d0 msgr2=0x7eff88157f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:57.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.811+0000 7eff8df33640 1 --2- 172.21.15.106:0/2391788594 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7eff880c58d0 0x7eff88157f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:57.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.811+0000 7eff8df33640 1 -- 172.21.15.106:0/2391788594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff880cb2b0 msgr2=0x7eff88159160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:57.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.811+0000 7eff8df33640 1 --2- 172.21.15.106:0/2391788594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff880cb2b0 0x7eff88159160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:57.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.811+0000 7eff8df33640 1 -- 172.21.15.106:0/2391788594 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff88159e80 con 0x7eff880cc600 2024-04-25T10:32:57.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.811+0000 7eff8d732640 1 --2- 172.21.15.106:0/2391788594 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7eff880c58d0 0x7eff88157f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:32:57.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.811+0000 7eff8df33640 1 --2- 172.21.15.106:0/2391788594 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff880cc600 0x7eff881f18e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7eff800090b0 tx=0x7eff800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:57.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.811+0000 7eff7dffb640 1 -- 172.21.15.106:0/2391788594 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff80037040 con 0x7eff880cc600 2024-04-25T10:32:57.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.811+0000 7eff7dffb640 1 -- 172.21.15.106:0/2391788594 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff80042070 con 0x7eff880cc600 2024-04-25T10:32:57.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.811+0000 7eff7dffb640 1 -- 172.21.15.106:0/2391788594 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff80030bd0 con 0x7eff880cc600 2024-04-25T10:32:57.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.811+0000 7eff8f996640 1 -- 172.21.15.106:0/2391788594 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff8815a420 con 0x7eff880cc600 2024-04-25T10:32:57.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.812+0000 7eff8f996640 1 -- 172.21.15.106:0/2391788594 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff88158680 con 0x7eff880cc600 2024-04-25T10:32:57.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.813+0000 7eff8f996640 1 --2- 172.21.15.106:0/2391788594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff88156b40 0x7eff881588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:57.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.813+0000 7eff8f996640 1 -- 172.21.15.106:0/2391788594 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7eff880b40d0 con 0x7eff88156b40 2024-04-25T10:32:57.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.813+0000 7eff8d732640 1 --2- 172.21.15.106:0/2391788594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff88156b40 0x7eff881588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:57.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.813+0000 7eff8d732640 1 -- 172.21.15.106:0/2391788594 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7eff880b5940 con 0x7eff88156b40 2024-04-25T10:32:57.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.813+0000 7eff7dffb640 1 -- 172.21.15.106:0/2391788594 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7eff80047460 con 0x7eff880cc600 2024-04-25T10:32:57.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.813+0000 7eff8d732640 1 --2- 172.21.15.106:0/2391788594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff88156b40 0x7eff881588c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7eff8400dc60 tx=0x7eff8400b630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:57.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.814+0000 7eff7dffb640 1 --2- 172.21.15.106:0/2391788594 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7eff6c0744b0 0x7eff6c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:57.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.814+0000 7eff8cf31640 1 --2- 172.21.15.106:0/2391788594 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7eff6c0744b0 0x7eff6c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:57.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.814+0000 7eff7dffb640 1 -- 172.21.15.106:0/2391788594 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7eff800b5fc0 con 0x7eff880cc600 2024-04-25T10:32:57.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.814+0000 7eff7dffb640 1 -- 172.21.15.106:0/2391788594 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff84010460 con 0x7eff88156b40 2024-04-25T10:32:57.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.814+0000 7eff7dffb640 1 -- 172.21.15.106:0/2391788594 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7eff84010600 con 0x7eff88156b40 2024-04-25T10:32:57.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.814+0000 7eff7dffb640 1 -- 172.21.15.106:0/2391788594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff88156b40 msgr2=0x7eff881588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:57.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.814+0000 7eff7dffb640 1 --2- 172.21.15.106:0/2391788594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff88156b40 0x7eff881588c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7eff8400dc60 tx=0x7eff8400b630 comp rx=0 tx=0).stop 2024-04-25T10:32:57.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.814+0000 7eff7dffb640 1 -- 172.21.15.106:0/2391788594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff88156b40 msgr2=0x7eff881588c0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T10:32:57.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.814+0000 7eff7dffb640 1 --2- 172.21.15.106:0/2391788594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff88156b40 0x7eff881588c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:57.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.814+0000 7eff8cf31640 1 --2- 172.21.15.106:0/2391788594 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7eff6c0744b0 0x7eff6c0769c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7eff88159c90 tx=0x7eff78008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:57.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.821+0000 7eff8f996640 1 --2- 172.21.15.106:0/2391788594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff880cff50 0x7eff8815c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:57.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.821+0000 7eff8f996640 1 -- 172.21.15.106:0/2391788594 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7eff88158e00 con 0x7eff880cff50 2024-04-25T10:32:57.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.821+0000 7eff8d732640 1 --2- 172.21.15.106:0/2391788594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff880cff50 0x7eff8815c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:57.824 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.821+0000 7eff8d732640 1 -- 172.21.15.106:0/2391788594 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7eff8806a3d0 con 0x7eff880cff50 2024-04-25T10:32:57.824 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.821+0000 7eff8d732640 1 --2- 172.21.15.106:0/2391788594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff880cff50 0x7eff8815c000 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7eff8400dc60 tx=0x7eff84004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:57.824 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.822+0000 7eff7dffb640 1 -- 172.21.15.106:0/2391788594 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff8401ea10 con 0x7eff880cff50 2024-04-25T10:32:57.824 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.822+0000 7eff7dffb640 1 -- 172.21.15.106:0/2391788594 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4730 (secure 0 0 0) 0x7eff84028400 con 0x7eff880cff50 2024-04-25T10:32:57.824 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.822+0000 7eff7dffb640 1 -- 172.21.15.106:0/2391788594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff880cff50 msgr2=0x7eff8815c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:57.824 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.822+0000 7eff7dffb640 1 --2- 172.21.15.106:0/2391788594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff880cff50 0x7eff8815c000 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7eff8400dc60 tx=0x7eff84004a40 comp rx=0 tx=0).stop 2024-04-25T10:32:57.824 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.822+0000 7eff7dffb640 1 -- 172.21.15.106:0/2391788594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff880cff50 msgr2=0x7eff8815c000 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T10:32:57.824 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.822+0000 7eff7dffb640 1 --2- 172.21.15.106:0/2391788594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff880cff50 0x7eff8815c000 secure :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7eff8400dc60 tx=0x7eff84004a40 comp rx=0 tx=0).stop 2024-04-25T10:32:57.825 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-25T10:32:57.825 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "b", 2024-04-25T10:32:57.825 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 1, 2024-04-25T10:32:57.825 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "peon", 2024-04-25T10:32:57.825 INFO:teuthology.orchestra.run.smithi106.stdout: "election_epoch": 16, 2024-04-25T10:32:57.825 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 170017, 2024-04-25T10:32:57.825 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum": [ 2024-04-25T10:32:57.825 INFO:teuthology.orchestra.run.smithi106.stdout: 0, 2024-04-25T10:32:57.825 INFO:teuthology.orchestra.run.smithi106.stdout: 1, 2024-04-25T10:32:57.825 INFO:teuthology.orchestra.run.smithi106.stdout: 2 2024-04-25T10:32:57.825 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:32:57.825 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_age": 16, 2024-04-25T10:32:57.826 INFO:teuthology.orchestra.run.smithi106.stdout: "features": { 2024-04-25T10:32:57.826 INFO:teuthology.orchestra.run.smithi106.stdout: "required_con": "2450521708007866372", 2024-04-25T10:32:57.826 INFO:teuthology.orchestra.run.smithi106.stdout: "required_mon": [ 2024-04-25T10:32:57.826 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:32:57.826 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:32:57.826 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:32:57.826 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:32:57.826 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:32:57.826 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:32:57.826 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:32:57.826 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:32:57.826 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:32:57.827 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:32:57.827 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:32:57.827 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:32:57.827 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_con": "4540701547738038271", 2024-04-25T10:32:57.827 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_mon": [ 2024-04-25T10:32:57.827 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:32:57.827 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:32:57.827 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:32:57.827 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:32:57.827 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:32:57.827 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:32:57.827 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:32:57.827 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:32:57.828 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:32:57.828 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:32:57.828 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:32:57.828 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:32:57.828 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:57.828 INFO:teuthology.orchestra.run.smithi106.stdout: "outside_quorum": [], 2024-04-25T10:32:57.828 INFO:teuthology.orchestra.run.smithi106.stdout: "extra_probe_peers": [], 2024-04-25T10:32:57.828 INFO:teuthology.orchestra.run.smithi106.stdout: "sync_provider": [], 2024-04-25T10:32:57.828 INFO:teuthology.orchestra.run.smithi106.stdout: "monmap": { 2024-04-25T10:32:57.828 INFO:teuthology.orchestra.run.smithi106.stdout: "epoch": 1, 2024-04-25T10:32:57.828 INFO:teuthology.orchestra.run.smithi106.stdout: "fsid": "3ee77985-e8e4-41cf-81e9-7c774cf7891f", 2024-04-25T10:32:57.828 INFO:teuthology.orchestra.run.smithi106.stdout: "modified": "2024-04-25T10:29:33.446228Z", 2024-04-25T10:32:57.828 INFO:teuthology.orchestra.run.smithi106.stdout: "created": "2024-04-25T10:29:33.446228Z", 2024-04-25T10:32:57.828 INFO:teuthology.orchestra.run.smithi106.stdout: "min_mon_release": 19, 2024-04-25T10:32:57.829 INFO:teuthology.orchestra.run.smithi106.stdout: "min_mon_release_name": "squid", 2024-04-25T10:32:57.829 INFO:teuthology.orchestra.run.smithi106.stdout: "election_strategy": 1, 2024-04-25T10:32:57.829 INFO:teuthology.orchestra.run.smithi106.stdout: "disallowed_leaders: ": "", 2024-04-25T10:32:57.829 INFO:teuthology.orchestra.run.smithi106.stdout: "stretch_mode": false, 2024-04-25T10:32:57.829 INFO:teuthology.orchestra.run.smithi106.stdout: "tiebreaker_mon": "", 2024-04-25T10:32:57.829 INFO:teuthology.orchestra.run.smithi106.stdout: "removed_ranks: ": "", 2024-04-25T10:32:57.829 INFO:teuthology.orchestra.run.smithi106.stdout: "features": { 2024-04-25T10:32:57.829 INFO:teuthology.orchestra.run.smithi106.stdout: "persistent": [ 2024-04-25T10:32:57.829 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:32:57.829 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:32:57.829 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:32:57.829 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:32:57.829 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:32:57.829 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:32:57.830 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:32:57.830 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:32:57.830 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:32:57.830 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:32:57.830 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:32:57.830 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:32:57.830 INFO:teuthology.orchestra.run.smithi106.stdout: "optional": [] 2024-04-25T10:32:57.830 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:57.830 INFO:teuthology.orchestra.run.smithi106.stdout: "mons": [ 2024-04-25T10:32:57.830 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:57.830 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 0, 2024-04-25T10:32:57.830 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "a", 2024-04-25T10:32:57.830 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:32:57.830 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:32:57.831 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:57.831 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:32:57.831 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:3300", 2024-04-25T10:32:57.831 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:32:57.831 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:57.831 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:57.831 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:32:57.831 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:6789", 2024-04-25T10:32:57.831 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:32:57.831 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:57.831 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:32:57.831 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:57.832 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:6789/0", 2024-04-25T10:32:57.832 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.106:6789/0", 2024-04-25T10:32:57.832 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:32:57.832 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:32:57.832 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:32:57.832 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:57.832 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:57.832 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 1, 2024-04-25T10:32:57.832 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "b", 2024-04-25T10:32:57.832 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:32:57.832 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:32:57.832 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:57.832 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:32:57.832 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:3300", 2024-04-25T10:32:57.833 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:32:57.833 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:57.833 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:57.833 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:32:57.833 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6789", 2024-04-25T10:32:57.833 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:32:57.833 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:57.833 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:32:57.833 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:57.833 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6789/0", 2024-04-25T10:32:57.833 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.134:6789/0", 2024-04-25T10:32:57.833 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:32:57.833 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:32:57.833 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:32:57.834 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:57.834 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:57.834 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 2, 2024-04-25T10:32:57.834 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "c", 2024-04-25T10:32:57.834 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:32:57.834 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:32:57.834 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:57.834 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:32:57.834 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:3301", 2024-04-25T10:32:57.834 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:32:57.834 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:57.834 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:57.834 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:32:57.835 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6790", 2024-04-25T10:32:57.835 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:32:57.835 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:57.835 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:32:57.835 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:57.835 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6790/0", 2024-04-25T10:32:57.835 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.134:6790/0", 2024-04-25T10:32:57.835 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:32:57.835 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:32:57.835 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:32:57.835 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:57.835 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:32:57.835 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:57.835 INFO:teuthology.orchestra.run.smithi106.stdout: "feature_map": { 2024-04-25T10:32:57.835 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": [ 2024-04-25T10:32:57.835 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:57.836 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:32:57.836 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:32:57.836 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 1 2024-04-25T10:32:57.836 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:57.836 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:32:57.836 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": [ 2024-04-25T10:32:57.836 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:57.836 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:32:57.836 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:32:57.836 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 4 2024-04-25T10:32:57.836 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:57.836 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:32:57.836 INFO:teuthology.orchestra.run.smithi106.stdout: "client": [ 2024-04-25T10:32:57.836 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:57.836 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:32:57.837 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:32:57.837 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 1 2024-04-25T10:32:57.837 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:57.837 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:32:57.837 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:57.837 INFO:teuthology.orchestra.run.smithi106.stdout: "stretch_mode": false 2024-04-25T10:32:57.837 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-25T10:32:57.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.822+0000 7eff8f996640 1 -- 172.21.15.106:0/2391788594 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7eff6c0744b0 msgr2=0x7eff6c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:57.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.822+0000 7eff8f996640 1 --2- 172.21.15.106:0/2391788594 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7eff6c0744b0 0x7eff6c0769c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7eff88159c90 tx=0x7eff78008040 comp rx=0 tx=0).stop 2024-04-25T10:32:57.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.822+0000 7eff8f996640 1 -- 172.21.15.106:0/2391788594 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff880cc600 msgr2=0x7eff881f18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:57.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.822+0000 7eff8f996640 1 --2- 172.21.15.106:0/2391788594 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff880cc600 0x7eff881f18e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7eff800090b0 tx=0x7eff800046c0 comp rx=0 tx=0).stop 2024-04-25T10:32:57.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.822+0000 7eff8df33640 1 -- 172.21.15.106:0/2391788594 reap_dead start 2024-04-25T10:32:57.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.822+0000 7eff8df33640 1 -- 172.21.15.106:0/2391788594 reap_dead start 2024-04-25T10:32:57.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.822+0000 7eff8f996640 1 -- 172.21.15.106:0/2391788594 shutdown_connections 2024-04-25T10:32:57.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.822+0000 7eff8f996640 1 -- 172.21.15.106:0/2391788594 >> 172.21.15.106:0/2391788594 conn(0x7eff880727d0 msgr2=0x7eff8815d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:57.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.822+0000 7eff8f996640 1 -- 172.21.15.106:0/2391788594 shutdown_connections 2024-04-25T10:32:57.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.822+0000 7eff8f996640 1 -- 172.21.15.106:0/2391788594 wait complete. 2024-04-25T10:32:57.839 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T10:32:57.839 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-04-25T10:32:57.839 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-25T10:32:57.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.945+0000 7f6cc69ee640 1 Processor -- start 2024-04-25T10:32:57.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.945+0000 7f6cc69ee640 1 -- start start 2024-04-25T10:32:57.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.946+0000 7f6cc69ee640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cc015a590 0x7f6cc00c5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:57.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.946+0000 7f6cc69ee640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6cc015d9a0 0x7f6cc0153710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:57.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.946+0000 7f6cc69ee640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6cc00c7c40 con 0x7f6cc0072870 2024-04-25T10:32:57.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.946+0000 7f6cc69ee640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f6cc00cb590 con 0x7f6cc015d9a0 2024-04-25T10:32:57.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.946+0000 7f6cc69ee640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6cc015a9a0 con 0x7f6cc015a590 2024-04-25T10:32:57.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.947+0000 7f6cc4f8b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6cc015d9a0 0x7f6cc0153710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:57.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.947+0000 7f6cbffff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f6cc0072870 0x7f6cc00cae80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:44906/0 (socket says 172.21.15.106:44906) 2024-04-25T10:32:57.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.947+0000 7f6cbffff640 1 -- 172.21.15.106:0/2181317792 learned_addr learned my addr 172.21.15.106:0/2181317792 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:32:57.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.947+0000 7f6cbf7fe640 1 --2- 172.21.15.106:0/2181317792 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cc015a590 0x7f6cc00c5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:57.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.947+0000 7f6cc4f8b640 1 -- 172.21.15.106:0/2181317792 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cc015a590 msgr2=0x7f6cc00c5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:57.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.947+0000 7f6cc4f8b640 1 --2- 172.21.15.106:0/2181317792 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cc015a590 0x7f6cc00c5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:57.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.947+0000 7f6cc4f8b640 1 -- 172.21.15.106:0/2181317792 >> v1:172.21.15.106:6789/0 conn(0x7f6cc0072870 legacy=0x7f6cc00cae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:57.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.947+0000 7f6cbeffd640 1 -- 172.21.15.106:0/2181317792 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1796711049 0 0) 0x7f6cc00c7c40 con 0x7f6cc0072870 2024-04-25T10:32:57.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.947+0000 7f6cc4f8b640 1 -- 172.21.15.106:0/2181317792 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cc00c7f20 con 0x7f6cc015d9a0 2024-04-25T10:32:57.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.948+0000 7f6cc4f8b640 1 --2- 172.21.15.106:0/2181317792 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6cc015d9a0 0x7f6cc0153710 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6cb8009d90 tx=0x7f6cb802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6e1e36e74b01239 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:57.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.948+0000 7f6cbeffd640 1 -- 172.21.15.106:0/2181317792 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb8007740 con 0x7f6cc015d9a0 2024-04-25T10:32:57.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.948+0000 7f6cbeffd640 1 -- 172.21.15.106:0/2181317792 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cb80078e0 con 0x7f6cc015d9a0 2024-04-25T10:32:57.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.948+0000 7f6cbeffd640 1 -- 172.21.15.106:0/2181317792 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb8007cb0 con 0x7f6cc015d9a0 2024-04-25T10:32:57.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.948+0000 7f6cc69ee640 1 -- 172.21.15.106:0/2181317792 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6cc015d9a0 msgr2=0x7f6cc0153710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:57.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.948+0000 7f6cc69ee640 1 --2- 172.21.15.106:0/2181317792 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6cc015d9a0 0x7f6cc0153710 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6cb8009d90 tx=0x7f6cb802a660 comp rx=0 tx=0).stop 2024-04-25T10:32:57.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.948+0000 7f6cc69ee640 1 -- 172.21.15.106:0/2181317792 shutdown_connections 2024-04-25T10:32:57.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.948+0000 7f6cc69ee640 1 --2- 172.21.15.106:0/2181317792 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6cc015d9a0 0x7f6cc0153710 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:57.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.948+0000 7f6cc69ee640 1 --2- 172.21.15.106:0/2181317792 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cc015a590 0x7f6cc00c5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:57.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.948+0000 7f6cc69ee640 1 -- 172.21.15.106:0/2181317792 >> 172.21.15.106:0/2181317792 conn(0x7f6cc006c9b0 msgr2=0x7f6cc006cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:57.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.949+0000 7f6cc69ee640 1 -- 172.21.15.106:0/2181317792 shutdown_connections 2024-04-25T10:32:57.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.949+0000 7f6cc69ee640 1 -- 172.21.15.106:0/2181317792 wait complete. 2024-04-25T10:32:57.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.950+0000 7f6cc69ee640 1 Processor -- start 2024-04-25T10:32:57.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.950+0000 7f6cc69ee640 1 -- start start 2024-04-25T10:32:57.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.952+0000 7f6cc69ee640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cc0072870 0x7f6cc01f31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:57.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.952+0000 7f6cc69ee640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6cc015a590 0x7f6cc00cd410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:57.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.952+0000 7f6cc69ee640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cc015d9a0 0x7f6cc014a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:57.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.952+0000 7f6cc69ee640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc0159d90 con 0x7f6cc015a590 2024-04-25T10:32:57.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.952+0000 7f6cc69ee640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc0159ab0 con 0x7f6cc0072870 2024-04-25T10:32:57.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.952+0000 7f6cc69ee640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6cc015a070 con 0x7f6cc015d9a0 2024-04-25T10:32:57.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.952+0000 7f6cbf7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6cc015a590 0x7f6cc00cd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:57.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.952+0000 7f6cbf7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6cc015a590 0x7f6cc00cd410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55748/0 (socket says 172.21.15.106:55748) 2024-04-25T10:32:57.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.952+0000 7f6cbf7fe640 1 -- 172.21.15.106:0/1647340093 learned_addr learned my addr 172.21.15.106:0/1647340093 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:32:57.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.952+0000 7f6cc4f8b640 1 --2- 172.21.15.106:0/1647340093 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cc015d9a0 0x7f6cc014a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:57.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.952+0000 7f6cbffff640 1 --2- 172.21.15.106:0/1647340093 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cc0072870 0x7f6cc01f31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:57.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.952+0000 7f6cbf7fe640 1 -- 172.21.15.106:0/1647340093 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cc015d9a0 msgr2=0x7f6cc014a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:57.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.952+0000 7f6cbf7fe640 1 --2- 172.21.15.106:0/1647340093 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cc015d9a0 0x7f6cc014a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:57.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.952+0000 7f6cbf7fe640 1 -- 172.21.15.106:0/1647340093 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cc0072870 msgr2=0x7f6cc01f31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:57.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.952+0000 7f6cbf7fe640 1 --2- 172.21.15.106:0/1647340093 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cc0072870 0x7f6cc01f31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:57.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.952+0000 7f6cbf7fe640 1 -- 172.21.15.106:0/1647340093 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cc01f36e0 con 0x7f6cc015a590 2024-04-25T10:32:57.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.952+0000 7f6cbf7fe640 1 --2- 172.21.15.106:0/1647340093 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6cc015a590 0x7f6cc00cd410 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f6cb400e870 tx=0x7f6cb400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:57.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.952+0000 7f6cbcff9640 1 -- 172.21.15.106:0/1647340093 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb4017070 con 0x7f6cc015a590 2024-04-25T10:32:57.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.952+0000 7f6cbcff9640 1 -- 172.21.15.106:0/1647340093 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cb400ee60 con 0x7f6cc015a590 2024-04-25T10:32:57.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.952+0000 7f6cbcff9640 1 -- 172.21.15.106:0/1647340093 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb400cb40 con 0x7f6cc015a590 2024-04-25T10:32:57.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.952+0000 7f6cc69ee640 1 -- 172.21.15.106:0/1647340093 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cc01646a0 con 0x7f6cc015a590 2024-04-25T10:32:57.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.952+0000 7f6cc69ee640 1 -- 172.21.15.106:0/1647340093 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cc01f2c80 con 0x7f6cc015a590 2024-04-25T10:32:57.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.953+0000 7f6cc69ee640 1 --2- 172.21.15.106:0/1647340093 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cc0155a90 0x7f6cc00cd950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:57.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.953+0000 7f6cc69ee640 1 -- 172.21.15.106:0/1647340093 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6cc0164450 con 0x7f6cc0155a90 2024-04-25T10:32:57.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.953+0000 7f6cbffff640 1 --2- 172.21.15.106:0/1647340093 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cc0155a90 0x7f6cc00cd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:57.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.954+0000 7f6cbffff640 1 -- 172.21.15.106:0/1647340093 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6cc006a3d0 con 0x7f6cc0155a90 2024-04-25T10:32:57.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.954+0000 7f6cbffff640 1 --2- 172.21.15.106:0/1647340093 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cc0155a90 0x7f6cc00cd950 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6cb0000f60 tx=0x7f6cb00018d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:57.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.954+0000 7f6cbcff9640 1 -- 172.21.15.106:0/1647340093 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb0011580 con 0x7f6cc0155a90 2024-04-25T10:32:57.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.954+0000 7f6cbcff9640 1 -- 172.21.15.106:0/1647340093 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6cb401a070 con 0x7f6cc015a590 2024-04-25T10:32:57.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.955+0000 7f6cbcff9640 1 --2- 172.21.15.106:0/1647340093 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f6cac0743b0 0x7f6cac0768c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:57.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.955+0000 7f6cbcff9640 1 -- 172.21.15.106:0/1647340093 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f6cb4095320 con 0x7f6cc015a590 2024-04-25T10:32:57.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.955+0000 7f6cc4f8b640 1 --2- 172.21.15.106:0/1647340093 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f6cac0743b0 0x7f6cac0768c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:57.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.955+0000 7f6cbcff9640 1 -- 172.21.15.106:0/1647340093 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f6cb0018070 con 0x7f6cc0155a90 2024-04-25T10:32:57.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.955+0000 7f6cbcff9640 1 -- 172.21.15.106:0/1647340093 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cc0155a90 msgr2=0x7f6cc00cd950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:57.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.955+0000 7f6cbcff9640 1 --2- 172.21.15.106:0/1647340093 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cc0155a90 0x7f6cc00cd950 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6cb0000f60 tx=0x7f6cb00018d0 comp rx=0 tx=0).stop 2024-04-25T10:32:57.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.955+0000 7f6cbcff9640 1 -- 172.21.15.106:0/1647340093 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cc0155a90 msgr2=0x7f6cc00cd950 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T10:32:57.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.955+0000 7f6cbcff9640 1 --2- 172.21.15.106:0/1647340093 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cc0155a90 0x7f6cc00cd950 secure :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6cb0000f60 tx=0x7f6cb00018d0 comp rx=0 tx=0).stop 2024-04-25T10:32:57.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.955+0000 7f6cc4f8b640 1 --2- 172.21.15.106:0/1647340093 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f6cac0743b0 0x7f6cac0768c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6cb8034000 tx=0x7f6cb802f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:57.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.961+0000 7f6cc69ee640 1 --2- 172.21.15.106:0/1647340093 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cc00ce440 0x7f6cc014acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:57.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.961+0000 7f6cc69ee640 1 -- 172.21.15.106:0/1647340093 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6cc00c5960 con 0x7f6cc00ce440 2024-04-25T10:32:57.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.961+0000 7f6cbffff640 1 --2- 172.21.15.106:0/1647340093 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cc00ce440 0x7f6cc014acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:57.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.962+0000 7f6cbffff640 1 -- 172.21.15.106:0/1647340093 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f6cc00ca610 con 0x7f6cc00ce440 2024-04-25T10:32:57.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.962+0000 7f6cbffff640 1 --2- 172.21.15.106:0/1647340093 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cc00ce440 0x7f6cc014acf0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6cb0000f60 tx=0x7f6cb0001080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:57.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.962+0000 7f6cbcff9640 1 -- 172.21.15.106:0/1647340093 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb0026410 con 0x7f6cc00ce440 2024-04-25T10:32:57.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.962+0000 7f6cbcff9640 1 -- 172.21.15.106:0/1647340093 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4559 (secure 0 0 0) 0x7f6cb00133f0 con 0x7f6cc00ce440 2024-04-25T10:32:57.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.962+0000 7f6cbcff9640 1 -- 172.21.15.106:0/1647340093 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cc00ce440 msgr2=0x7f6cc014acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:57.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.962+0000 7f6cbcff9640 1 --2- 172.21.15.106:0/1647340093 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cc00ce440 0x7f6cc014acf0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6cb0000f60 tx=0x7f6cb0001080 comp rx=0 tx=0).stop 2024-04-25T10:32:57.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.962+0000 7f6cbcff9640 1 -- 172.21.15.106:0/1647340093 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cc00ce440 msgr2=0x7f6cc014acf0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T10:32:57.965 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.962+0000 7f6cbcff9640 1 --2- 172.21.15.106:0/1647340093 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cc00ce440 0x7f6cc014acf0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:57.966 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-25T10:32:57.966 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "c", 2024-04-25T10:32:57.966 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 2, 2024-04-25T10:32:57.966 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "peon", 2024-04-25T10:32:57.966 INFO:teuthology.orchestra.run.smithi106.stdout: "election_epoch": 16, 2024-04-25T10:32:57.966 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 16886, 2024-04-25T10:32:57.966 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum": [ 2024-04-25T10:32:57.966 INFO:teuthology.orchestra.run.smithi106.stdout: 0, 2024-04-25T10:32:57.966 INFO:teuthology.orchestra.run.smithi106.stdout: 1, 2024-04-25T10:32:57.966 INFO:teuthology.orchestra.run.smithi106.stdout: 2 2024-04-25T10:32:57.966 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:32:57.967 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_age": 16, 2024-04-25T10:32:57.967 INFO:teuthology.orchestra.run.smithi106.stdout: "features": { 2024-04-25T10:32:57.967 INFO:teuthology.orchestra.run.smithi106.stdout: "required_con": "2450521708007866372", 2024-04-25T10:32:57.967 INFO:teuthology.orchestra.run.smithi106.stdout: "required_mon": [ 2024-04-25T10:32:57.967 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:32:57.967 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:32:57.967 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:32:57.967 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:32:57.967 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:32:57.967 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:32:57.967 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:32:57.967 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:32:57.967 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:32:57.967 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:32:57.967 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:32:57.968 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:32:57.968 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_con": "4540701547738038271", 2024-04-25T10:32:57.968 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_mon": [ 2024-04-25T10:32:57.968 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:32:57.968 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:32:57.968 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:32:57.968 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:32:57.968 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:32:57.968 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:32:57.968 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:32:57.968 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:32:57.968 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:32:57.968 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:32:57.968 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:32:57.969 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:32:57.969 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:57.969 INFO:teuthology.orchestra.run.smithi106.stdout: "outside_quorum": [], 2024-04-25T10:32:57.969 INFO:teuthology.orchestra.run.smithi106.stdout: "extra_probe_peers": [], 2024-04-25T10:32:57.969 INFO:teuthology.orchestra.run.smithi106.stdout: "sync_provider": [], 2024-04-25T10:32:57.969 INFO:teuthology.orchestra.run.smithi106.stdout: "monmap": { 2024-04-25T10:32:57.969 INFO:teuthology.orchestra.run.smithi106.stdout: "epoch": 1, 2024-04-25T10:32:57.969 INFO:teuthology.orchestra.run.smithi106.stdout: "fsid": "3ee77985-e8e4-41cf-81e9-7c774cf7891f", 2024-04-25T10:32:57.969 INFO:teuthology.orchestra.run.smithi106.stdout: "modified": "2024-04-25T10:29:33.446228Z", 2024-04-25T10:32:57.969 INFO:teuthology.orchestra.run.smithi106.stdout: "created": "2024-04-25T10:29:33.446228Z", 2024-04-25T10:32:57.969 INFO:teuthology.orchestra.run.smithi106.stdout: "min_mon_release": 19, 2024-04-25T10:32:57.969 INFO:teuthology.orchestra.run.smithi106.stdout: "min_mon_release_name": "squid", 2024-04-25T10:32:57.969 INFO:teuthology.orchestra.run.smithi106.stdout: "election_strategy": 1, 2024-04-25T10:32:57.969 INFO:teuthology.orchestra.run.smithi106.stdout: "disallowed_leaders: ": "", 2024-04-25T10:32:57.970 INFO:teuthology.orchestra.run.smithi106.stdout: "stretch_mode": false, 2024-04-25T10:32:57.970 INFO:teuthology.orchestra.run.smithi106.stdout: "tiebreaker_mon": "", 2024-04-25T10:32:57.970 INFO:teuthology.orchestra.run.smithi106.stdout: "removed_ranks: ": "", 2024-04-25T10:32:57.970 INFO:teuthology.orchestra.run.smithi106.stdout: "features": { 2024-04-25T10:32:57.970 INFO:teuthology.orchestra.run.smithi106.stdout: "persistent": [ 2024-04-25T10:32:57.970 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:32:57.970 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:32:57.970 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:32:57.970 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:32:57.970 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:32:57.970 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:32:57.970 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:32:57.970 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:32:57.971 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:32:57.971 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:32:57.971 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:32:57.971 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:32:57.971 INFO:teuthology.orchestra.run.smithi106.stdout: "optional": [] 2024-04-25T10:32:57.971 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:57.971 INFO:teuthology.orchestra.run.smithi106.stdout: "mons": [ 2024-04-25T10:32:57.971 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:57.971 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 0, 2024-04-25T10:32:57.971 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "a", 2024-04-25T10:32:57.971 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:32:57.971 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:32:57.971 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:57.971 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:32:57.971 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:3300", 2024-04-25T10:32:57.972 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:32:57.972 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:57.972 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:57.972 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:32:57.972 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:6789", 2024-04-25T10:32:57.972 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:32:57.972 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:57.972 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:32:57.972 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:57.972 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:6789/0", 2024-04-25T10:32:57.972 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.106:6789/0", 2024-04-25T10:32:57.972 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:32:57.972 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:32:57.972 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:32:57.972 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:57.973 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:57.973 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 1, 2024-04-25T10:32:57.973 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "b", 2024-04-25T10:32:57.973 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:32:57.973 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:32:57.973 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:57.973 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:32:57.973 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:3300", 2024-04-25T10:32:57.973 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:32:57.973 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:57.973 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:57.973 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:32:57.973 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6789", 2024-04-25T10:32:57.973 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:32:57.974 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:57.974 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:32:57.974 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:57.974 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6789/0", 2024-04-25T10:32:57.974 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.134:6789/0", 2024-04-25T10:32:57.974 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:32:57.974 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:32:57.974 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:32:57.974 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:57.974 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:57.974 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 2, 2024-04-25T10:32:57.974 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "c", 2024-04-25T10:32:57.974 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:32:57.974 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:32:57.975 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:57.975 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:32:57.975 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:3301", 2024-04-25T10:32:57.975 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:32:57.975 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:57.975 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:57.975 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:32:57.975 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6790", 2024-04-25T10:32:57.975 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:32:57.975 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:57.975 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:32:57.975 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:57.975 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6790/0", 2024-04-25T10:32:57.975 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.134:6790/0", 2024-04-25T10:32:57.975 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:32:57.976 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:32:57.976 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:32:57.976 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:57.976 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:32:57.976 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:57.976 INFO:teuthology.orchestra.run.smithi106.stdout: "feature_map": { 2024-04-25T10:32:57.976 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": [ 2024-04-25T10:32:57.976 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:57.976 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:32:57.976 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:32:57.976 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 1 2024-04-25T10:32:57.976 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:57.976 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:32:57.976 INFO:teuthology.orchestra.run.smithi106.stdout: "client": [ 2024-04-25T10:32:57.976 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:32:57.977 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:32:57.977 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:32:57.977 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 1 2024-04-25T10:32:57.977 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:32:57.977 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:32:57.977 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:32:57.977 INFO:teuthology.orchestra.run.smithi106.stdout: "stretch_mode": false 2024-04-25T10:32:57.977 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-25T10:32:57.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.963+0000 7f6cc69ee640 1 -- 172.21.15.106:0/1647340093 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f6cac0743b0 msgr2=0x7f6cac0768c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:57.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.963+0000 7f6cc69ee640 1 --2- 172.21.15.106:0/1647340093 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f6cac0743b0 0x7f6cac0768c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6cb8034000 tx=0x7f6cb802f000 comp rx=0 tx=0).stop 2024-04-25T10:32:57.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.963+0000 7f6cc69ee640 1 -- 172.21.15.106:0/1647340093 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6cc015a590 msgr2=0x7f6cc00cd410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:57.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.963+0000 7f6cc69ee640 1 --2- 172.21.15.106:0/1647340093 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6cc015a590 0x7f6cc00cd410 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f6cb400e870 tx=0x7f6cb400ed30 comp rx=0 tx=0).stop 2024-04-25T10:32:57.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.963+0000 7f6cc4f8b640 1 -- 172.21.15.106:0/1647340093 reap_dead start 2024-04-25T10:32:57.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.963+0000 7f6cc4f8b640 1 -- 172.21.15.106:0/1647340093 reap_dead start 2024-04-25T10:32:57.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.963+0000 7f6cc69ee640 1 -- 172.21.15.106:0/1647340093 shutdown_connections 2024-04-25T10:32:57.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.963+0000 7f6cc69ee640 1 -- 172.21.15.106:0/1647340093 >> 172.21.15.106:0/1647340093 conn(0x7f6cc006c9b0 msgr2=0x7f6cc006cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:57.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.963+0000 7f6cc69ee640 1 -- 172.21.15.106:0/1647340093 shutdown_connections 2024-04-25T10:32:57.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:57.963+0000 7f6cc69ee640 1 -- 172.21.15.106:0/1647340093 wait complete. 2024-04-25T10:32:57.980 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T10:32:57.980 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b'] 2024-04-25T10:32:57.980 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-04-25T10:32:57.980 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2024-04-25T10:32:57.980 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-04-25T10:32:57.980 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-25T10:32:57.980 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:32:57.995 INFO:tasks.ceph.mon.b:Stopped 2024-04-25T10:32:57.995 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-04-25T10:32:57.996 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T10:32:58.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:58.099+0000 7fc90b50d640 1 Processor -- start 2024-04-25T10:32:58.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:58.100+0000 7fc90b50d640 1 -- start start 2024-04-25T10:32:58.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:58.101+0000 7fc90b50d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc9040c61a0 0x7fc904156f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:58.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:58.101+0000 7fc90b50d640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc90415cbd0 con 0x7fc90415d690 2024-04-25T10:32:58.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:58.101+0000 7fc90b50d640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc904160990 con 0x7fc90406dc00 2024-04-25T10:32:58.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:58.101+0000 7fc90b50d640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fc90415d200 con 0x7fc9040c61a0 2024-04-25T10:32:58.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:58.101+0000 7fc908aa8640 1 -- >> v1:172.21.15.134:6789/0 conn(0x7fc90406dc00 legacy=0x7fc9041601c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-04-25T10:32:58.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:58.101+0000 7fc9092a9640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc9040c61a0 0x7fc904156f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:58.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:58.101+0000 7fc909aaa640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fc90415d690 0x7fc9040cce20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:44916/0 (socket says 172.21.15.106:44916) 2024-04-25T10:32:58.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:58.101+0000 7fc909aaa640 1 -- 172.21.15.106:0/4031336114 learned_addr learned my addr 172.21.15.106:0/4031336114 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:32:58.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:58.101+0000 7fc9092a9640 1 -- 172.21.15.106:0/4031336114 >> v1:172.21.15.134:6789/0 conn(0x7fc90406dc00 legacy=0x7fc9041601c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:32:58.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:58.101+0000 7fc9092a9640 1 -- 172.21.15.106:0/4031336114 >> v1:172.21.15.106:6789/0 conn(0x7fc90415d690 legacy=0x7fc9040cce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:58.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:58.101+0000 7fc9092a9640 1 -- 172.21.15.106:0/4031336114 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc904160c70 con 0x7fc9040c61a0 2024-04-25T10:32:58.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:58.101+0000 7fc8fbfff640 1 -- 172.21.15.106:0/4031336114 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3177313304 0 0) 0x7fc90415cbd0 con 0x7fc90415d690 2024-04-25T10:32:58.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:58.102+0000 7fc9092a9640 1 --2- 172.21.15.106:0/4031336114 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc9040c61a0 0x7fc904156f30 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc90415d380 tx=0x7fc90002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf73867d7539ea59 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:58.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:58.102+0000 7fc8fbfff640 1 -- 172.21.15.106:0/4031336114 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc900037070 con 0x7fc9040c61a0 2024-04-25T10:32:58.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:58.102+0000 7fc8fbfff640 1 -- 172.21.15.106:0/4031336114 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc900032440 con 0x7fc9040c61a0 2024-04-25T10:32:58.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:58.102+0000 7fc8fbfff640 1 -- 172.21.15.106:0/4031336114 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc900032810 con 0x7fc9040c61a0 2024-04-25T10:32:58.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:58.103+0000 7fc90b50d640 1 -- 172.21.15.106:0/4031336114 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc9040c61a0 msgr2=0x7fc904156f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:58.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:58.103+0000 7fc90b50d640 1 --2- 172.21.15.106:0/4031336114 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc9040c61a0 0x7fc904156f30 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc90415d380 tx=0x7fc90002a1c0 comp rx=0 tx=0).stop 2024-04-25T10:32:58.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:58.103+0000 7fc90b50d640 1 -- 172.21.15.106:0/4031336114 shutdown_connections 2024-04-25T10:32:58.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:58.103+0000 7fc90b50d640 1 --2- 172.21.15.106:0/4031336114 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc9040c61a0 0x7fc904156f30 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:58.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:58.103+0000 7fc90b50d640 1 -- 172.21.15.106:0/4031336114 >> 172.21.15.106:0/4031336114 conn(0x7fc9040727d0 msgr2=0x7fc904072c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:58.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:58.103+0000 7fc90b50d640 1 -- 172.21.15.106:0/4031336114 shutdown_connections 2024-04-25T10:32:58.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:58.103+0000 7fc90b50d640 1 -- 172.21.15.106:0/4031336114 wait complete. 2024-04-25T10:32:58.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:58.105+0000 7fc90b50d640 1 Processor -- start 2024-04-25T10:32:58.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:58.105+0000 7fc90b50d640 1 -- start start 2024-04-25T10:32:58.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:58.106+0000 7fc90b50d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc90406dc00 0x7fc90415e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:58.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:58.106+0000 7fc90b50d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc9040c61a0 0x7fc904154c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:58.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:58.106+0000 7fc90b50d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc90415d690 0x7fc9040cf320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:58.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:58.106+0000 7fc90b50d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc90415daa0 con 0x7fc90415d690 2024-04-25T10:32:58.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:58.106+0000 7fc90b50d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc90415ce90 con 0x7fc90406dc00 2024-04-25T10:32:58.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:58.106+0000 7fc90b50d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc90406d6d0 con 0x7fc9040c61a0 2024-04-25T10:32:58.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:58.106+0000 7fc909aaa640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc90415d690 0x7fc9040cf320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:58.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:58.106+0000 7fc9092a9640 1 -- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc90406dc00 msgr2=0x7fc90415e7a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3300/0 2024-04-25T10:32:58.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:58.106+0000 7fc9092a9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc90406dc00 0x7fc90415e7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:32:58.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:58.106+0000 7fc909aaa640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc90415d690 0x7fc9040cf320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55788/0 (socket says 172.21.15.106:55788) 2024-04-25T10:32:58.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:58.106+0000 7fc909aaa640 1 -- 172.21.15.106:0/1731556920 learned_addr learned my addr 172.21.15.106:0/1731556920 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:32:58.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:58.106+0000 7fc908aa8640 1 --2- 172.21.15.106:0/1731556920 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc9040c61a0 0x7fc904154c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:58.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:58.106+0000 7fc909aaa640 1 -- 172.21.15.106:0/1731556920 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc9040c61a0 msgr2=0x7fc904154c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:58.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:58.106+0000 7fc909aaa640 1 --2- 172.21.15.106:0/1731556920 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc9040c61a0 0x7fc904154c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:58.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:58.106+0000 7fc909aaa640 1 -- 172.21.15.106:0/1731556920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc90406dc00 msgr2=0x7fc90415e7a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:32:58.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:58.106+0000 7fc909aaa640 1 --2- 172.21.15.106:0/1731556920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc90406dc00 0x7fc90415e7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:58.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:58.106+0000 7fc909aaa640 1 -- 172.21.15.106:0/1731556920 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc904155150 con 0x7fc90415d690 2024-04-25T10:32:58.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:58.106+0000 7fc909aaa640 1 --2- 172.21.15.106:0/1731556920 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc90415d690 0x7fc9040cf320 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc00c730 tx=0x7fc8fc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:58.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:58.107+0000 7fc8f9ffb640 1 -- 172.21.15.106:0/1731556920 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc018070 con 0x7fc90415d690 2024-04-25T10:32:58.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:58.107+0000 7fc8f9ffb640 1 -- 172.21.15.106:0/1731556920 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8fc00ce40 con 0x7fc90415d690 2024-04-25T10:32:58.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:58.107+0000 7fc90b50d640 1 -- 172.21.15.106:0/1731556920 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc90414cdb0 con 0x7fc90415d690 2024-04-25T10:32:58.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:58.107+0000 7fc8f9ffb640 1 -- 172.21.15.106:0/1731556920 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc00a880 con 0x7fc90415d690 2024-04-25T10:32:58.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:58.107+0000 7fc90b50d640 1 -- 172.21.15.106:0/1731556920 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9040c9fd0 con 0x7fc90415d690 2024-04-25T10:32:58.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:58.108+0000 7fc90b50d640 1 -- 172.21.15.106:0/1731556920 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc90406a3d0 con 0x7fc90415d690 2024-04-25T10:32:58.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:58.108+0000 7fc8f9ffb640 1 -- 172.21.15.106:0/1731556920 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc8fc01b070 con 0x7fc90415d690 2024-04-25T10:32:58.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:58.109+0000 7fc8f9ffb640 1 --2- 172.21.15.106:0/1731556920 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fc8e8074490 0x7fc8e80769a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:58.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:58.109+0000 7fc8f9ffb640 1 -- 172.21.15.106:0/1731556920 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fc8fc094800 con 0x7fc90415d690 2024-04-25T10:32:58.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:58.109+0000 7fc9092a9640 1 --2- 172.21.15.106:0/1731556920 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fc8e8074490 0x7fc8e80769a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:58.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:58.109+0000 7fc9092a9640 1 --2- 172.21.15.106:0/1731556920 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fc8e8074490 0x7fc8e80769a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc900034000 tx=0x7fc90002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:58.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:58.114+0000 7fc8f9ffb640 1 -- 172.21.15.106:0/1731556920 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc8fc05ef10 con 0x7fc90415d690 2024-04-25T10:32:58.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:58.361+0000 7fc90b50d640 1 -- 172.21.15.106:0/1731556920 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fc9040c7010 con 0x7fc90415d690 2024-04-25T10:32:58.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:58.362+0000 7fc8f9ffb640 1 -- 172.21.15.106:0/1731556920 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1569 (secure 0 0 0) 0x7fc8fc05e810 con 0x7fc90415d690 2024-04-25T10:32:58.364 INFO:teuthology.orchestra.run.smithi106.stdout:{"election_epoch":16,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":16,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"3ee77985-e8e4-41cf-81e9-7c774cf7891f","modified":"2024-04-25T10:29:33.446228Z","created":"2024-04-25T10:29:33.446228Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:3300","nonce":0},{"type":"v1","addr":"172.21.15.106:6789","nonce":0}]},"addr":"172.21.15.106:6789/0","public_addr":"172.21.15.106:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T10:32:58.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:58.364+0000 7fc90b50d640 1 -- 172.21.15.106:0/1731556920 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fc8e8074490 msgr2=0x7fc8e80769a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:58.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:58.364+0000 7fc90b50d640 1 --2- 172.21.15.106:0/1731556920 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fc8e8074490 0x7fc8e80769a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc900034000 tx=0x7fc90002e000 comp rx=0 tx=0).stop 2024-04-25T10:32:58.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:58.364+0000 7fc90b50d640 1 -- 172.21.15.106:0/1731556920 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc90415d690 msgr2=0x7fc9040cf320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:58.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:58.364+0000 7fc90b50d640 1 --2- 172.21.15.106:0/1731556920 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc90415d690 0x7fc9040cf320 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc00c730 tx=0x7fc8fc00cbf0 comp rx=0 tx=0).stop 2024-04-25T10:32:58.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:58.364+0000 7fc90b50d640 1 -- 172.21.15.106:0/1731556920 shutdown_connections 2024-04-25T10:32:58.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:58.364+0000 7fc90b50d640 1 --2- 172.21.15.106:0/1731556920 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fc8e8074490 0x7fc8e80769a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:58.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:58.364+0000 7fc90b50d640 1 --2- 172.21.15.106:0/1731556920 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc90415d690 0x7fc9040cf320 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:58.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:58.364+0000 7fc90b50d640 1 --2- 172.21.15.106:0/1731556920 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc9040c61a0 0x7fc904154c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:58.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:58.364+0000 7fc90b50d640 1 --2- 172.21.15.106:0/1731556920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc90406dc00 0x7fc90415e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:58.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:58.364+0000 7fc90b50d640 1 -- 172.21.15.106:0/1731556920 >> 172.21.15.106:0/1731556920 conn(0x7fc9040727d0 msgr2=0x7fc90406e890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:58.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:58.364+0000 7fc90b50d640 1 -- 172.21.15.106:0/1731556920 shutdown_connections 2024-04-25T10:32:58.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:32:58.364+0000 7fc90b50d640 1 -- 172.21.15.106:0/1731556920 wait complete. 2024-04-25T10:33:01.378 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T10:33:01.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.494+0000 7fac56a27640 1 Processor -- start 2024-04-25T10:33:01.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.494+0000 7fac56a27640 1 -- start start 2024-04-25T10:33:01.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.495+0000 7fac56a27640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fac500c7ad0 0x7fac5006ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:01.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.495+0000 7fac56a27640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fac500c8120 0x7fac500c8530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:01.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.495+0000 7fac56a27640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac5015d6d0 con 0x7fac50156f70 2024-04-25T10:33:01.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.495+0000 7fac56a27640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fac5015d9b0 con 0x7fac500c8120 2024-04-25T10:33:01.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.495+0000 7fac56a27640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fac500c7150 con 0x7fac500c7ad0 2024-04-25T10:33:01.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.496+0000 7fac54fc4640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fac500c8120 0x7fac500c8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:01.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.496+0000 7fac4ffff640 1 -- >> v2:172.21.15.134:3300/0 conn(0x7fac500c7ad0 msgr2=0x7fac5006ea60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-04-25T10:33:01.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.496+0000 7fac4f7fe640 1 -- >> v1:172.21.15.134:6789/0 conn(0x7fac50156f70 legacy=0x7fac5006d560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-04-25T10:33:01.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.496+0000 7fac54fc4640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fac500c8120 0x7fac500c8530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55790/0 (socket says 172.21.15.106:55790) 2024-04-25T10:33:01.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.496+0000 7fac4ffff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fac500c7ad0 0x7fac5006ea60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:33:01.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.496+0000 7fac54fc4640 1 -- 172.21.15.106:0/2357228297 learned_addr learned my addr 172.21.15.106:0/2357228297 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:33:01.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.496+0000 7fac54fc4640 1 -- 172.21.15.106:0/2357228297 >> v2:172.21.15.134:3300/0 conn(0x7fac500c7ad0 msgr2=0x7fac5006ea60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:33:01.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.496+0000 7fac54fc4640 1 --2- 172.21.15.106:0/2357228297 >> v2:172.21.15.134:3300/0 conn(0x7fac500c7ad0 0x7fac5006ea60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:01.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.496+0000 7fac54fc4640 1 -- 172.21.15.106:0/2357228297 >> v1:172.21.15.134:6789/0 conn(0x7fac50156f70 legacy=0x7fac5006d560 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:33:01.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.496+0000 7fac54fc4640 1 -- 172.21.15.106:0/2357228297 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac500c7430 con 0x7fac500c8120 2024-04-25T10:33:01.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.497+0000 7fac54fc4640 1 --2- 172.21.15.106:0/2357228297 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fac500c8120 0x7fac500c8530 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fac5006e780 tx=0x7fac4802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f0a012097799ea0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:01.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.497+0000 7fac4effd640 1 -- 172.21.15.106:0/2357228297 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac48036070 con 0x7fac500c8120 2024-04-25T10:33:01.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.497+0000 7fac4effd640 1 -- 172.21.15.106:0/2357228297 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac48031600 con 0x7fac500c8120 2024-04-25T10:33:01.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.497+0000 7fac4effd640 1 -- 172.21.15.106:0/2357228297 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac480319d0 con 0x7fac500c8120 2024-04-25T10:33:01.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.498+0000 7fac56a27640 1 -- 172.21.15.106:0/2357228297 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fac500c8120 msgr2=0x7fac500c8530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:01.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.498+0000 7fac56a27640 1 --2- 172.21.15.106:0/2357228297 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fac500c8120 0x7fac500c8530 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fac5006e780 tx=0x7fac4802a710 comp rx=0 tx=0).stop 2024-04-25T10:33:01.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.498+0000 7fac56a27640 1 -- 172.21.15.106:0/2357228297 shutdown_connections 2024-04-25T10:33:01.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.498+0000 7fac56a27640 1 --2- 172.21.15.106:0/2357228297 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fac500c8120 0x7fac500c8530 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:01.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.498+0000 7fac56a27640 1 --2- 172.21.15.106:0/2357228297 >> v2:172.21.15.134:3300/0 conn(0x7fac500c7ad0 0x7fac5006ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:01.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.498+0000 7fac56a27640 1 -- 172.21.15.106:0/2357228297 >> 172.21.15.106:0/2357228297 conn(0x7fac50068ed0 msgr2=0x7fac500d20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:01.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.498+0000 7fac56a27640 1 -- 172.21.15.106:0/2357228297 shutdown_connections 2024-04-25T10:33:01.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.498+0000 7fac56a27640 1 -- 172.21.15.106:0/2357228297 wait complete. 2024-04-25T10:33:01.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.499+0000 7fac56a27640 1 Processor -- start 2024-04-25T10:33:01.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.500+0000 7fac56a27640 1 -- start start 2024-04-25T10:33:01.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.501+0000 7fac56a27640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fac500c7ad0 0x7fac5015e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:01.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.501+0000 7fac56a27640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fac500c8120 0x7fac5014fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:01.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.501+0000 7fac56a27640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac50156f70 0x7fac5014ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:01.504 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.501+0000 7fac56a27640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fac500ce360 con 0x7fac500c8120 2024-04-25T10:33:01.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.501+0000 7fac56a27640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fac500cdea0 con 0x7fac50156f70 2024-04-25T10:33:01.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.501+0000 7fac56a27640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fac500ce6c0 con 0x7fac500c7ad0 2024-04-25T10:33:01.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.501+0000 7fac4f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fac500c8120 0x7fac5014fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:01.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.501+0000 7fac54fc4640 1 -- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac50156f70 msgr2=0x7fac5014ffa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3300/0 2024-04-25T10:33:01.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.501+0000 7fac4f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fac500c8120 0x7fac5014fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55794/0 (socket says 172.21.15.106:55794) 2024-04-25T10:33:01.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.501+0000 7fac4f7fe640 1 -- 172.21.15.106:0/2809558331 learned_addr learned my addr 172.21.15.106:0/2809558331 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:33:01.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.501+0000 7fac54fc4640 1 --2- 172.21.15.106:0/2809558331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac50156f70 0x7fac5014ffa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:33:01.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.501+0000 7fac4ffff640 1 --2- 172.21.15.106:0/2809558331 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fac500c7ad0 0x7fac5015e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:01.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.501+0000 7fac4f7fe640 1 -- 172.21.15.106:0/2809558331 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fac500c7ad0 msgr2=0x7fac5015e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:01.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.501+0000 7fac4f7fe640 1 --2- 172.21.15.106:0/2809558331 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fac500c7ad0 0x7fac5015e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:01.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.501+0000 7fac4f7fe640 1 -- 172.21.15.106:0/2809558331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac50156f70 msgr2=0x7fac5014ffa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:33:01.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.501+0000 7fac4f7fe640 1 --2- 172.21.15.106:0/2809558331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac50156f70 0x7fac5014ffa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:01.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.501+0000 7fac4f7fe640 1 -- 172.21.15.106:0/2809558331 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac5015ebe0 con 0x7fac500c8120 2024-04-25T10:33:01.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.502+0000 7fac4f7fe640 1 --2- 172.21.15.106:0/2809558331 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fac500c8120 0x7fac5014fa60 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fac4400b580 tx=0x7fac4400ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:01.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.502+0000 7fac4cff9640 1 -- 172.21.15.106:0/2809558331 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac44003cf0 con 0x7fac500c8120 2024-04-25T10:33:01.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.502+0000 7fac4cff9640 1 -- 172.21.15.106:0/2809558331 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac44015070 con 0x7fac500c8120 2024-04-25T10:33:01.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.502+0000 7fac56a27640 1 -- 172.21.15.106:0/2809558331 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac50158a20 con 0x7fac500c8120 2024-04-25T10:33:01.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.502+0000 7fac4cff9640 1 -- 172.21.15.106:0/2809558331 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac44005330 con 0x7fac500c8120 2024-04-25T10:33:01.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.502+0000 7fac56a27640 1 -- 172.21.15.106:0/2809558331 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac50158f60 con 0x7fac500c8120 2024-04-25T10:33:01.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.503+0000 7fac56a27640 1 -- 172.21.15.106:0/2809558331 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac5006a3d0 con 0x7fac500c8120 2024-04-25T10:33:01.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.504+0000 7fac4cff9640 1 -- 172.21.15.106:0/2809558331 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fac440054d0 con 0x7fac500c8120 2024-04-25T10:33:01.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.504+0000 7fac4cff9640 1 --2- 172.21.15.106:0/2809558331 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fac3c074490 0x7fac3c0769a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:01.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.504+0000 7fac4cff9640 1 -- 172.21.15.106:0/2809558331 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fac44094fe0 con 0x7fac500c8120 2024-04-25T10:33:01.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.504+0000 7fac4ffff640 1 --2- 172.21.15.106:0/2809558331 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fac3c074490 0x7fac3c0769a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:01.507 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.505+0000 7fac4ffff640 1 --2- 172.21.15.106:0/2809558331 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fac3c074490 0x7fac3c0769a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fac400093d0 tx=0x7fac40009360 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:01.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.510+0000 7fac4cff9640 1 -- 172.21.15.106:0/2809558331 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fac4405f790 con 0x7fac500c8120 2024-04-25T10:33:01.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.797+0000 7fac56a27640 1 -- 172.21.15.106:0/2809558331 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fac500b4b70 con 0x7fac500c8120 2024-04-25T10:33:01.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.797+0000 7fac4cff9640 1 -- 172.21.15.106:0/2809558331 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1569 (secure 0 0 0) 0x7fac44080080 con 0x7fac500c8120 2024-04-25T10:33:01.799 INFO:teuthology.orchestra.run.smithi106.stdout:{"election_epoch":16,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":20,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"3ee77985-e8e4-41cf-81e9-7c774cf7891f","modified":"2024-04-25T10:29:33.446228Z","created":"2024-04-25T10:29:33.446228Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:3300","nonce":0},{"type":"v1","addr":"172.21.15.106:6789","nonce":0}]},"addr":"172.21.15.106:6789/0","public_addr":"172.21.15.106:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T10:33:01.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.799+0000 7fac56a27640 1 -- 172.21.15.106:0/2809558331 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fac3c074490 msgr2=0x7fac3c0769a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:01.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.799+0000 7fac56a27640 1 --2- 172.21.15.106:0/2809558331 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fac3c074490 0x7fac3c0769a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fac400093d0 tx=0x7fac40009360 comp rx=0 tx=0).stop 2024-04-25T10:33:01.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.799+0000 7fac56a27640 1 -- 172.21.15.106:0/2809558331 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fac500c8120 msgr2=0x7fac5014fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:01.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.799+0000 7fac56a27640 1 --2- 172.21.15.106:0/2809558331 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fac500c8120 0x7fac5014fa60 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fac4400b580 tx=0x7fac4400ba40 comp rx=0 tx=0).stop 2024-04-25T10:33:01.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.799+0000 7fac56a27640 1 -- 172.21.15.106:0/2809558331 shutdown_connections 2024-04-25T10:33:01.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.799+0000 7fac56a27640 1 --2- 172.21.15.106:0/2809558331 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fac3c074490 0x7fac3c0769a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:01.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.799+0000 7fac56a27640 1 --2- 172.21.15.106:0/2809558331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac50156f70 0x7fac5014ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:01.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.799+0000 7fac56a27640 1 --2- 172.21.15.106:0/2809558331 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fac500c8120 0x7fac5014fa60 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:01.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.799+0000 7fac56a27640 1 --2- 172.21.15.106:0/2809558331 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fac500c7ad0 0x7fac5015e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:01.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.799+0000 7fac56a27640 1 -- 172.21.15.106:0/2809558331 >> 172.21.15.106:0/2809558331 conn(0x7fac50068ed0 msgr2=0x7fac501520f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:01.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.799+0000 7fac56a27640 1 -- 172.21.15.106:0/2809558331 shutdown_connections 2024-04-25T10:33:01.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:01.799+0000 7fac56a27640 1 -- 172.21.15.106:0/2809558331 wait complete. 2024-04-25T10:33:02.582 DEBUG:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:33:02.585 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:33:04.816 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T10:33:04.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:04.931+0000 7f985ef4d640 1 Processor -- start 2024-04-25T10:33:04.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:04.931+0000 7f985ef4d640 1 -- start start 2024-04-25T10:33:04.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:04.933+0000 7f985ef4d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f98580c8d90 0x7f9858156ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:04.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:04.933+0000 7f985ef4d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f98580b4ce0 0x7f985815ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:04.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:04.933+0000 7f985ef4d640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98580ce630 con 0x7f98580c7e30 2024-04-25T10:33:04.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:04.933+0000 7f985ef4d640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f98580b50f0 con 0x7f98580c8d90 2024-04-25T10:33:04.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:04.933+0000 7f985ef4d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f98580c7c60 con 0x7f98580b4ce0 2024-04-25T10:33:04.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:04.933+0000 7f985cce9640 1 -- >> v1:172.21.15.134:6789/0 conn(0x7f98580c7e30 legacy=0x7f985806d000 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-04-25T10:33:04.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:04.933+0000 7f9857fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f98580c8d90 0x7f9858156ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:04.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:04.933+0000 7f9857fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f98580c8d90 0x7f9858156ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54076/0 (socket says 172.21.15.106:54076) 2024-04-25T10:33:04.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:04.933+0000 7f9857fff640 1 -- 172.21.15.106:0/2041394482 learned_addr learned my addr 172.21.15.106:0/2041394482 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:33:04.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:04.933+0000 7f985d4ea640 1 --2- 172.21.15.106:0/2041394482 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f98580b4ce0 0x7f985815ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:04.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:04.933+0000 7f9857fff640 1 -- 172.21.15.106:0/2041394482 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f98580b4ce0 msgr2=0x7f985815ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:04.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:04.933+0000 7f9857fff640 1 --2- 172.21.15.106:0/2041394482 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f98580b4ce0 0x7f985815ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:04.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:04.933+0000 7f9857fff640 1 -- 172.21.15.106:0/2041394482 >> v1:172.21.15.134:6789/0 conn(0x7f98580c7e30 legacy=0x7f985806d000 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:33:04.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:04.933+0000 7f9857fff640 1 -- 172.21.15.106:0/2041394482 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98580c7980 con 0x7f98580c8d90 2024-04-25T10:33:04.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:04.934+0000 7f9857fff640 1 --2- 172.21.15.106:0/2041394482 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98580c8d90 0x7f9858156ef0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f985808ffe0 tx=0x7f984402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a02d4e047c074aa5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:04.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:04.934+0000 7f98577fe640 1 -- 172.21.15.106:0/2041394482 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f984402ae50 con 0x7f98580c8d90 2024-04-25T10:33:04.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:04.934+0000 7f98577fe640 1 -- 172.21.15.106:0/2041394482 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9844004030 con 0x7f98580c8d90 2024-04-25T10:33:04.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:04.934+0000 7f98577fe640 1 -- 172.21.15.106:0/2041394482 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9844004460 con 0x7f98580c8d90 2024-04-25T10:33:04.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:04.935+0000 7f985ef4d640 1 -- 172.21.15.106:0/2041394482 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98580c8d90 msgr2=0x7f9858156ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:04.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:04.935+0000 7f985ef4d640 1 --2- 172.21.15.106:0/2041394482 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98580c8d90 0x7f9858156ef0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f985808ffe0 tx=0x7f984402a140 comp rx=0 tx=0).stop 2024-04-25T10:33:04.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:04.935+0000 7f985ef4d640 1 -- 172.21.15.106:0/2041394482 shutdown_connections 2024-04-25T10:33:04.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:04.935+0000 7f985ef4d640 1 --2- 172.21.15.106:0/2041394482 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f98580b4ce0 0x7f985815ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:04.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:04.935+0000 7f985ef4d640 1 --2- 172.21.15.106:0/2041394482 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98580c8d90 0x7f9858156ef0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:04.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:04.935+0000 7f985ef4d640 1 -- 172.21.15.106:0/2041394482 >> 172.21.15.106:0/2041394482 conn(0x7f9858068b10 msgr2=0x7f98580d1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:04.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:04.935+0000 7f985ef4d640 1 -- 172.21.15.106:0/2041394482 shutdown_connections 2024-04-25T10:33:04.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:04.935+0000 7f985ef4d640 1 -- 172.21.15.106:0/2041394482 wait complete. 2024-04-25T10:33:04.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:04.937+0000 7f985ef4d640 1 Processor -- start 2024-04-25T10:33:04.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:04.937+0000 7f985ef4d640 1 -- start start 2024-04-25T10:33:04.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:04.939+0000 7f985ef4d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98580b4ce0 0x7f985814f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:04.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:04.939+0000 7f985ef4d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98580c7e30 0x7f98581f7010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:04.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:04.939+0000 7f985ef4d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f98580c8d90 0x7f98581e92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:04.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:04.939+0000 7f985ef4d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f985806d710 con 0x7f98580c7e30 2024-04-25T10:33:04.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:04.939+0000 7f985ef4d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f98580c72f0 con 0x7f98580b4ce0 2024-04-25T10:33:04.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:04.939+0000 7f985ef4d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f985806d890 con 0x7f98580c8d90 2024-04-25T10:33:04.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:04.939+0000 7f985cce9640 1 -- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98580b4ce0 msgr2=0x7f985814f5d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3300/0 2024-04-25T10:33:04.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:04.939+0000 7f985cce9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98580b4ce0 0x7f985814f5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:33:04.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:04.939+0000 7f9857fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98580c7e30 0x7f98581f7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:04.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:04.939+0000 7f9857fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98580c7e30 0x7f98581f7010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54086/0 (socket says 172.21.15.106:54086) 2024-04-25T10:33:04.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:04.939+0000 7f9857fff640 1 -- 172.21.15.106:0/3476233557 learned_addr learned my addr 172.21.15.106:0/3476233557 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:33:04.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:04.939+0000 7f985d4ea640 1 --2- 172.21.15.106:0/3476233557 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f98580c8d90 0x7f98581e92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:04.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:04.939+0000 7f9857fff640 1 -- 172.21.15.106:0/3476233557 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f98580c8d90 msgr2=0x7f98581e92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:04.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:04.939+0000 7f9857fff640 1 --2- 172.21.15.106:0/3476233557 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f98580c8d90 0x7f98581e92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:04.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:04.939+0000 7f9857fff640 1 -- 172.21.15.106:0/3476233557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98580b4ce0 msgr2=0x7f985814f5d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:33:04.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:04.939+0000 7f9857fff640 1 --2- 172.21.15.106:0/3476233557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98580b4ce0 0x7f985814f5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:04.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:04.939+0000 7f9857fff640 1 -- 172.21.15.106:0/3476233557 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98581f7550 con 0x7f98580c7e30 2024-04-25T10:33:04.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:04.940+0000 7f9857fff640 1 --2- 172.21.15.106:0/3476233557 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98580c7e30 0x7f98581f7010 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f984402a180 tx=0x7f98440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:04.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:04.940+0000 7f98557fa640 1 -- 172.21.15.106:0/3476233557 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9844041020 con 0x7f98580c7e30 2024-04-25T10:33:04.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:04.940+0000 7f98557fa640 1 -- 172.21.15.106:0/3476233557 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9844003820 con 0x7f98580c7e30 2024-04-25T10:33:04.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:04.940+0000 7f985ef4d640 1 -- 172.21.15.106:0/3476233557 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98580d1580 con 0x7f98580c7e30 2024-04-25T10:33:04.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:04.940+0000 7f98557fa640 1 -- 172.21.15.106:0/3476233557 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9844003c10 con 0x7f98580c7e30 2024-04-25T10:33:04.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:04.940+0000 7f985ef4d640 1 -- 172.21.15.106:0/3476233557 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98580d1b40 con 0x7f98580c7e30 2024-04-25T10:33:04.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:04.941+0000 7f985ef4d640 1 -- 172.21.15.106:0/3476233557 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98580d12d0 con 0x7f98580c7e30 2024-04-25T10:33:04.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:04.942+0000 7f98557fa640 1 -- 172.21.15.106:0/3476233557 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9844044020 con 0x7f98580c7e30 2024-04-25T10:33:04.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:04.942+0000 7f98557fa640 1 --2- 172.21.15.106:0/3476233557 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f983c074460 0x7f983c076970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:04.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:04.943+0000 7f985cce9640 1 --2- 172.21.15.106:0/3476233557 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f983c074460 0x7f983c076970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:04.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:04.943+0000 7f98557fa640 1 -- 172.21.15.106:0/3476233557 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f98440b5aa0 con 0x7f98580c7e30 2024-04-25T10:33:04.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:04.943+0000 7f985cce9640 1 --2- 172.21.15.106:0/3476233557 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f983c074460 0x7f983c076970 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f984c006fd0 tx=0x7f984c006d80 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:04.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:04.947+0000 7f98557fa640 1 -- 172.21.15.106:0/3476233557 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f984404b030 con 0x7f98580c7e30 2024-04-25T10:33:05.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:05.256+0000 7f985ef4d640 1 -- 172.21.15.106:0/3476233557 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f985806a3d0 con 0x7f98580c7e30 2024-04-25T10:33:05.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:05.256+0000 7f98557fa640 1 -- 172.21.15.106:0/3476233557 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1569 (secure 0 0 0) 0x7f98440801b0 con 0x7f98580c7e30 2024-04-25T10:33:05.259 INFO:teuthology.orchestra.run.smithi106.stdout:{"election_epoch":16,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":23,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"3ee77985-e8e4-41cf-81e9-7c774cf7891f","modified":"2024-04-25T10:29:33.446228Z","created":"2024-04-25T10:29:33.446228Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:3300","nonce":0},{"type":"v1","addr":"172.21.15.106:6789","nonce":0}]},"addr":"172.21.15.106:6789/0","public_addr":"172.21.15.106:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T10:33:05.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:05.258+0000 7f985ef4d640 1 -- 172.21.15.106:0/3476233557 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f983c074460 msgr2=0x7f983c076970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:05.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:05.258+0000 7f985ef4d640 1 --2- 172.21.15.106:0/3476233557 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f983c074460 0x7f983c076970 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f984c006fd0 tx=0x7f984c006d80 comp rx=0 tx=0).stop 2024-04-25T10:33:05.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:05.258+0000 7f985ef4d640 1 -- 172.21.15.106:0/3476233557 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98580c7e30 msgr2=0x7f98581f7010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:05.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:05.258+0000 7f985ef4d640 1 --2- 172.21.15.106:0/3476233557 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98580c7e30 0x7f98581f7010 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f984402a180 tx=0x7f98440046c0 comp rx=0 tx=0).stop 2024-04-25T10:33:05.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:05.258+0000 7f985ef4d640 1 -- 172.21.15.106:0/3476233557 shutdown_connections 2024-04-25T10:33:05.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:05.258+0000 7f985ef4d640 1 --2- 172.21.15.106:0/3476233557 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f983c074460 0x7f983c076970 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:05.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:05.258+0000 7f985ef4d640 1 --2- 172.21.15.106:0/3476233557 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f98580c8d90 0x7f98581e92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:05.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:05.258+0000 7f985ef4d640 1 --2- 172.21.15.106:0/3476233557 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98580c7e30 0x7f98581f7010 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:05.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:05.258+0000 7f985ef4d640 1 --2- 172.21.15.106:0/3476233557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98580b4ce0 0x7f985814f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:05.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:05.258+0000 7f985ef4d640 1 -- 172.21.15.106:0/3476233557 >> 172.21.15.106:0/3476233557 conn(0x7f9858068b10 msgr2=0x7f985815bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:05.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:05.258+0000 7f985ef4d640 1 -- 172.21.15.106:0/3476233557 shutdown_connections 2024-04-25T10:33:05.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:05.259+0000 7f985ef4d640 1 -- 172.21.15.106:0/3476233557 wait complete. 2024-04-25T10:33:07.565 INFO:tasks.workunit.client.0.smithi106.stdout: 2024-04-25T10:33:07.565 INFO:tasks.workunit.client.0.smithi106.stdout: Children see throughput for 1 rewriters = 126543.27 kB/sec 2024-04-25T10:33:07.565 INFO:tasks.workunit.client.0.smithi106.stdout: Parent sees throughput for 1 rewriters = 126540.55 kB/sec 2024-04-25T10:33:07.565 INFO:tasks.workunit.client.0.smithi106.stdout: Min throughput per process = 126543.27 kB/sec 2024-04-25T10:33:07.565 INFO:tasks.workunit.client.0.smithi106.stdout: Max throughput per process = 126543.27 kB/sec 2024-04-25T10:33:07.565 INFO:tasks.workunit.client.0.smithi106.stdout: Avg throughput per process = 126543.27 kB/sec 2024-04-25T10:33:07.565 INFO:tasks.workunit.client.0.smithi106.stdout: Min xfer = 1048576.00 kB 2024-04-25T10:33:08.274 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T10:33:08.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:08.394+0000 7fa578ef0640 1 Processor -- start 2024-04-25T10:33:08.396 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:08.394+0000 7fa578ef0640 1 -- start start 2024-04-25T10:33:08.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:08.394+0000 7fa578ef0640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fa574153180 0x7fa57406ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:08.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:08.395+0000 7fa578ef0640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa574153590 0x7fa574070510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:08.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:08.395+0000 7fa578ef0640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa574074790 con 0x7fa5740c9c00 2024-04-25T10:33:08.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:08.395+0000 7fa578ef0640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fa5740741d0 con 0x7fa574153180 2024-04-25T10:33:08.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:08.395+0000 7fa578ef0640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa5740744b0 con 0x7fa574153590 2024-04-25T10:33:08.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:08.395+0000 7fa57259c640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fa5740c9c00 0x7fa5740c9190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:37298/0 (socket says 172.21.15.106:37298) 2024-04-25T10:33:08.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:08.395+0000 7fa56bfff640 1 -- >> v2:172.21.15.134:3300/0 conn(0x7fa574153180 msgr2=0x7fa57406ffd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-04-25T10:33:08.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:08.395+0000 7fa57259c640 1 -- 172.21.15.106:0/4181488724 learned_addr learned my addr 172.21.15.106:0/4181488724 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:33:08.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:08.395+0000 7fa56bfff640 1 --2- 172.21.15.106:0/4181488724 >> v2:172.21.15.134:3300/0 conn(0x7fa574153180 0x7fa57406ffd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:33:08.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:08.395+0000 7fa572d9d640 1 --2- 172.21.15.106:0/4181488724 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa574153590 0x7fa574070510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:08.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:08.395+0000 7fa572d9d640 1 -- 172.21.15.106:0/4181488724 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa574153590 msgr2=0x7fa574070510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T10:33:08.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:08.395+0000 7fa572d9d640 1 -- 172.21.15.106:0/4181488724 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa574153590 msgr2=0x7fa574070510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:33:08.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:08.395+0000 7fa572d9d640 1 --2- 172.21.15.106:0/4181488724 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa574153590 0x7fa574070510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:33:08.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:08.395+0000 7fa572d9d640 1 --2- 172.21.15.106:0/4181488724 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa574153590 0x7fa574070510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:33:08.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:08.596+0000 7fa56bfff640 1 -- 172.21.15.106:0/4181488724 >> v2:172.21.15.134:3300/0 conn(0x7fa574153180 msgr2=0x7fa57406ffd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-04-25T10:33:08.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:08.596+0000 7fa56bfff640 1 --2- 172.21.15.106:0/4181488724 >> v2:172.21.15.134:3300/0 conn(0x7fa574153180 0x7fa57406ffd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T10:33:08.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:08.596+0000 7fa572d9d640 1 --2- 172.21.15.106:0/4181488724 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa574153590 0x7fa574070510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:08.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:08.596+0000 7fa572d9d640 1 -- 172.21.15.106:0/4181488724 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa574153590 msgr2=0x7fa574070510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T10:33:08.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:08.596+0000 7fa572d9d640 1 -- 172.21.15.106:0/4181488724 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa574153590 msgr2=0x7fa574070510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:33:08.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:08.596+0000 7fa572d9d640 1 --2- 172.21.15.106:0/4181488724 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa574153590 0x7fa574070510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:33:08.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:08.596+0000 7fa572d9d640 1 --2- 172.21.15.106:0/4181488724 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa574153590 0x7fa574070510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T10:33:08.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:08.997+0000 7fa56bfff640 1 -- 172.21.15.106:0/4181488724 >> v2:172.21.15.134:3300/0 conn(0x7fa574153180 msgr2=0x7fa57406ffd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-04-25T10:33:08.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:08.997+0000 7fa56bfff640 1 --2- 172.21.15.106:0/4181488724 >> v2:172.21.15.134:3300/0 conn(0x7fa574153180 0x7fa57406ffd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T10:33:08.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:08.997+0000 7fa572d9d640 1 --2- 172.21.15.106:0/4181488724 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa574153590 0x7fa574070510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:08.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:08.998+0000 7fa572d9d640 1 -- 172.21.15.106:0/4181488724 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa574153590 msgr2=0x7fa574070510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T10:33:08.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:08.998+0000 7fa572d9d640 1 -- 172.21.15.106:0/4181488724 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa574153590 msgr2=0x7fa574070510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:33:08.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:08.998+0000 7fa572d9d640 1 --2- 172.21.15.106:0/4181488724 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa574153590 0x7fa574070510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:33:08.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:08.998+0000 7fa572d9d640 1 --2- 172.21.15.106:0/4181488724 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa574153590 0x7fa574070510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T10:33:09.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:09.798+0000 7fa56bfff640 1 -- 172.21.15.106:0/4181488724 >> v2:172.21.15.134:3300/0 conn(0x7fa574153180 msgr2=0x7fa57406ffd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-04-25T10:33:09.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:09.798+0000 7fa56bfff640 1 --2- 172.21.15.106:0/4181488724 >> v2:172.21.15.134:3300/0 conn(0x7fa574153180 0x7fa57406ffd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T10:33:09.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:09.799+0000 7fa572d9d640 1 --2- 172.21.15.106:0/4181488724 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa574153590 0x7fa574070510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:09.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:09.799+0000 7fa572d9d640 1 -- 172.21.15.106:0/4181488724 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa574153590 msgr2=0x7fa574070510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T10:33:09.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:09.799+0000 7fa572d9d640 1 -- 172.21.15.106:0/4181488724 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa574153590 msgr2=0x7fa574070510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:33:09.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:09.799+0000 7fa572d9d640 1 --2- 172.21.15.106:0/4181488724 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa574153590 0x7fa574070510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:33:09.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:09.799+0000 7fa572d9d640 1 --2- 172.21.15.106:0/4181488724 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa574153590 0x7fa574070510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T10:33:11.317 INFO:tasks.workunit.client.0.smithi106.stdout: 2024-04-25T10:33:11.317 INFO:tasks.workunit.client.0.smithi106.stdout: Children see throughput for 1 readers = 7684300.50 kB/sec 2024-04-25T10:33:11.317 INFO:tasks.workunit.client.0.smithi106.stdout: Parent sees throughput for 1 readers = 7668677.68 kB/sec 2024-04-25T10:33:11.317 INFO:tasks.workunit.client.0.smithi106.stdout: Min throughput per process = 7684300.50 kB/sec 2024-04-25T10:33:11.317 INFO:tasks.workunit.client.0.smithi106.stdout: Max throughput per process = 7684300.50 kB/sec 2024-04-25T10:33:11.317 INFO:tasks.workunit.client.0.smithi106.stdout: Avg throughput per process = 7684300.50 kB/sec 2024-04-25T10:33:11.317 INFO:tasks.workunit.client.0.smithi106.stdout: Min xfer = 1048576.00 kB 2024-04-25T10:33:11.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:11.396+0000 7fa570d99640 1 -- 172.21.15.106:0/4181488724 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa574153590 msgr2=0x7fa574070510 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:33:11.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:11.396+0000 7fa570d99640 1 --2- 172.21.15.106:0/4181488724 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa574153590 0x7fa574070510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:11.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:11.396+0000 7fa570d99640 1 -- 172.21.15.106:0/4181488724 >> v2:172.21.15.134:3300/0 conn(0x7fa574153180 msgr2=0x7fa57406ffd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:33:11.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:11.396+0000 7fa570d99640 1 --2- 172.21.15.106:0/4181488724 >> v2:172.21.15.134:3300/0 conn(0x7fa574153180 0x7fa57406ffd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:11.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:11.396+0000 7fa570d99640 1 -- 172.21.15.106:0/4181488724 >> v1:172.21.15.106:6789/0 conn(0x7fa5740c9c00 legacy=0x7fa5740c9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:11.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:11.396+0000 7fa570d99640 1 --2- 172.21.15.106:0/4181488724 >> v2:172.21.15.106:3300/0 conn(0x7fa554007450 0x7fa554009940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:11.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:11.396+0000 7fa570d99640 1 -- 172.21.15.106:0/4181488724 --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5740c5320 con 0x7fa5540036c0 2024-04-25T10:33:11.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:11.396+0000 7fa570d99640 1 -- 172.21.15.106:0/4181488724 --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fa5740741d0 con 0x7fa554007450 2024-04-25T10:33:11.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:11.396+0000 7fa56bfff640 1 --2- 172.21.15.106:0/4181488724 >> v2:172.21.15.106:3300/0 conn(0x7fa554007450 0x7fa554009940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:11.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:11.396+0000 7fa57259c640 1 -- 172.21.15.106:0/4181488724 >> v1:172.21.15.134:6789/0 conn(0x7fa5540036c0 legacy=0x7fa554005b90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-04-25T10:33:11.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:11.396+0000 7fa56bfff640 1 -- 172.21.15.106:0/4181488724 >> v2:172.21.15.106:3300/0 conn(0x7fa554007450 msgr2=0x7fa554009940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T10:33:11.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:11.396+0000 7fa56bfff640 1 -- 172.21.15.106:0/4181488724 >> v2:172.21.15.106:3300/0 conn(0x7fa554007450 msgr2=0x7fa554009940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:33:11.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:11.396+0000 7fa56bfff640 1 --2- 172.21.15.106:0/4181488724 >> v2:172.21.15.106:3300/0 conn(0x7fa554007450 0x7fa554009940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:33:11.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:11.396+0000 7fa56bfff640 1 --2- 172.21.15.106:0/4181488724 >> v2:172.21.15.106:3300/0 conn(0x7fa554007450 0x7fa554009940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:33:11.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:11.597+0000 7fa56bfff640 1 --2- 172.21.15.106:0/4181488724 >> v2:172.21.15.106:3300/0 conn(0x7fa554007450 0x7fa554009940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:11.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:11.597+0000 7fa57259c640 1 -- 172.21.15.106:0/4181488724 >> v1:172.21.15.134:6789/0 conn(0x7fa5540036c0 legacy=0x7fa554005b90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-04-25T10:33:11.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:11.597+0000 7fa56bfff640 1 -- 172.21.15.106:0/4181488724 >> v2:172.21.15.106:3300/0 conn(0x7fa554007450 msgr2=0x7fa554009940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T10:33:11.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:11.597+0000 7fa56bfff640 1 -- 172.21.15.106:0/4181488724 >> v2:172.21.15.106:3300/0 conn(0x7fa554007450 msgr2=0x7fa554009940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:33:11.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:11.597+0000 7fa56bfff640 1 --2- 172.21.15.106:0/4181488724 >> v2:172.21.15.106:3300/0 conn(0x7fa554007450 0x7fa554009940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:33:11.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:11.597+0000 7fa56bfff640 1 --2- 172.21.15.106:0/4181488724 >> v2:172.21.15.106:3300/0 conn(0x7fa554007450 0x7fa554009940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T10:33:11.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:11.998+0000 7fa56bfff640 1 --2- 172.21.15.106:0/4181488724 >> v2:172.21.15.106:3300/0 conn(0x7fa554007450 0x7fa554009940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:12.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:11.998+0000 7fa57259c640 1 -- 172.21.15.106:0/4181488724 >> v1:172.21.15.134:6789/0 conn(0x7fa5540036c0 legacy=0x7fa554005b90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-04-25T10:33:12.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:11.998+0000 7fa56bfff640 1 -- 172.21.15.106:0/4181488724 >> v2:172.21.15.106:3300/0 conn(0x7fa554007450 msgr2=0x7fa554009940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T10:33:12.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:11.998+0000 7fa56bfff640 1 -- 172.21.15.106:0/4181488724 >> v2:172.21.15.106:3300/0 conn(0x7fa554007450 msgr2=0x7fa554009940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:33:12.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:11.998+0000 7fa56bfff640 1 --2- 172.21.15.106:0/4181488724 >> v2:172.21.15.106:3300/0 conn(0x7fa554007450 0x7fa554009940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:33:12.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:11.998+0000 7fa56bfff640 1 --2- 172.21.15.106:0/4181488724 >> v2:172.21.15.106:3300/0 conn(0x7fa554007450 0x7fa554009940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T10:33:12.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:12.799+0000 7fa57259c640 1 -- 172.21.15.106:0/4181488724 >> v1:172.21.15.134:6789/0 conn(0x7fa5540036c0 legacy=0x7fa554005b90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-04-25T10:33:12.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:12.799+0000 7fa56bfff640 1 --2- 172.21.15.106:0/4181488724 >> v2:172.21.15.106:3300/0 conn(0x7fa554007450 0x7fa554009940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:12.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:12.799+0000 7fa56bfff640 1 -- 172.21.15.106:0/4181488724 >> v1:172.21.15.134:6789/0 conn(0x7fa5540036c0 legacy=0x7fa554005b90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:33:12.801 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:12.799+0000 7fa56bfff640 1 -- 172.21.15.106:0/4181488724 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa55400b040 con 0x7fa554007450 2024-04-25T10:33:12.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:12.800+0000 7fa56bfff640 1 --2- 172.21.15.106:0/4181488724 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa554007450 0x7fa554009940 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa55c003930 tx=0x7fa55c02b1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61231e9b8cd21755 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:12.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:12.800+0000 7fa571d9b640 1 -- 172.21.15.106:0/4181488724 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55c038070 con 0x7fa554007450 2024-04-25T10:33:12.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:12.800+0000 7fa571d9b640 1 -- 172.21.15.106:0/4181488724 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa55c033440 con 0x7fa554007450 2024-04-25T10:33:12.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:12.800+0000 7fa571d9b640 1 -- 172.21.15.106:0/4181488724 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55c033810 con 0x7fa554007450 2024-04-25T10:33:12.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:12.801+0000 7fa578ef0640 1 -- 172.21.15.106:0/4181488724 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa554007450 msgr2=0x7fa554009940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:12.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:12.801+0000 7fa578ef0640 1 --2- 172.21.15.106:0/4181488724 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa554007450 0x7fa554009940 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa55c003930 tx=0x7fa55c02b1e0 comp rx=0 tx=0).stop 2024-04-25T10:33:12.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:12.801+0000 7fa572d9d640 1 -- 172.21.15.106:0/4181488724 reap_dead start 2024-04-25T10:33:12.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:12.801+0000 7fa578ef0640 1 -- 172.21.15.106:0/4181488724 shutdown_connections 2024-04-25T10:33:12.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:12.801+0000 7fa578ef0640 1 --2- 172.21.15.106:0/4181488724 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa554007450 0x7fa554009940 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:12.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:12.802+0000 7fa578ef0640 1 -- 172.21.15.106:0/4181488724 >> 172.21.15.106:0/4181488724 conn(0x7fa57406c9b0 msgr2=0x7fa57406cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:12.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:12.802+0000 7fa578ef0640 1 -- 172.21.15.106:0/4181488724 shutdown_connections 2024-04-25T10:33:12.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:12.802+0000 7fa578ef0640 1 -- 172.21.15.106:0/4181488724 wait complete. 2024-04-25T10:33:12.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:12.804+0000 7fa578ef0640 1 Processor -- start 2024-04-25T10:33:12.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:12.804+0000 7fa578ef0640 1 -- start start 2024-04-25T10:33:12.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:12.805+0000 7fa578ef0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa554007450 0x7fa5740ce700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:12.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:12.805+0000 7fa578ef0640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa574141120 0x7fa574141e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:12.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:12.805+0000 7fa578ef0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa574152200 0x7fa5740ccf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:12.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:12.805+0000 7fa578ef0640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa5740ca5e0 con 0x7fa574152200 2024-04-25T10:33:12.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:12.805+0000 7fa578ef0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa5740c58e0 con 0x7fa554007450 2024-04-25T10:33:12.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:12.805+0000 7fa578ef0640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa5740caca0 con 0x7fa574141120 2024-04-25T10:33:12.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:12.805+0000 7fa57259c640 1 -- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa554007450 msgr2=0x7fa5740ce700 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3300/0 2024-04-25T10:33:12.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:12.806+0000 7fa57259c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa554007450 0x7fa5740ce700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:33:12.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:12.806+0000 7fa572d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa574152200 0x7fa5740ccf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:12.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:12.806+0000 7fa572d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa574152200 0x7fa5740ccf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54126/0 (socket says 172.21.15.106:54126) 2024-04-25T10:33:12.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:12.806+0000 7fa572d9d640 1 -- 172.21.15.106:0/994785756 learned_addr learned my addr 172.21.15.106:0/994785756 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:33:12.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:12.806+0000 7fa56bfff640 1 --2- 172.21.15.106:0/994785756 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa574141120 0x7fa574141e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:12.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:12.806+0000 7fa572d9d640 1 -- 172.21.15.106:0/994785756 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa574141120 msgr2=0x7fa574141e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:12.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:12.806+0000 7fa572d9d640 1 --2- 172.21.15.106:0/994785756 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa574141120 0x7fa574141e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:12.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:12.806+0000 7fa572d9d640 1 -- 172.21.15.106:0/994785756 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa554007450 msgr2=0x7fa5740ce700 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:33:12.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:12.806+0000 7fa572d9d640 1 --2- 172.21.15.106:0/994785756 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa554007450 0x7fa5740ce700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:12.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:12.806+0000 7fa572d9d640 1 -- 172.21.15.106:0/994785756 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5741423b0 con 0x7fa574152200 2024-04-25T10:33:12.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:12.806+0000 7fa572d9d640 1 --2- 172.21.15.106:0/994785756 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa574152200 0x7fa5740ccf90 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa57406fb50 tx=0x7fa56c015f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:12.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:12.806+0000 7fa56affd640 1 -- 172.21.15.106:0/994785756 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa56c022070 con 0x7fa574152200 2024-04-25T10:33:12.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:12.807+0000 7fa56affd640 1 -- 172.21.15.106:0/994785756 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa56c01d3f0 con 0x7fa574152200 2024-04-25T10:33:12.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:12.807+0000 7fa578ef0640 1 -- 172.21.15.106:0/994785756 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa574140e40 con 0x7fa574152200 2024-04-25T10:33:12.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:12.807+0000 7fa56affd640 1 -- 172.21.15.106:0/994785756 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa56c01d7e0 con 0x7fa574152200 2024-04-25T10:33:12.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:12.807+0000 7fa578ef0640 1 -- 172.21.15.106:0/994785756 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa57413ff50 con 0x7fa574152200 2024-04-25T10:33:12.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:12.808+0000 7fa578ef0640 1 -- 172.21.15.106:0/994785756 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5740cc090 con 0x7fa574152200 2024-04-25T10:33:12.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:12.808+0000 7fa56affd640 1 -- 172.21.15.106:0/994785756 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa56c025070 con 0x7fa574152200 2024-04-25T10:33:12.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:12.809+0000 7fa56affd640 1 --2- 172.21.15.106:0/994785756 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fa554075340 0x7fa554077850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:12.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:12.809+0000 7fa57259c640 1 --2- 172.21.15.106:0/994785756 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fa554075340 0x7fa554077850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:12.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:12.809+0000 7fa56affd640 1 -- 172.21.15.106:0/994785756 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fa56c097610 con 0x7fa574152200 2024-04-25T10:33:12.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:12.810+0000 7fa57259c640 1 --2- 172.21.15.106:0/994785756 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fa554075340 0x7fa554077850 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa564005880 tx=0x7fa564007a10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:12.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:12.814+0000 7fa56affd640 1 -- 172.21.15.106:0/994785756 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa5740cc090 con 0x7fa574152200 2024-04-25T10:33:13.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.061+0000 7fa578ef0640 1 -- 172.21.15.106:0/994785756 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fa57406a3d0 con 0x7fa574152200 2024-04-25T10:33:13.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.062+0000 7fa56affd640 1 -- 172.21.15.106:0/994785756 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1562 (secure 0 0 0) 0x7fa56c01a070 con 0x7fa574152200 2024-04-25T10:33:13.063 INFO:teuthology.orchestra.run.smithi106.stdout:{"election_epoch":20,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"3ee77985-e8e4-41cf-81e9-7c774cf7891f","modified":"2024-04-25T10:29:33.446228Z","created":"2024-04-25T10:29:33.446228Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:3300","nonce":0},{"type":"v1","addr":"172.21.15.106:6789","nonce":0}]},"addr":"172.21.15.106:6789/0","public_addr":"172.21.15.106:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T10:33:13.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.064+0000 7fa578ef0640 1 -- 172.21.15.106:0/994785756 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fa554075340 msgr2=0x7fa554077850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:13.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.064+0000 7fa578ef0640 1 --2- 172.21.15.106:0/994785756 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fa554075340 0x7fa554077850 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa564005880 tx=0x7fa564007a10 comp rx=0 tx=0).stop 2024-04-25T10:33:13.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.064+0000 7fa578ef0640 1 -- 172.21.15.106:0/994785756 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa574152200 msgr2=0x7fa5740ccf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:13.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.064+0000 7fa578ef0640 1 --2- 172.21.15.106:0/994785756 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa574152200 0x7fa5740ccf90 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa57406fb50 tx=0x7fa56c015f00 comp rx=0 tx=0).stop 2024-04-25T10:33:13.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.064+0000 7fa578ef0640 1 -- 172.21.15.106:0/994785756 shutdown_connections 2024-04-25T10:33:13.065 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.064+0000 7fa578ef0640 1 --2- 172.21.15.106:0/994785756 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fa554075340 0x7fa554077850 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:13.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.064+0000 7fa578ef0640 1 --2- 172.21.15.106:0/994785756 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa574152200 0x7fa5740ccf90 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:13.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.064+0000 7fa578ef0640 1 --2- 172.21.15.106:0/994785756 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa574141120 0x7fa574141e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:13.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.064+0000 7fa578ef0640 1 --2- 172.21.15.106:0/994785756 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa554007450 0x7fa5740ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:13.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.064+0000 7fa578ef0640 1 -- 172.21.15.106:0/994785756 >> 172.21.15.106:0/994785756 conn(0x7fa57406c9b0 msgr2=0x7fa574144880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:13.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.064+0000 7fa578ef0640 1 -- 172.21.15.106:0/994785756 shutdown_connections 2024-04-25T10:33:13.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.064+0000 7fa578ef0640 1 -- 172.21.15.106:0/994785756 wait complete. 2024-04-25T10:33:13.076 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-04-25T10:33:13.076 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-04-25T10:33:13.076 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-25T10:33:13.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.183+0000 7f4ce2b7d640 1 Processor -- start 2024-04-25T10:33:13.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.183+0000 7f4ce2b7d640 1 -- start start 2024-04-25T10:33:13.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.185+0000 7f4ce2b7d640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f4cdc0c9c00 0x7f4cdc0c9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:13.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.185+0000 7f4ce2b7d640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4cdc0c96d0 con 0x7f4cdc153590 2024-04-25T10:33:13.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.185+0000 7f4ce2b7d640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4cdc074790 con 0x7f4cdc153180 2024-04-25T10:33:13.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.185+0000 7f4ce2b7d640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f4cdc0741d0 con 0x7f4cdc0c9c00 2024-04-25T10:33:13.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.185+0000 7f4ce111a640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f4cdc153590 0x7f4cdc159340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:37314/0 (socket says 172.21.15.106:37314) 2024-04-25T10:33:13.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.185+0000 7f4ce111a640 1 -- 172.21.15.106:0/3799813699 learned_addr learned my addr 172.21.15.106:0/3799813699 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:33:13.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.185+0000 7f4ce0919640 1 -- 172.21.15.106:0/3799813699 >> v2:172.21.15.134:3300/0 conn(0x7f4cdc0c9c00 msgr2=0x7f4cdc0c9190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-04-25T10:33:13.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.185+0000 7f4cdbfff640 1 -- 172.21.15.106:0/3799813699 >> v1:172.21.15.134:6789/0 conn(0x7f4cdc153180 legacy=0x7f4cdc06ffd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-04-25T10:33:13.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.185+0000 7f4ce0919640 1 --2- 172.21.15.106:0/3799813699 >> v2:172.21.15.134:3300/0 conn(0x7f4cdc0c9c00 0x7f4cdc0c9190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:33:13.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.185+0000 7f4cdb7fe640 1 -- 172.21.15.106:0/3799813699 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 798075896 0 0) 0x7f4cdc0c96d0 con 0x7f4cdc153590 2024-04-25T10:33:13.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.185+0000 7f4cdb7fe640 1 -- 172.21.15.106:0/3799813699 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4cc4004630 con 0x7f4cdc153590 2024-04-25T10:33:13.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.185+0000 7f4cdb7fe640 1 -- 172.21.15.106:0/3799813699 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3272155717 0 0) 0x7f4cc4004630 con 0x7f4cdc153590 2024-04-25T10:33:13.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.185+0000 7f4cdb7fe640 1 -- 172.21.15.106:0/3799813699 >> v2:172.21.15.134:3300/0 conn(0x7f4cdc0c9c00 msgr2=0x7f4cdc0c9190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:33:13.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.185+0000 7f4cdb7fe640 1 --2- 172.21.15.106:0/3799813699 >> v2:172.21.15.134:3300/0 conn(0x7f4cdc0c9c00 0x7f4cdc0c9190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:13.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.185+0000 7f4cdb7fe640 1 -- 172.21.15.106:0/3799813699 >> v1:172.21.15.134:6789/0 conn(0x7f4cdc153180 legacy=0x7f4cdc06ffd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:33:13.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.185+0000 7f4cdb7fe640 1 -- 172.21.15.106:0/3799813699 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cdc0744b0 con 0x7f4cdc153590 2024-04-25T10:33:13.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.185+0000 7f4cdb7fe640 1 -- 172.21.15.106:0/3799813699 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3776459350 0 0) 0x7f4cd40019f0 con 0x7f4cdc153590 2024-04-25T10:33:13.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.185+0000 7f4cdb7fe640 1 -- 172.21.15.106:0/3799813699 >> v1:172.21.15.106:6789/0 conn(0x7f4cdc153590 legacy=0x7f4cdc159340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:13.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.185+0000 7f4cdb7fe640 1 --2- 172.21.15.106:0/3799813699 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4cc4004ac0 0x7f4cc4004f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:13.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.185+0000 7f4cdb7fe640 1 --2- 172.21.15.106:0/3799813699 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4cc4005910 0x7f4cc4005d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:13.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.185+0000 7f4cdb7fe640 1 --2- 172.21.15.106:0/3799813699 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4cc4006460 0x7f4cc400cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:13.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.185+0000 7f4cdb7fe640 1 -- 172.21.15.106:0/3799813699 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4cdc0741d0 con 0x7f4cc4006460 2024-04-25T10:33:13.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.185+0000 7f4cdb7fe640 1 -- 172.21.15.106:0/3799813699 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc400d380 con 0x7f4cc4004ac0 2024-04-25T10:33:13.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.185+0000 7f4cdb7fe640 1 -- 172.21.15.106:0/3799813699 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f4cc400d580 con 0x7f4cc4005910 2024-04-25T10:33:13.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.185+0000 7f4ce111a640 1 --2- 172.21.15.106:0/3799813699 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4cc4006460 0x7f4cc400cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:13.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.185+0000 7f4ce0919640 1 -- 172.21.15.106:0/3799813699 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4cc4004ac0 msgr2=0x7f4cc4004f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-04-25T10:33:13.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.185+0000 7f4ce0919640 1 --2- 172.21.15.106:0/3799813699 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4cc4004ac0 0x7f4cc4004f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:33:13.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.186+0000 7f4cdbfff640 1 --2- 172.21.15.106:0/3799813699 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4cc4005910 0x7f4cc4005d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:13.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.186+0000 7f4ce111a640 1 -- 172.21.15.106:0/3799813699 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4cc4005910 msgr2=0x7f4cc4005d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:13.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.186+0000 7f4ce111a640 1 --2- 172.21.15.106:0/3799813699 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4cc4005910 0x7f4cc4005d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:13.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.186+0000 7f4ce111a640 1 -- 172.21.15.106:0/3799813699 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4cc4004ac0 msgr2=0x7f4cc4004f10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:33:13.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.186+0000 7f4ce111a640 1 --2- 172.21.15.106:0/3799813699 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4cc4004ac0 0x7f4cc4004f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:13.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.186+0000 7f4ce111a640 1 -- 172.21.15.106:0/3799813699 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cc4004630 con 0x7f4cc4006460 2024-04-25T10:33:13.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.186+0000 7f4ce111a640 1 -- 172.21.15.106:0/3799813699 reap_dead start 2024-04-25T10:33:13.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.186+0000 7f4ce111a640 1 --2- 172.21.15.106:0/3799813699 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4cc4006460 0x7f4cc400cad0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4cd400a810 tx=0x7f4cc4006940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d587b40f2a616680 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:13.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.186+0000 7f4cdb7fe640 1 -- 172.21.15.106:0/3799813699 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd400a460 con 0x7f4cc4006460 2024-04-25T10:33:13.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.187+0000 7f4cdb7fe640 1 -- 172.21.15.106:0/3799813699 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cd402f070 con 0x7f4cc4006460 2024-04-25T10:33:13.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.187+0000 7f4cdb7fe640 1 -- 172.21.15.106:0/3799813699 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd402bc50 con 0x7f4cc4006460 2024-04-25T10:33:13.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.187+0000 7f4ce2b7d640 1 -- 172.21.15.106:0/3799813699 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4cc4006460 msgr2=0x7f4cc400cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:13.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.187+0000 7f4ce2b7d640 1 --2- 172.21.15.106:0/3799813699 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4cc4006460 0x7f4cc400cad0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4cd400a810 tx=0x7f4cc4006940 comp rx=0 tx=0).stop 2024-04-25T10:33:13.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.187+0000 7f4ce2b7d640 1 -- 172.21.15.106:0/3799813699 shutdown_connections 2024-04-25T10:33:13.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.187+0000 7f4ce2b7d640 1 --2- 172.21.15.106:0/3799813699 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4cc4006460 0x7f4cc400cad0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:13.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.187+0000 7f4ce2b7d640 1 -- 172.21.15.106:0/3799813699 >> 172.21.15.106:0/3799813699 conn(0x7f4cdc06c9b0 msgr2=0x7f4cdc06cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:13.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.187+0000 7f4ce2b7d640 1 -- 172.21.15.106:0/3799813699 shutdown_connections 2024-04-25T10:33:13.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.187+0000 7f4ce2b7d640 1 -- 172.21.15.106:0/3799813699 wait complete. 2024-04-25T10:33:13.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.189+0000 7f4ce2b7d640 1 Processor -- start 2024-04-25T10:33:13.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.189+0000 7f4ce2b7d640 1 -- start start 2024-04-25T10:33:13.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.190+0000 7f4ce2b7d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4cc4006460 0x7f4cdc155f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:13.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.190+0000 7f4ce2b7d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4cdc1522c0 0x7f4cdc145580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:13.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.190+0000 7f4ce2b7d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4cdc14acb0 0x7f4cdc14b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:13.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.190+0000 7f4ce2b7d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4cdc0c58e0 con 0x7f4cdc1522c0 2024-04-25T10:33:13.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.190+0000 7f4ce2b7d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4cdc0c5600 con 0x7f4cdc14acb0 2024-04-25T10:33:13.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.190+0000 7f4ce2b7d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f4cdc0ca5e0 con 0x7f4cc4006460 2024-04-25T10:33:13.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.190+0000 7f4cdbfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4cdc1522c0 0x7f4cdc145580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:13.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.190+0000 7f4ce111a640 1 -- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4cdc14acb0 msgr2=0x7f4cdc14b0c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3300/0 2024-04-25T10:33:13.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.190+0000 7f4cdbfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4cdc1522c0 0x7f4cdc145580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54142/0 (socket says 172.21.15.106:54142) 2024-04-25T10:33:13.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.190+0000 7f4cdbfff640 1 -- 172.21.15.106:0/578002601 learned_addr learned my addr 172.21.15.106:0/578002601 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:33:13.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.190+0000 7f4ce111a640 1 --2- 172.21.15.106:0/578002601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4cdc14acb0 0x7f4cdc14b0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:33:13.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.191+0000 7f4ce0919640 1 --2- 172.21.15.106:0/578002601 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4cc4006460 0x7f4cdc155f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:13.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.191+0000 7f4cdbfff640 1 -- 172.21.15.106:0/578002601 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4cc4006460 msgr2=0x7f4cdc155f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:13.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.191+0000 7f4cdbfff640 1 --2- 172.21.15.106:0/578002601 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4cc4006460 0x7f4cdc155f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:13.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.191+0000 7f4cdbfff640 1 -- 172.21.15.106:0/578002601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4cdc14acb0 msgr2=0x7f4cdc14b0c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:33:13.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.191+0000 7f4cdbfff640 1 --2- 172.21.15.106:0/578002601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4cdc14acb0 0x7f4cdc14b0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:13.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.191+0000 7f4cdbfff640 1 -- 172.21.15.106:0/578002601 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cdc1564b0 con 0x7f4cdc1522c0 2024-04-25T10:33:13.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.191+0000 7f4ce0919640 1 --2- 172.21.15.106:0/578002601 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4cc4006460 0x7f4cdc155f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:33:13.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.191+0000 7f4cdbfff640 1 --2- 172.21.15.106:0/578002601 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4cdc1522c0 0x7f4cdc145580 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4cd000c640 tx=0x7f4cd000cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:13.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.191+0000 7f4cd97fa640 1 -- 172.21.15.106:0/578002601 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd0003cf0 con 0x7f4cdc1522c0 2024-04-25T10:33:13.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.191+0000 7f4cd97fa640 1 -- 172.21.15.106:0/578002601 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cd0016070 con 0x7f4cdc1522c0 2024-04-25T10:33:13.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.191+0000 7f4cd97fa640 1 -- 172.21.15.106:0/578002601 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd0005330 con 0x7f4cdc1522c0 2024-04-25T10:33:13.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.191+0000 7f4ce2b7d640 1 -- 172.21.15.106:0/578002601 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cdc154130 con 0x7f4cdc1522c0 2024-04-25T10:33:13.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.191+0000 7f4ce2b7d640 1 -- 172.21.15.106:0/578002601 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cdc1543d0 con 0x7f4cdc1522c0 2024-04-25T10:33:13.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.192+0000 7f4ce2b7d640 1 --2- 172.21.15.106:0/578002601 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4cdc155840 0x7f4cdc1f16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:13.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.192+0000 7f4ce2b7d640 1 -- 172.21.15.106:0/578002601 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4cdc14b600 con 0x7f4cdc155840 2024-04-25T10:33:13.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.192+0000 7f4ce0919640 1 --2- 172.21.15.106:0/578002601 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4cdc155840 0x7f4cdc1f16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:13.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.192+0000 7f4ce0919640 1 -- 172.21.15.106:0/578002601 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4cdc06a3d0 con 0x7f4cdc155840 2024-04-25T10:33:13.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.192+0000 7f4ce0919640 1 --2- 172.21.15.106:0/578002601 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4cdc155840 0x7f4cdc1f16f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc00d700 tx=0x7f4ccc00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:13.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.193+0000 7f4cd97fa640 1 -- 172.21.15.106:0/578002601 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ccc0039b0 con 0x7f4cdc155840 2024-04-25T10:33:13.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.193+0000 7f4cd97fa640 1 -- 172.21.15.106:0/578002601 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4cd00054d0 con 0x7f4cdc1522c0 2024-04-25T10:33:13.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.193+0000 7f4cd97fa640 1 --2- 172.21.15.106:0/578002601 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f4cc80743e0 0x7f4cc80768f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:13.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.193+0000 7f4ce111a640 1 --2- 172.21.15.106:0/578002601 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f4cc80743e0 0x7f4cc80768f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:13.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.193+0000 7f4cd97fa640 1 -- 172.21.15.106:0/578002601 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f4cd0094fe0 con 0x7f4cdc1522c0 2024-04-25T10:33:13.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.193+0000 7f4cd97fa640 1 -- 172.21.15.106:0/578002601 <== mon.0 v2:172.21.15.106:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f4ccc017070 con 0x7f4cdc155840 2024-04-25T10:33:13.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.193+0000 7f4cd97fa640 1 -- 172.21.15.106:0/578002601 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4cdc155840 msgr2=0x7f4cdc1f16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:13.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.193+0000 7f4cd97fa640 1 --2- 172.21.15.106:0/578002601 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4cdc155840 0x7f4cdc1f16f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc00d700 tx=0x7f4ccc00dbc0 comp rx=0 tx=0).stop 2024-04-25T10:33:13.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.193+0000 7f4cd97fa640 1 -- 172.21.15.106:0/578002601 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4cdc155840 msgr2=0x7f4cdc1f16f0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T10:33:13.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.193+0000 7f4cd97fa640 1 --2- 172.21.15.106:0/578002601 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4cdc155840 0x7f4cdc1f16f0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:13.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.194+0000 7f4ce111a640 1 --2- 172.21.15.106:0/578002601 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f4cc80743e0 0x7f4cc80768f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4cc400d350 tx=0x7f4cd4007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:13.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.200+0000 7f4ce2b7d640 1 --2- 172.21.15.106:0/578002601 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4cdc1f1c30 0x7f4cdc1f2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:13.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.200+0000 7f4ce2b7d640 1 -- 172.21.15.106:0/578002601 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4cdc1f2580 con 0x7f4cdc1f1c30 2024-04-25T10:33:13.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.200+0000 7f4ce0919640 1 --2- 172.21.15.106:0/578002601 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4cdc1f1c30 0x7f4cdc1f2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:13.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.200+0000 7f4ce0919640 1 -- 172.21.15.106:0/578002601 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f4cdc156b40 con 0x7f4cdc1f1c30 2024-04-25T10:33:13.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.200+0000 7f4ce0919640 1 --2- 172.21.15.106:0/578002601 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4cdc1f1c30 0x7f4cdc1f2040 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc016000 tx=0x7f4ccc0059d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:13.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.200+0000 7f4cd97fa640 1 -- 172.21.15.106:0/578002601 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ccc00b940 con 0x7f4cdc1f1c30 2024-04-25T10:33:13.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.201+0000 7f4cd97fa640 1 -- 172.21.15.106:0/578002601 <== mon.0 v2:172.21.15.106:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5060 (secure 0 0 0) 0x7f4ccc00b940 con 0x7f4cdc1f1c30 2024-04-25T10:33:13.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.201+0000 7f4cd97fa640 1 -- 172.21.15.106:0/578002601 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4cdc1f1c30 msgr2=0x7f4cdc1f2040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:13.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.201+0000 7f4cd97fa640 1 --2- 172.21.15.106:0/578002601 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4cdc1f1c30 0x7f4cdc1f2040 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc016000 tx=0x7f4ccc0059d0 comp rx=0 tx=0).stop 2024-04-25T10:33:13.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.201+0000 7f4cd97fa640 1 -- 172.21.15.106:0/578002601 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4cdc1f1c30 msgr2=0x7f4cdc1f2040 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T10:33:13.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.201+0000 7f4cd97fa640 1 --2- 172.21.15.106:0/578002601 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4cdc1f1c30 0x7f4cdc1f2040 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:13.203 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-25T10:33:13.203 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "a", 2024-04-25T10:33:13.203 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 0, 2024-04-25T10:33:13.203 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "leader", 2024-04-25T10:33:13.204 INFO:teuthology.orchestra.run.smithi106.stdout: "election_epoch": 20, 2024-04-25T10:33:13.204 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 185438, 2024-04-25T10:33:13.204 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum": [ 2024-04-25T10:33:13.204 INFO:teuthology.orchestra.run.smithi106.stdout: 0, 2024-04-25T10:33:13.204 INFO:teuthology.orchestra.run.smithi106.stdout: 2 2024-04-25T10:33:13.204 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:33:13.204 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_age": 0, 2024-04-25T10:33:13.204 INFO:teuthology.orchestra.run.smithi106.stdout: "features": { 2024-04-25T10:33:13.204 INFO:teuthology.orchestra.run.smithi106.stdout: "required_con": "2450521708007866372", 2024-04-25T10:33:13.204 INFO:teuthology.orchestra.run.smithi106.stdout: "required_mon": [ 2024-04-25T10:33:13.204 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:33:13.204 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:33:13.204 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:33:13.204 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:33:13.205 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:33:13.205 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:33:13.205 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:33:13.205 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:33:13.205 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:33:13.205 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:33:13.205 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:33:13.205 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:33:13.205 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_con": "4540701547738038271", 2024-04-25T10:33:13.205 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_mon": [ 2024-04-25T10:33:13.205 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:33:13.205 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:33:13.205 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:33:13.206 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:33:13.206 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:33:13.206 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:33:13.206 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:33:13.206 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:33:13.206 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:33:13.206 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:33:13.206 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:33:13.206 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:33:13.206 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:13.206 INFO:teuthology.orchestra.run.smithi106.stdout: "outside_quorum": [], 2024-04-25T10:33:13.206 INFO:teuthology.orchestra.run.smithi106.stdout: "extra_probe_peers": [], 2024-04-25T10:33:13.206 INFO:teuthology.orchestra.run.smithi106.stdout: "sync_provider": [], 2024-04-25T10:33:13.206 INFO:teuthology.orchestra.run.smithi106.stdout: "monmap": { 2024-04-25T10:33:13.207 INFO:teuthology.orchestra.run.smithi106.stdout: "epoch": 1, 2024-04-25T10:33:13.207 INFO:teuthology.orchestra.run.smithi106.stdout: "fsid": "3ee77985-e8e4-41cf-81e9-7c774cf7891f", 2024-04-25T10:33:13.207 INFO:teuthology.orchestra.run.smithi106.stdout: "modified": "2024-04-25T10:29:33.446228Z", 2024-04-25T10:33:13.207 INFO:teuthology.orchestra.run.smithi106.stdout: "created": "2024-04-25T10:29:33.446228Z", 2024-04-25T10:33:13.207 INFO:teuthology.orchestra.run.smithi106.stdout: "min_mon_release": 19, 2024-04-25T10:33:13.207 INFO:teuthology.orchestra.run.smithi106.stdout: "min_mon_release_name": "squid", 2024-04-25T10:33:13.207 INFO:teuthology.orchestra.run.smithi106.stdout: "election_strategy": 1, 2024-04-25T10:33:13.207 INFO:teuthology.orchestra.run.smithi106.stdout: "disallowed_leaders: ": "", 2024-04-25T10:33:13.207 INFO:teuthology.orchestra.run.smithi106.stdout: "stretch_mode": false, 2024-04-25T10:33:13.207 INFO:teuthology.orchestra.run.smithi106.stdout: "tiebreaker_mon": "", 2024-04-25T10:33:13.208 INFO:teuthology.orchestra.run.smithi106.stdout: "removed_ranks: ": "", 2024-04-25T10:33:13.208 INFO:teuthology.orchestra.run.smithi106.stdout: "features": { 2024-04-25T10:33:13.208 INFO:teuthology.orchestra.run.smithi106.stdout: "persistent": [ 2024-04-25T10:33:13.208 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:33:13.208 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:33:13.208 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:33:13.208 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:33:13.208 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:33:13.208 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:33:13.208 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:33:13.208 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:33:13.208 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:33:13.209 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:33:13.209 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:33:13.209 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:33:13.209 INFO:teuthology.orchestra.run.smithi106.stdout: "optional": [] 2024-04-25T10:33:13.209 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:13.209 INFO:teuthology.orchestra.run.smithi106.stdout: "mons": [ 2024-04-25T10:33:13.209 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:13.209 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 0, 2024-04-25T10:33:13.209 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "a", 2024-04-25T10:33:13.209 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:33:13.209 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:33:13.209 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:13.209 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:33:13.209 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:3300", 2024-04-25T10:33:13.210 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:33:13.210 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:13.210 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:13.210 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:33:13.210 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:6789", 2024-04-25T10:33:13.210 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:33:13.210 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:33:13.210 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:33:13.210 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:13.210 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:6789/0", 2024-04-25T10:33:13.210 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.106:6789/0", 2024-04-25T10:33:13.210 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:33:13.211 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:33:13.211 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:33:13.211 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:13.211 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:13.211 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 1, 2024-04-25T10:33:13.211 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "b", 2024-04-25T10:33:13.211 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:33:13.211 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:33:13.211 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:13.211 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:33:13.212 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:3300", 2024-04-25T10:33:13.212 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:33:13.212 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:13.212 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:13.212 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:33:13.212 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6789", 2024-04-25T10:33:13.212 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:33:13.212 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:33:13.212 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:33:13.212 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:13.212 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6789/0", 2024-04-25T10:33:13.212 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.134:6789/0", 2024-04-25T10:33:13.213 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:33:13.213 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:33:13.213 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:33:13.213 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:13.213 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:13.213 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 2, 2024-04-25T10:33:13.213 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "c", 2024-04-25T10:33:13.213 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:33:13.213 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:33:13.213 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:13.213 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:33:13.213 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:3301", 2024-04-25T10:33:13.213 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:33:13.214 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:13.214 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:13.214 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:33:13.214 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6790", 2024-04-25T10:33:13.214 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:33:13.214 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:33:13.214 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:33:13.214 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:13.214 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6790/0", 2024-04-25T10:33:13.214 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.134:6790/0", 2024-04-25T10:33:13.214 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:33:13.214 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:33:13.214 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:33:13.214 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:33:13.215 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:33:13.215 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:13.215 INFO:teuthology.orchestra.run.smithi106.stdout: "feature_map": { 2024-04-25T10:33:13.215 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": [ 2024-04-25T10:33:13.215 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:13.215 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:33:13.215 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:33:13.215 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 1 2024-04-25T10:33:13.215 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:33:13.215 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:33:13.215 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": [ 2024-04-25T10:33:13.215 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:13.215 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:33:13.215 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:33:13.216 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 6 2024-04-25T10:33:13.216 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:33:13.216 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:33:13.216 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": [ 2024-04-25T10:33:13.216 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:13.216 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:33:13.216 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:33:13.216 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 6 2024-04-25T10:33:13.216 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:33:13.216 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:33:13.216 INFO:teuthology.orchestra.run.smithi106.stdout: "client": [ 2024-04-25T10:33:13.216 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:13.216 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:33:13.216 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:33:13.217 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 7 2024-04-25T10:33:13.217 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:33:13.217 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:33:13.217 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": [ 2024-04-25T10:33:13.217 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:13.217 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:33:13.217 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:33:13.217 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 2 2024-04-25T10:33:13.217 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:33:13.217 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:33:13.217 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:13.217 INFO:teuthology.orchestra.run.smithi106.stdout: "stretch_mode": false 2024-04-25T10:33:13.217 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-25T10:33:13.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.201+0000 7f4ce2b7d640 1 -- 172.21.15.106:0/578002601 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f4cc80743e0 msgr2=0x7f4cc80768f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:13.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.201+0000 7f4ce2b7d640 1 --2- 172.21.15.106:0/578002601 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f4cc80743e0 0x7f4cc80768f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4cc400d350 tx=0x7f4cd4007000 comp rx=0 tx=0).stop 2024-04-25T10:33:13.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.201+0000 7f4ce2b7d640 1 -- 172.21.15.106:0/578002601 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4cdc1522c0 msgr2=0x7f4cdc145580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:13.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.201+0000 7f4ce2b7d640 1 --2- 172.21.15.106:0/578002601 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4cdc1522c0 0x7f4cdc145580 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4cd000c640 tx=0x7f4cd000cb00 comp rx=0 tx=0).stop 2024-04-25T10:33:13.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.201+0000 7f4ce111a640 1 -- 172.21.15.106:0/578002601 reap_dead start 2024-04-25T10:33:13.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.201+0000 7f4ce111a640 1 -- 172.21.15.106:0/578002601 reap_dead start 2024-04-25T10:33:13.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.201+0000 7f4ce2b7d640 1 -- 172.21.15.106:0/578002601 shutdown_connections 2024-04-25T10:33:13.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.201+0000 7f4ce2b7d640 1 -- 172.21.15.106:0/578002601 >> 172.21.15.106:0/578002601 conn(0x7f4cdc06c9b0 msgr2=0x7f4cdc070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:13.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.201+0000 7f4ce2b7d640 1 -- 172.21.15.106:0/578002601 shutdown_connections 2024-04-25T10:33:13.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.201+0000 7f4ce2b7d640 1 -- 172.21.15.106:0/578002601 wait complete. 2024-04-25T10:33:13.220 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-04-25T10:33:13.220 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-04-25T10:33:13.220 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-25T10:33:13.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.327+0000 7f4d30015640 1 Processor -- start 2024-04-25T10:33:13.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.327+0000 7f4d30015640 1 -- start start 2024-04-25T10:33:13.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.328+0000 7f4d30015640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4d280c61a0 0x7f4d28156f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:13.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.328+0000 7f4d30015640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f4d2806dc00 0x7f4d2815d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:13.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.328+0000 7f4d30015640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d2815cbd0 con 0x7f4d281601c0 2024-04-25T10:33:13.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.328+0000 7f4d30015640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f4d2815d200 con 0x7f4d2806dc00 2024-04-25T10:33:13.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.328+0000 7f4d30015640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f4d2815dbd0 con 0x7f4d280c61a0 2024-04-25T10:33:13.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.329+0000 7f4d2e5b2640 1 -- >> v1:172.21.15.134:6789/0 conn(0x7f4d281601c0 legacy=0x7f4d280cce20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-04-25T10:33:13.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.329+0000 7f4d2d5b0640 1 -- >> v2:172.21.15.134:3300/0 conn(0x7f4d2806dc00 msgr2=0x7f4d2815d690 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-04-25T10:33:13.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.329+0000 7f4d2d5b0640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f4d2806dc00 0x7f4d2815d690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:33:13.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.329+0000 7f4d2ddb1640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4d280c61a0 0x7f4d28156f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:13.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.329+0000 7f4d2ddb1640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4d280c61a0 0x7f4d28156f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.106:36508/0 (socket says 172.21.15.106:36508) 2024-04-25T10:33:13.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.329+0000 7f4d2ddb1640 1 -- 172.21.15.106:0/3762129826 learned_addr learned my addr 172.21.15.106:0/3762129826 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:33:13.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.329+0000 7f4d2ddb1640 1 -- 172.21.15.106:0/3762129826 >> v2:172.21.15.134:3300/0 conn(0x7f4d2806dc00 msgr2=0x7f4d2815d690 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:33:13.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.329+0000 7f4d2ddb1640 1 --2- 172.21.15.106:0/3762129826 >> v2:172.21.15.134:3300/0 conn(0x7f4d2806dc00 0x7f4d2815d690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:13.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.329+0000 7f4d2ddb1640 1 -- 172.21.15.106:0/3762129826 >> v1:172.21.15.134:6789/0 conn(0x7f4d281601c0 legacy=0x7f4d280cce20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:33:13.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.329+0000 7f4d2ddb1640 1 -- 172.21.15.106:0/3762129826 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d28160990 con 0x7f4d280c61a0 2024-04-25T10:33:13.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.330+0000 7f4d2ddb1640 1 --2- 172.21.15.106:0/3762129826 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4d280c61a0 0x7f4d28156f30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4d24009e50 tx=0x7f4d2402a6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d86e9cb72a306da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:13.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.330+0000 7f4d2cdaf640 1 -- 172.21.15.106:0/3762129826 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d24036070 con 0x7f4d280c61a0 2024-04-25T10:33:13.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.330+0000 7f4d2cdaf640 1 -- 172.21.15.106:0/3762129826 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d24031600 con 0x7f4d280c61a0 2024-04-25T10:33:13.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.330+0000 7f4d2cdaf640 1 -- 172.21.15.106:0/3762129826 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d240319d0 con 0x7f4d280c61a0 2024-04-25T10:33:13.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.331+0000 7f4d30015640 1 -- 172.21.15.106:0/3762129826 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4d280c61a0 msgr2=0x7f4d28156f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:13.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.331+0000 7f4d30015640 1 --2- 172.21.15.106:0/3762129826 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4d280c61a0 0x7f4d28156f30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4d24009e50 tx=0x7f4d2402a6f0 comp rx=0 tx=0).stop 2024-04-25T10:33:13.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.331+0000 7f4d30015640 1 -- 172.21.15.106:0/3762129826 shutdown_connections 2024-04-25T10:33:13.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.331+0000 7f4d30015640 1 --2- 172.21.15.106:0/3762129826 >> v2:172.21.15.134:3300/0 conn(0x7f4d2806dc00 0x7f4d2815d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:13.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.331+0000 7f4d30015640 1 --2- 172.21.15.106:0/3762129826 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4d280c61a0 0x7f4d28156f30 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:13.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.331+0000 7f4d30015640 1 -- 172.21.15.106:0/3762129826 >> 172.21.15.106:0/3762129826 conn(0x7f4d280727d0 msgr2=0x7f4d28072c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:13.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.331+0000 7f4d30015640 1 -- 172.21.15.106:0/3762129826 shutdown_connections 2024-04-25T10:33:13.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.331+0000 7f4d30015640 1 -- 172.21.15.106:0/3762129826 wait complete. 2024-04-25T10:33:13.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.333+0000 7f4d30015640 1 Processor -- start 2024-04-25T10:33:13.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.333+0000 7f4d30015640 1 -- start start 2024-04-25T10:33:13.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.334+0000 7f4d30015640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d2806dc00 0x7f4d281f69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:13.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.334+0000 7f4d30015640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d280c61a0 0x7f4d2814ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:13.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.334+0000 7f4d30015640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4d281601c0 0x7f4d281519b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:13.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.334+0000 7f4d30015640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4d2815ce90 con 0x7f4d280c61a0 2024-04-25T10:33:13.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.334+0000 7f4d30015640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4d28160f50 con 0x7f4d2806dc00 2024-04-25T10:33:13.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.334+0000 7f4d30015640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f4d2806d6d0 con 0x7f4d281601c0 2024-04-25T10:33:13.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.334+0000 7f4d2ddb1640 1 -- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d2806dc00 msgr2=0x7f4d281f69e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3300/0 2024-04-25T10:33:13.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.334+0000 7f4d2d5b0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d280c61a0 0x7f4d2814ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:13.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.334+0000 7f4d2ddb1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d2806dc00 0x7f4d281f69e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:33:13.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.334+0000 7f4d2d5b0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d280c61a0 0x7f4d2814ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54172/0 (socket says 172.21.15.106:54172) 2024-04-25T10:33:13.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.334+0000 7f4d2d5b0640 1 -- 172.21.15.106:0/3872049715 learned_addr learned my addr 172.21.15.106:0/3872049715 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:33:13.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.334+0000 7f4d2e5b2640 1 --2- 172.21.15.106:0/3872049715 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4d281601c0 0x7f4d281519b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:13.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.334+0000 7f4d2d5b0640 1 -- 172.21.15.106:0/3872049715 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4d281601c0 msgr2=0x7f4d281519b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:13.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.334+0000 7f4d2d5b0640 1 --2- 172.21.15.106:0/3872049715 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4d281601c0 0x7f4d281519b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:13.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.334+0000 7f4d2d5b0640 1 -- 172.21.15.106:0/3872049715 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d2806dc00 msgr2=0x7f4d281f69e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:33:13.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.334+0000 7f4d2d5b0640 1 --2- 172.21.15.106:0/3872049715 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d2806dc00 0x7f4d281f69e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:13.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.334+0000 7f4d2d5b0640 1 -- 172.21.15.106:0/3872049715 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d28151240 con 0x7f4d280c61a0 2024-04-25T10:33:13.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.335+0000 7f4d2d5b0640 1 --2- 172.21.15.106:0/3872049715 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d280c61a0 0x7f4d2814ea40 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f4d1800b580 tx=0x7f4d1800ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:13.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.335+0000 7f4d1e7fc640 1 -- 172.21.15.106:0/3872049715 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d18003cf0 con 0x7f4d280c61a0 2024-04-25T10:33:13.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.335+0000 7f4d1e7fc640 1 -- 172.21.15.106:0/3872049715 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d18015070 con 0x7f4d280c61a0 2024-04-25T10:33:13.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.335+0000 7f4d30015640 1 -- 172.21.15.106:0/3872049715 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d28151520 con 0x7f4d280c61a0 2024-04-25T10:33:13.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.335+0000 7f4d1e7fc640 1 -- 172.21.15.106:0/3872049715 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d18005330 con 0x7f4d280c61a0 2024-04-25T10:33:13.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.335+0000 7f4d30015640 1 -- 172.21.15.106:0/3872049715 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d281f64c0 con 0x7f4d280c61a0 2024-04-25T10:33:13.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.336+0000 7f4d30015640 1 --2- 172.21.15.106:0/3872049715 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4d2815e7a0 0x7f4d28151ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:13.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.336+0000 7f4d30015640 1 -- 172.21.15.106:0/3872049715 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f4d281f6f20 con 0x7f4d2815e7a0 2024-04-25T10:33:13.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.336+0000 7f4d2ddb1640 1 --2- 172.21.15.106:0/3872049715 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4d2815e7a0 0x7f4d28151ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:13.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.336+0000 7f4d2ddb1640 1 -- 172.21.15.106:0/3872049715 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4d2806a3d0 con 0x7f4d2815e7a0 2024-04-25T10:33:13.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.336+0000 7f4d2ddb1640 1 --2- 172.21.15.106:0/3872049715 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4d2815e7a0 0x7f4d28151ef0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4d280703d0 tx=0x7f4d24004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:13.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.336+0000 7f4d1e7fc640 1 -- 172.21.15.106:0/3872049715 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d24036040 con 0x7f4d2815e7a0 2024-04-25T10:33:13.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.337+0000 7f4d1e7fc640 1 -- 172.21.15.106:0/3872049715 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4d180054d0 con 0x7f4d280c61a0 2024-04-25T10:33:13.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.337+0000 7f4d1e7fc640 1 --2- 172.21.15.106:0/3872049715 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f4d0c0743b0 0x7f4d0c0768c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:13.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.337+0000 7f4d2e5b2640 1 --2- 172.21.15.106:0/3872049715 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f4d0c0743b0 0x7f4d0c0768c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:13.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.337+0000 7f4d1e7fc640 1 -- 172.21.15.106:0/3872049715 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f4d18094f50 con 0x7f4d280c61a0 2024-04-25T10:33:13.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.337+0000 7f4d1e7fc640 1 -- 172.21.15.106:0/3872049715 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f4d24003990 con 0x7f4d2815e7a0 2024-04-25T10:33:13.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.337+0000 7f4d1e7fc640 1 -- 172.21.15.106:0/3872049715 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4d2815e7a0 msgr2=0x7f4d28151ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:13.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.337+0000 7f4d1e7fc640 1 --2- 172.21.15.106:0/3872049715 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4d2815e7a0 0x7f4d28151ef0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4d280703d0 tx=0x7f4d24004a40 comp rx=0 tx=0).stop 2024-04-25T10:33:13.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.337+0000 7f4d1e7fc640 1 -- 172.21.15.106:0/3872049715 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4d2815e7a0 msgr2=0x7f4d28151ef0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T10:33:13.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.337+0000 7f4d1e7fc640 1 --2- 172.21.15.106:0/3872049715 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4d2815e7a0 0x7f4d28151ef0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:13.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.338+0000 7f4d2e5b2640 1 --2- 172.21.15.106:0/3872049715 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f4d0c0743b0 0x7f4d0c0768c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4d28160140 tx=0x7f4d200090e0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:13.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.344+0000 7f4d30015640 1 --2- 172.21.15.106:0/3872049715 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4d28156740 0x7f4d2814d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:13.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.344+0000 7f4d30015640 1 -- 172.21.15.106:0/3872049715 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f4d28145860 con 0x7f4d28156740 2024-04-25T10:33:13.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.344+0000 7f4d2ddb1640 1 --2- 172.21.15.106:0/3872049715 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4d28156740 0x7f4d2814d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:13.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.344+0000 7f4d2ddb1640 1 -- 172.21.15.106:0/3872049715 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f4d281516f0 con 0x7f4d28156740 2024-04-25T10:33:13.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.345+0000 7f4d2ddb1640 1 --2- 172.21.15.106:0/3872049715 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4d28156740 0x7f4d2814d200 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4d240093e0 tx=0x7f4d24009810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:13.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.345+0000 7f4d1e7fc640 1 -- 172.21.15.106:0/3872049715 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d2403c9c0 con 0x7f4d28156740 2024-04-25T10:33:13.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.345+0000 7f4d1e7fc640 1 -- 172.21.15.106:0/3872049715 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4717 (secure 0 0 0) 0x7f4d2403c9c0 con 0x7f4d28156740 2024-04-25T10:33:13.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.345+0000 7f4d1e7fc640 1 -- 172.21.15.106:0/3872049715 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4d28156740 msgr2=0x7f4d2814d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:13.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.345+0000 7f4d1e7fc640 1 --2- 172.21.15.106:0/3872049715 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4d28156740 0x7f4d2814d200 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4d240093e0 tx=0x7f4d24009810 comp rx=0 tx=0).stop 2024-04-25T10:33:13.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.345+0000 7f4d1e7fc640 1 -- 172.21.15.106:0/3872049715 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4d28156740 msgr2=0x7f4d2814d200 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T10:33:13.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.345+0000 7f4d1e7fc640 1 --2- 172.21.15.106:0/3872049715 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4d28156740 0x7f4d2814d200 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:13.347 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-25T10:33:13.347 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "c", 2024-04-25T10:33:13.347 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 2, 2024-04-25T10:33:13.347 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "peon", 2024-04-25T10:33:13.348 INFO:teuthology.orchestra.run.smithi106.stdout: "election_epoch": 20, 2024-04-25T10:33:13.348 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 32268, 2024-04-25T10:33:13.348 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum": [ 2024-04-25T10:33:13.348 INFO:teuthology.orchestra.run.smithi106.stdout: 0, 2024-04-25T10:33:13.348 INFO:teuthology.orchestra.run.smithi106.stdout: 2 2024-04-25T10:33:13.348 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:33:13.348 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_age": 1, 2024-04-25T10:33:13.348 INFO:teuthology.orchestra.run.smithi106.stdout: "features": { 2024-04-25T10:33:13.348 INFO:teuthology.orchestra.run.smithi106.stdout: "required_con": "2450521708007866372", 2024-04-25T10:33:13.348 INFO:teuthology.orchestra.run.smithi106.stdout: "required_mon": [ 2024-04-25T10:33:13.348 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:33:13.348 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:33:13.348 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:33:13.348 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:33:13.349 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:33:13.349 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:33:13.349 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:33:13.349 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:33:13.349 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:33:13.349 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:33:13.349 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:33:13.349 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:33:13.349 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_con": "4540701547738038271", 2024-04-25T10:33:13.349 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_mon": [ 2024-04-25T10:33:13.349 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:33:13.349 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:33:13.349 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:33:13.350 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:33:13.350 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:33:13.350 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:33:13.350 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:33:13.350 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:33:13.350 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:33:13.350 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:33:13.350 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:33:13.350 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:33:13.350 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:13.350 INFO:teuthology.orchestra.run.smithi106.stdout: "outside_quorum": [], 2024-04-25T10:33:13.350 INFO:teuthology.orchestra.run.smithi106.stdout: "extra_probe_peers": [], 2024-04-25T10:33:13.350 INFO:teuthology.orchestra.run.smithi106.stdout: "sync_provider": [], 2024-04-25T10:33:13.350 INFO:teuthology.orchestra.run.smithi106.stdout: "monmap": { 2024-04-25T10:33:13.351 INFO:teuthology.orchestra.run.smithi106.stdout: "epoch": 1, 2024-04-25T10:33:13.351 INFO:teuthology.orchestra.run.smithi106.stdout: "fsid": "3ee77985-e8e4-41cf-81e9-7c774cf7891f", 2024-04-25T10:33:13.351 INFO:teuthology.orchestra.run.smithi106.stdout: "modified": "2024-04-25T10:29:33.446228Z", 2024-04-25T10:33:13.351 INFO:teuthology.orchestra.run.smithi106.stdout: "created": "2024-04-25T10:29:33.446228Z", 2024-04-25T10:33:13.351 INFO:teuthology.orchestra.run.smithi106.stdout: "min_mon_release": 19, 2024-04-25T10:33:13.351 INFO:teuthology.orchestra.run.smithi106.stdout: "min_mon_release_name": "squid", 2024-04-25T10:33:13.351 INFO:teuthology.orchestra.run.smithi106.stdout: "election_strategy": 1, 2024-04-25T10:33:13.351 INFO:teuthology.orchestra.run.smithi106.stdout: "disallowed_leaders: ": "", 2024-04-25T10:33:13.351 INFO:teuthology.orchestra.run.smithi106.stdout: "stretch_mode": false, 2024-04-25T10:33:13.351 INFO:teuthology.orchestra.run.smithi106.stdout: "tiebreaker_mon": "", 2024-04-25T10:33:13.351 INFO:teuthology.orchestra.run.smithi106.stdout: "removed_ranks: ": "", 2024-04-25T10:33:13.351 INFO:teuthology.orchestra.run.smithi106.stdout: "features": { 2024-04-25T10:33:13.351 INFO:teuthology.orchestra.run.smithi106.stdout: "persistent": [ 2024-04-25T10:33:13.351 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:33:13.352 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:33:13.352 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:33:13.352 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:33:13.352 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:33:13.352 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:33:13.352 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:33:13.352 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:33:13.352 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:33:13.352 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:33:13.352 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:33:13.352 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:33:13.352 INFO:teuthology.orchestra.run.smithi106.stdout: "optional": [] 2024-04-25T10:33:13.352 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:13.352 INFO:teuthology.orchestra.run.smithi106.stdout: "mons": [ 2024-04-25T10:33:13.353 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:13.353 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 0, 2024-04-25T10:33:13.353 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "a", 2024-04-25T10:33:13.353 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:33:13.353 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:33:13.353 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:13.353 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:33:13.353 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:3300", 2024-04-25T10:33:13.353 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:33:13.353 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:13.353 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:13.353 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:33:13.353 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:6789", 2024-04-25T10:33:13.353 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:33:13.354 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:33:13.354 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:33:13.354 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:13.354 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:6789/0", 2024-04-25T10:33:13.354 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.106:6789/0", 2024-04-25T10:33:13.354 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:33:13.354 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:33:13.354 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:33:13.354 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:13.354 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:13.354 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 1, 2024-04-25T10:33:13.354 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "b", 2024-04-25T10:33:13.354 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:33:13.354 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:33:13.355 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:13.355 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:33:13.355 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:3300", 2024-04-25T10:33:13.355 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:33:13.355 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:13.355 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:13.355 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:33:13.355 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6789", 2024-04-25T10:33:13.355 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:33:13.355 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:33:13.355 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:33:13.355 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:13.355 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6789/0", 2024-04-25T10:33:13.356 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.134:6789/0", 2024-04-25T10:33:13.356 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:33:13.356 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:33:13.356 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:33:13.356 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:13.356 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:13.356 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 2, 2024-04-25T10:33:13.356 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "c", 2024-04-25T10:33:13.356 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:33:13.356 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:33:13.356 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:13.356 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:33:13.356 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:3301", 2024-04-25T10:33:13.356 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:33:13.357 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:13.357 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:13.357 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:33:13.357 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6790", 2024-04-25T10:33:13.357 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:33:13.357 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:33:13.357 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:33:13.357 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:13.357 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6790/0", 2024-04-25T10:33:13.357 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.134:6790/0", 2024-04-25T10:33:13.357 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:33:13.357 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:33:13.357 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:33:13.357 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:33:13.358 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:33:13.358 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:13.358 INFO:teuthology.orchestra.run.smithi106.stdout: "feature_map": { 2024-04-25T10:33:13.358 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": [ 2024-04-25T10:33:13.358 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:13.358 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:33:13.358 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:33:13.358 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 1 2024-04-25T10:33:13.358 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:33:13.358 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:33:13.358 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": [ 2024-04-25T10:33:13.358 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:13.358 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:33:13.358 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:33:13.359 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 2 2024-04-25T10:33:13.359 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:33:13.359 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:33:13.359 INFO:teuthology.orchestra.run.smithi106.stdout: "client": [ 2024-04-25T10:33:13.359 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:13.359 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:33:13.359 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:33:13.359 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 1 2024-04-25T10:33:13.359 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:33:13.359 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:33:13.359 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:13.359 INFO:teuthology.orchestra.run.smithi106.stdout: "stretch_mode": false 2024-04-25T10:33:13.359 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-25T10:33:13.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.345+0000 7f4d30015640 1 -- 172.21.15.106:0/3872049715 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f4d0c0743b0 msgr2=0x7f4d0c0768c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:13.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.345+0000 7f4d30015640 1 --2- 172.21.15.106:0/3872049715 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f4d0c0743b0 0x7f4d0c0768c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4d28160140 tx=0x7f4d200090e0 comp rx=0 tx=0).stop 2024-04-25T10:33:13.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.345+0000 7f4d30015640 1 -- 172.21.15.106:0/3872049715 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d280c61a0 msgr2=0x7f4d2814ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:13.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.345+0000 7f4d30015640 1 --2- 172.21.15.106:0/3872049715 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d280c61a0 0x7f4d2814ea40 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f4d1800b580 tx=0x7f4d1800ba40 comp rx=0 tx=0).stop 2024-04-25T10:33:13.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.345+0000 7f4d2e5b2640 1 -- 172.21.15.106:0/3872049715 reap_dead start 2024-04-25T10:33:13.361 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.345+0000 7f4d2e5b2640 1 -- 172.21.15.106:0/3872049715 reap_dead start 2024-04-25T10:33:13.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.345+0000 7f4d30015640 1 -- 172.21.15.106:0/3872049715 shutdown_connections 2024-04-25T10:33:13.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.345+0000 7f4d30015640 1 -- 172.21.15.106:0/3872049715 >> 172.21.15.106:0/3872049715 conn(0x7f4d280727d0 msgr2=0x7f4d2806e010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:13.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.346+0000 7f4d30015640 1 -- 172.21.15.106:0/3872049715 shutdown_connections 2024-04-25T10:33:13.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:13.346+0000 7f4d30015640 1 -- 172.21.15.106:0/3872049715 wait complete. 2024-04-25T10:33:13.362 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-04-25T10:33:13.362 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-04-25T10:33:13.653 INFO:tasks.workunit.client.0.smithi106.stdout: 2024-04-25T10:33:13.653 INFO:tasks.workunit.client.0.smithi106.stdout: Children see throughput for 1 re-readers = 7898041.50 kB/sec 2024-04-25T10:33:13.653 INFO:tasks.workunit.client.0.smithi106.stdout: Parent sees throughput for 1 re-readers = 7886470.57 kB/sec 2024-04-25T10:33:13.653 INFO:tasks.workunit.client.0.smithi106.stdout: Min throughput per process = 7898041.50 kB/sec 2024-04-25T10:33:13.653 INFO:tasks.workunit.client.0.smithi106.stdout: Max throughput per process = 7898041.50 kB/sec 2024-04-25T10:33:13.653 INFO:tasks.workunit.client.0.smithi106.stdout: Avg throughput per process = 7898041.50 kB/sec 2024-04-25T10:33:13.653 INFO:tasks.workunit.client.0.smithi106.stdout: Min xfer = 1048576.00 kB 2024-04-25T10:33:20.035 INFO:tasks.workunit.client.0.smithi106.stdout: 2024-04-25T10:33:20.035 INFO:tasks.workunit.client.0.smithi106.stdout: 2024-04-25T10:33:20.035 INFO:tasks.workunit.client.0.smithi106.stdout: 2024-04-25T10:33:20.035 INFO:tasks.workunit.client.0.smithi106.stdout:iozone test complete. 2024-04-25T10:33:20.035 INFO:tasks.workunit.client.0.smithi106.stderr:+ iozone -c -e -s 1024M -r 1M -t 1 -F f2 -i 0 -i 1 2024-04-25T10:33:20.101 INFO:tasks.workunit.client.0.smithi106.stdout: Iozone: Performance Test of File I/O 2024-04-25T10:33:20.101 INFO:tasks.workunit.client.0.smithi106.stdout: Version $Revision: 3.494 $ 2024-04-25T10:33:20.101 INFO:tasks.workunit.client.0.smithi106.stdout: Compiled for 64 bit mode. 2024-04-25T10:33:20.101 INFO:tasks.workunit.client.0.smithi106.stdout: Build: linux-AMD64 2024-04-25T10:33:20.101 INFO:tasks.workunit.client.0.smithi106.stdout: 2024-04-25T10:33:20.101 INFO:tasks.workunit.client.0.smithi106.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-04-25T10:33:20.101 INFO:tasks.workunit.client.0.smithi106.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-04-25T10:33:20.101 INFO:tasks.workunit.client.0.smithi106.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-04-25T10:33:20.101 INFO:tasks.workunit.client.0.smithi106.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-04-25T10:33:20.101 INFO:tasks.workunit.client.0.smithi106.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-04-25T10:33:20.101 INFO:tasks.workunit.client.0.smithi106.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-04-25T10:33:20.101 INFO:tasks.workunit.client.0.smithi106.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-04-25T10:33:20.102 INFO:tasks.workunit.client.0.smithi106.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-04-25T10:33:20.102 INFO:tasks.workunit.client.0.smithi106.stdout: Alexey Skidanov, Sudhir Kumar. 2024-04-25T10:33:20.102 INFO:tasks.workunit.client.0.smithi106.stdout: 2024-04-25T10:33:20.102 INFO:tasks.workunit.client.0.smithi106.stdout: Run began: Thu Apr 25 10:33:20 2024 2024-04-25T10:33:20.102 INFO:tasks.workunit.client.0.smithi106.stdout: 2024-04-25T10:33:20.102 INFO:tasks.workunit.client.0.smithi106.stdout: Include close in write timing 2024-04-25T10:33:20.102 INFO:tasks.workunit.client.0.smithi106.stdout: Include fsync in write timing 2024-04-25T10:33:20.102 INFO:tasks.workunit.client.0.smithi106.stdout: File size set to 1048576 kB 2024-04-25T10:33:20.102 INFO:tasks.workunit.client.0.smithi106.stdout: Record Size 1024 kB 2024-04-25T10:33:20.102 INFO:tasks.workunit.client.0.smithi106.stdout: Command line used: iozone -c -e -s 1024M -r 1M -t 1 -F f2 -i 0 -i 1 2024-04-25T10:33:20.102 INFO:tasks.workunit.client.0.smithi106.stdout: Output is in kBytes/sec 2024-04-25T10:33:20.102 INFO:tasks.workunit.client.0.smithi106.stdout: Time Resolution = 0.000001 seconds. 2024-04-25T10:33:20.102 INFO:tasks.workunit.client.0.smithi106.stdout: Processor cache size set to 1024 kBytes. 2024-04-25T10:33:20.103 INFO:tasks.workunit.client.0.smithi106.stdout: Processor cache line size set to 32 bytes. 2024-04-25T10:33:20.103 INFO:tasks.workunit.client.0.smithi106.stdout: File stride size set to 17 * record size. 2024-04-25T10:33:20.103 INFO:tasks.workunit.client.0.smithi106.stdout: Throughput test with 1 process 2024-04-25T10:33:20.103 INFO:tasks.workunit.client.0.smithi106.stdout: Each process writes a 1048576 kByte file in 1024 kByte records 2024-04-25T10:33:21.696 INFO:tasks.workunit.client.0.smithi106.stdout: 2024-04-25T10:33:21.696 INFO:tasks.workunit.client.0.smithi106.stdout: Children see throughput for 1 initial writers = 670752.12 kB/sec 2024-04-25T10:33:21.696 INFO:tasks.workunit.client.0.smithi106.stdout: Parent sees throughput for 1 initial writers = 670673.99 kB/sec 2024-04-25T10:33:21.696 INFO:tasks.workunit.client.0.smithi106.stdout: Min throughput per process = 670752.12 kB/sec 2024-04-25T10:33:21.696 INFO:tasks.workunit.client.0.smithi106.stdout: Max throughput per process = 670752.12 kB/sec 2024-04-25T10:33:21.696 INFO:tasks.workunit.client.0.smithi106.stdout: Avg throughput per process = 670752.12 kB/sec 2024-04-25T10:33:21.696 INFO:tasks.workunit.client.0.smithi106.stdout: Min xfer = 1048576.00 kB 2024-04-25T10:33:25.445 INFO:tasks.workunit.client.0.smithi106.stdout: 2024-04-25T10:33:25.445 INFO:tasks.workunit.client.0.smithi106.stdout: Children see throughput for 1 rewriters = 795745.38 kB/sec 2024-04-25T10:33:25.445 INFO:tasks.workunit.client.0.smithi106.stdout: Parent sees throughput for 1 rewriters = 795618.58 kB/sec 2024-04-25T10:33:25.445 INFO:tasks.workunit.client.0.smithi106.stdout: Min throughput per process = 795745.38 kB/sec 2024-04-25T10:33:25.445 INFO:tasks.workunit.client.0.smithi106.stdout: Max throughput per process = 795745.38 kB/sec 2024-04-25T10:33:25.445 INFO:tasks.workunit.client.0.smithi106.stdout: Avg throughput per process = 795745.38 kB/sec 2024-04-25T10:33:25.445 INFO:tasks.workunit.client.0.smithi106.stdout: Min xfer = 1048576.00 kB 2024-04-25T10:33:27.884 INFO:tasks.workunit.client.0.smithi106.stdout: 2024-04-25T10:33:27.884 INFO:tasks.workunit.client.0.smithi106.stdout: Children see throughput for 1 readers = 7084826.50 kB/sec 2024-04-25T10:33:27.884 INFO:tasks.workunit.client.0.smithi106.stdout: Parent sees throughput for 1 readers = 7072306.04 kB/sec 2024-04-25T10:33:27.884 INFO:tasks.workunit.client.0.smithi106.stdout: Min throughput per process = 7084826.50 kB/sec 2024-04-25T10:33:27.884 INFO:tasks.workunit.client.0.smithi106.stdout: Max throughput per process = 7084826.50 kB/sec 2024-04-25T10:33:27.884 INFO:tasks.workunit.client.0.smithi106.stdout: Avg throughput per process = 7084826.50 kB/sec 2024-04-25T10:33:27.884 INFO:tasks.workunit.client.0.smithi106.stdout: Min xfer = 1048576.00 kB 2024-04-25T10:33:30.168 INFO:tasks.workunit.client.0.smithi106.stdout: 2024-04-25T10:33:30.168 INFO:tasks.workunit.client.0.smithi106.stdout: Children see throughput for 1 re-readers = 7054326.00 kB/sec 2024-04-25T10:33:30.169 INFO:tasks.workunit.client.0.smithi106.stdout: Parent sees throughput for 1 re-readers = 7043085.79 kB/sec 2024-04-25T10:33:30.169 INFO:tasks.workunit.client.0.smithi106.stdout: Min throughput per process = 7054326.00 kB/sec 2024-04-25T10:33:30.169 INFO:tasks.workunit.client.0.smithi106.stdout: Max throughput per process = 7054326.00 kB/sec 2024-04-25T10:33:30.169 INFO:tasks.workunit.client.0.smithi106.stdout: Avg throughput per process = 7054326.00 kB/sec 2024-04-25T10:33:30.169 INFO:tasks.workunit.client.0.smithi106.stdout: Min xfer = 1048576.00 kB 2024-04-25T10:33:32.641 DEBUG:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:33:32.645 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:33:33.363 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-04-25T10:33:33.364 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2024-04-25T10:33:33.364 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-25T10:33:33.364 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-25T10:33:33.366 INFO:tasks.ceph.mon.b:Started 2024-04-25T10:33:33.367 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-04-25T10:33:33.367 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T10:33:33.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.482+0000 7f5d5be4a640 1 Processor -- start 2024-04-25T10:33:33.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.483+0000 7f5d5be4a640 1 -- start start 2024-04-25T10:33:33.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.484+0000 7f5d5be4a640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f5d540c7e30 0x7f5d540c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:33.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.484+0000 7f5d5be4a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d540b4ce0 0x7f5d5415ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:33.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.484+0000 7f5d5be4a640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d540ce630 con 0x7f5d54156ef0 2024-04-25T10:33:33.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.484+0000 7f5d5be4a640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f5d540b50f0 con 0x7f5d540c7e30 2024-04-25T10:33:33.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.484+0000 7f5d593e5640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f5d54156ef0 0x7f5d5406d000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.106:43910/0 (socket says 172.21.15.106:43910) 2024-04-25T10:33:33.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.484+0000 7f5d5be4a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f5d540c7c60 con 0x7f5d540b4ce0 2024-04-25T10:33:33.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.484+0000 7f5d59be6640 1 --2- 172.21.15.106:0/813037836 >> v2:172.21.15.134:3300/0 conn(0x7f5d540c7e30 0x7f5d540c8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:33.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.484+0000 7f5d593e5640 1 -- 172.21.15.106:0/813037836 learned_addr learned my addr 172.21.15.106:0/813037836 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:33:33.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.484+0000 7f5d5a3e7640 1 --2- 172.21.15.106:0/813037836 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d540b4ce0 0x7f5d5415ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:33.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.484+0000 7f5d59be6640 1 -- 172.21.15.106:0/813037836 >> v2:172.21.15.134:3300/0 conn(0x7f5d540c7e30 msgr2=0x7f5d540c8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T10:33:33.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.484+0000 7f5d59be6640 1 -- 172.21.15.106:0/813037836 >> v2:172.21.15.134:3300/0 conn(0x7f5d540c7e30 msgr2=0x7f5d540c8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:33:33.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.484+0000 7f5d59be6640 1 --2- 172.21.15.106:0/813037836 >> v2:172.21.15.134:3300/0 conn(0x7f5d540c7e30 0x7f5d540c8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:33:33.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.484+0000 7f5d59be6640 1 --2- 172.21.15.106:0/813037836 >> v2:172.21.15.134:3300/0 conn(0x7f5d540c7e30 0x7f5d540c8d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:33:33.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.485+0000 7f5d5a3e7640 1 -- 172.21.15.106:0/813037836 >> v2:172.21.15.134:3300/0 conn(0x7f5d540c7e30 msgr2=0x7f5d540c8d90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:33:33.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.485+0000 7f5d5a3e7640 1 --2- 172.21.15.106:0/813037836 >> v2:172.21.15.134:3300/0 conn(0x7f5d540c7e30 0x7f5d540c8d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:33.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.485+0000 7f5d5a3e7640 1 -- 172.21.15.106:0/813037836 >> v1:172.21.15.134:6789/0 conn(0x7f5d54156ef0 legacy=0x7f5d5406d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:33.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.485+0000 7f5d5a3e7640 1 -- 172.21.15.106:0/813037836 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d540c7980 con 0x7f5d540b4ce0 2024-04-25T10:33:33.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.485+0000 7f5d5a3e7640 1 --2- 172.21.15.106:0/813037836 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d540b4ce0 0x7f5d5415ce30 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5d540c7de0 tx=0x7f5d4c02a5b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f120ec438c94103 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:33.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.486+0000 7f5d58be4640 1 -- 172.21.15.106:0/813037836 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c007740 con 0x7f5d540b4ce0 2024-04-25T10:33:33.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.486+0000 7f5d58be4640 1 -- 172.21.15.106:0/813037836 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d4c0078e0 con 0x7f5d540b4ce0 2024-04-25T10:33:33.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.486+0000 7f5d58be4640 1 -- 172.21.15.106:0/813037836 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c007cb0 con 0x7f5d540b4ce0 2024-04-25T10:33:33.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.486+0000 7f5d5be4a640 1 -- 172.21.15.106:0/813037836 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d540b4ce0 msgr2=0x7f5d5415ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:33.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.486+0000 7f5d5be4a640 1 --2- 172.21.15.106:0/813037836 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d540b4ce0 0x7f5d5415ce30 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5d540c7de0 tx=0x7f5d4c02a5b0 comp rx=0 tx=0).stop 2024-04-25T10:33:33.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.486+0000 7f5d5be4a640 1 -- 172.21.15.106:0/813037836 shutdown_connections 2024-04-25T10:33:33.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.486+0000 7f5d5be4a640 1 --2- 172.21.15.106:0/813037836 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d540b4ce0 0x7f5d5415ce30 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:33.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.486+0000 7f5d5be4a640 1 --2- 172.21.15.106:0/813037836 >> v2:172.21.15.134:3300/0 conn(0x7f5d540c7e30 0x7f5d540c8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:33.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.486+0000 7f5d5be4a640 1 -- 172.21.15.106:0/813037836 >> 172.21.15.106:0/813037836 conn(0x7f5d54068b10 msgr2=0x7f5d540d1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:33.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.486+0000 7f5d5be4a640 1 -- 172.21.15.106:0/813037836 shutdown_connections 2024-04-25T10:33:33.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.486+0000 7f5d5be4a640 1 -- 172.21.15.106:0/813037836 wait complete. 2024-04-25T10:33:33.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.488+0000 7f5d5be4a640 1 Processor -- start 2024-04-25T10:33:33.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.488+0000 7f5d5be4a640 1 -- start start 2024-04-25T10:33:33.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.489+0000 7f5d5be4a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d540b4ce0 0x7f5d541e6f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:33.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.489+0000 7f5d5be4a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d540c7e30 0x7f5d5414fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:33.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.489+0000 7f5d5be4a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d54156ef0 0x7f5d5414ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:33.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.489+0000 7f5d5be4a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5d5406d710 con 0x7f5d540b4ce0 2024-04-25T10:33:33.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.489+0000 7f5d5be4a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5d540c72f0 con 0x7f5d540c7e30 2024-04-25T10:33:33.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.489+0000 7f5d5be4a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f5d5406d890 con 0x7f5d54156ef0 2024-04-25T10:33:33.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.490+0000 7f5d59be6640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d540b4ce0 0x7f5d541e6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:33.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.490+0000 7f5d59be6640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d540b4ce0 0x7f5d541e6f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51594/0 (socket says 172.21.15.106:51594) 2024-04-25T10:33:33.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.490+0000 7f5d59be6640 1 -- 172.21.15.106:0/427320460 learned_addr learned my addr 172.21.15.106:0/427320460 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:33:33.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.490+0000 7f5d593e5640 1 --2- 172.21.15.106:0/427320460 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d540c7e30 0x7f5d5414fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:33.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.490+0000 7f5d5a3e7640 1 --2- 172.21.15.106:0/427320460 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d54156ef0 0x7f5d5414ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:33.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.490+0000 7f5d593e5640 1 -- 172.21.15.106:0/427320460 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d540c7e30 msgr2=0x7f5d5414fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-04-25T10:33:33.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.490+0000 7f5d593e5640 1 -- 172.21.15.106:0/427320460 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d540c7e30 msgr2=0x7f5d5414fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T10:33:33.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.490+0000 7f5d593e5640 1 --2- 172.21.15.106:0/427320460 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d540c7e30 0x7f5d5414fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:33:33.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.490+0000 7f5d593e5640 1 --2- 172.21.15.106:0/427320460 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d540c7e30 0x7f5d5414fa80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:33:33.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.490+0000 7f5d59be6640 1 -- 172.21.15.106:0/427320460 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d54156ef0 msgr2=0x7f5d5414ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:33.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.490+0000 7f5d59be6640 1 --2- 172.21.15.106:0/427320460 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d54156ef0 0x7f5d5414ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:33.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.490+0000 7f5d59be6640 1 -- 172.21.15.106:0/427320460 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d540c7e30 msgr2=0x7f5d5414fa80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:33:33.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.490+0000 7f5d59be6640 1 --2- 172.21.15.106:0/427320460 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d540c7e30 0x7f5d5414fa80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:33.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.490+0000 7f5d59be6640 1 -- 172.21.15.106:0/427320460 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d541e74c0 con 0x7f5d540b4ce0 2024-04-25T10:33:33.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.490+0000 7f5d59be6640 1 --2- 172.21.15.106:0/427320460 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d540b4ce0 0x7f5d541e6f80 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5d5000d800 tx=0x7f5d5000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:33.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.490+0000 7f5d4a7fc640 1 -- 172.21.15.106:0/427320460 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d5000b6c0 con 0x7f5d540b4ce0 2024-04-25T10:33:33.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.490+0000 7f5d4a7fc640 1 -- 172.21.15.106:0/427320460 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d50019070 con 0x7f5d540b4ce0 2024-04-25T10:33:33.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.490+0000 7f5d5be4a640 1 -- 172.21.15.106:0/427320460 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d5415bcc0 con 0x7f5d540b4ce0 2024-04-25T10:33:33.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.490+0000 7f5d4a7fc640 1 -- 172.21.15.106:0/427320460 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d5000bac0 con 0x7f5d540b4ce0 2024-04-25T10:33:33.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.490+0000 7f5d5be4a640 1 -- 172.21.15.106:0/427320460 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d541e64a0 con 0x7f5d540b4ce0 2024-04-25T10:33:33.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.491+0000 7f5d5be4a640 1 -- 172.21.15.106:0/427320460 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d541e7690 con 0x7f5d540b4ce0 2024-04-25T10:33:33.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.492+0000 7f5d4a7fc640 1 -- 172.21.15.106:0/427320460 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5d5000bc60 con 0x7f5d540b4ce0 2024-04-25T10:33:33.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.492+0000 7f5d4a7fc640 1 --2- 172.21.15.106:0/427320460 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f5d380744b0 0x7f5d380769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:33.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.493+0000 7f5d4a7fc640 1 -- 172.21.15.106:0/427320460 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f5d50095770 con 0x7f5d540b4ce0 2024-04-25T10:33:33.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.493+0000 7f5d593e5640 1 --2- 172.21.15.106:0/427320460 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f5d380744b0 0x7f5d380769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:33.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.493+0000 7f5d593e5640 1 --2- 172.21.15.106:0/427320460 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f5d380744b0 0x7f5d380769c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5d440058d0 tx=0x7f5d44009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:33.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.497+0000 7f5d4a7fc640 1 -- 172.21.15.106:0/427320460 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5d5005fe00 con 0x7f5d540b4ce0 2024-04-25T10:33:33.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.745+0000 7f5d5be4a640 1 -- 172.21.15.106:0/427320460 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f5d540b5280 con 0x7f5d540b4ce0 2024-04-25T10:33:33.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.746+0000 7f5d4a7fc640 1 -- 172.21.15.106:0/427320460 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1568 (secure 0 0 0) 0x7f5d540b5280 con 0x7f5d540b4ce0 2024-04-25T10:33:33.748 INFO:teuthology.orchestra.run.smithi106.stdout:{"election_epoch":22,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"3ee77985-e8e4-41cf-81e9-7c774cf7891f","modified":"2024-04-25T10:29:33.446228Z","created":"2024-04-25T10:29:33.446228Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:3300","nonce":0},{"type":"v1","addr":"172.21.15.106:6789","nonce":0}]},"addr":"172.21.15.106:6789/0","public_addr":"172.21.15.106:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T10:33:33.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.748+0000 7f5d5be4a640 1 -- 172.21.15.106:0/427320460 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f5d380744b0 msgr2=0x7f5d380769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:33.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.748+0000 7f5d5be4a640 1 --2- 172.21.15.106:0/427320460 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f5d380744b0 0x7f5d380769c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5d440058d0 tx=0x7f5d44009040 comp rx=0 tx=0).stop 2024-04-25T10:33:33.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.748+0000 7f5d5be4a640 1 -- 172.21.15.106:0/427320460 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d540b4ce0 msgr2=0x7f5d541e6f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:33.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.748+0000 7f5d5be4a640 1 --2- 172.21.15.106:0/427320460 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d540b4ce0 0x7f5d541e6f80 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5d5000d800 tx=0x7f5d5000dcc0 comp rx=0 tx=0).stop 2024-04-25T10:33:33.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.748+0000 7f5d5be4a640 1 -- 172.21.15.106:0/427320460 shutdown_connections 2024-04-25T10:33:33.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.748+0000 7f5d5be4a640 1 --2- 172.21.15.106:0/427320460 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f5d380744b0 0x7f5d380769c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:33.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.748+0000 7f5d5be4a640 1 --2- 172.21.15.106:0/427320460 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d54156ef0 0x7f5d5414ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:33.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.748+0000 7f5d5be4a640 1 --2- 172.21.15.106:0/427320460 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d540c7e30 0x7f5d5414fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:33.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.748+0000 7f5d5be4a640 1 --2- 172.21.15.106:0/427320460 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d540b4ce0 0x7f5d541e6f80 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:33.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.748+0000 7f5d5be4a640 1 -- 172.21.15.106:0/427320460 >> 172.21.15.106:0/427320460 conn(0x7f5d54068b10 msgr2=0x7f5d541533b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:33.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.748+0000 7f5d5be4a640 1 -- 172.21.15.106:0/427320460 shutdown_connections 2024-04-25T10:33:33.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.748+0000 7f5d5be4a640 1 -- 172.21.15.106:0/427320460 wait complete. 2024-04-25T10:33:33.761 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-04-25T10:33:33.761 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-04-25T10:33:33.762 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-25T10:33:33.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.867+0000 7f702e54d640 1 Processor -- start 2024-04-25T10:33:33.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.867+0000 7f702e54d640 1 -- start start 2024-04-25T10:33:33.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.868+0000 7f702e54d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7028072870 0x7f702815d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:33.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.868+0000 7f702e54d640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f7028156d70 0x7f70280caca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:33.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.868+0000 7f702e54d640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f702815dc60 con 0x7f7028157180 2024-04-25T10:33:33.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.868+0000 7f702e54d640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f70281606f0 con 0x7f7028156d70 2024-04-25T10:33:33.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.868+0000 7f702e54d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f70281609d0 con 0x7f7028072870 2024-04-25T10:33:33.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.868+0000 7f702caea640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f7028157180 0x7f70280c9a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:46336/0 (socket says 172.21.15.106:46336) 2024-04-25T10:33:33.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.868+0000 7f702caea640 1 -- 172.21.15.106:0/3897671177 learned_addr learned my addr 172.21.15.106:0/3897671177 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:33:33.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.869+0000 7f7027fff640 1 --2- 172.21.15.106:0/3897671177 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7028072870 0x7f702815d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:33.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.869+0000 7f70277fe640 1 --2- 172.21.15.106:0/3897671177 >> v2:172.21.15.134:3300/0 conn(0x7f7028156d70 0x7f70280caca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:33.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.869+0000 7f70277fe640 1 -- 172.21.15.106:0/3897671177 >> v2:172.21.15.134:3300/0 conn(0x7f7028156d70 msgr2=0x7f70280caca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T10:33:33.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.869+0000 7f70277fe640 1 -- 172.21.15.106:0/3897671177 >> v2:172.21.15.134:3300/0 conn(0x7f7028156d70 msgr2=0x7f70280caca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:33:33.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.869+0000 7f70277fe640 1 --2- 172.21.15.106:0/3897671177 >> v2:172.21.15.134:3300/0 conn(0x7f7028156d70 0x7f70280caca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:33:33.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.869+0000 7f7026ffd640 1 -- 172.21.15.106:0/3897671177 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3515852210 0 0) 0x7f7020001490 con 0x7f7028157180 2024-04-25T10:33:33.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.869+0000 7f70277fe640 1 --2- 172.21.15.106:0/3897671177 >> v2:172.21.15.134:3300/0 conn(0x7f7028156d70 0x7f70280caca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:33:33.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.869+0000 7f7026ffd640 1 -- 172.21.15.106:0/3897671177 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7010003520 con 0x7f7028157180 2024-04-25T10:33:33.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.869+0000 7f7027fff640 1 -- 172.21.15.106:0/3897671177 >> v2:172.21.15.134:3300/0 conn(0x7f7028156d70 msgr2=0x7f70280caca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:33:33.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.869+0000 7f7027fff640 1 --2- 172.21.15.106:0/3897671177 >> v2:172.21.15.134:3300/0 conn(0x7f7028156d70 0x7f70280caca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:33.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.869+0000 7f7027fff640 1 -- 172.21.15.106:0/3897671177 >> v1:172.21.15.106:6789/0 conn(0x7f7028157180 legacy=0x7f70280c9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:33.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.869+0000 7f7027fff640 1 -- 172.21.15.106:0/3897671177 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7028160cb0 con 0x7f7028072870 2024-04-25T10:33:33.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.870+0000 7f7027fff640 1 --2- 172.21.15.106:0/3897671177 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7028072870 0x7f702815d720 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f70180097a0 tx=0x7f701802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b890bc94ea598c94 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:33.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.870+0000 7f7026ffd640 1 -- 172.21.15.106:0/3897671177 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7018037070 con 0x7f7028072870 2024-04-25T10:33:33.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.870+0000 7f7026ffd640 1 -- 172.21.15.106:0/3897671177 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7018032440 con 0x7f7028072870 2024-04-25T10:33:33.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.870+0000 7f7026ffd640 1 -- 172.21.15.106:0/3897671177 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7018032810 con 0x7f7028072870 2024-04-25T10:33:33.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.870+0000 7f702e54d640 1 -- 172.21.15.106:0/3897671177 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7028072870 msgr2=0x7f702815d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:33.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.870+0000 7f702e54d640 1 --2- 172.21.15.106:0/3897671177 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7028072870 0x7f702815d720 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f70180097a0 tx=0x7f701802a210 comp rx=0 tx=0).stop 2024-04-25T10:33:33.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.871+0000 7f702e54d640 1 -- 172.21.15.106:0/3897671177 shutdown_connections 2024-04-25T10:33:33.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.871+0000 7f702e54d640 1 --2- 172.21.15.106:0/3897671177 >> v2:172.21.15.134:3300/0 conn(0x7f7028156d70 0x7f70280caca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:33.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.871+0000 7f702e54d640 1 --2- 172.21.15.106:0/3897671177 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7028072870 0x7f702815d720 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:33.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.871+0000 7f702e54d640 1 -- 172.21.15.106:0/3897671177 >> 172.21.15.106:0/3897671177 conn(0x7f702806c9b0 msgr2=0x7f702806cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:33.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.871+0000 7f702e54d640 1 -- 172.21.15.106:0/3897671177 shutdown_connections 2024-04-25T10:33:33.875 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.871+0000 7f702e54d640 1 -- 172.21.15.106:0/3897671177 wait complete. 2024-04-25T10:33:33.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.872+0000 7f702e54d640 1 Processor -- start 2024-04-25T10:33:33.876 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.872+0000 7f702e54d640 1 -- start start 2024-04-25T10:33:33.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.873+0000 7f702e54d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7028072870 0x7f7028159ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:33.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.874+0000 7f702e54d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7028156d70 0x7f70281f3140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:33.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.874+0000 7f702e54d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7028157180 0x7f702814ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:33.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.874+0000 7f702e54d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7028160290 con 0x7f7028157180 2024-04-25T10:33:33.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.874+0000 7f702e54d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f70280c94e0 con 0x7f7028072870 2024-04-25T10:33:33.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.874+0000 7f702e54d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f7028160410 con 0x7f7028156d70 2024-04-25T10:33:33.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.874+0000 7f702caea640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7028157180 0x7f702814ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:33.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.874+0000 7f702caea640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7028157180 0x7f702814ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51602/0 (socket says 172.21.15.106:51602) 2024-04-25T10:33:33.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.874+0000 7f702caea640 1 -- 172.21.15.106:0/420307308 learned_addr learned my addr 172.21.15.106:0/420307308 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:33:33.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.874+0000 7f70277fe640 1 --2- 172.21.15.106:0/420307308 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7028156d70 0x7f70281f3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:33.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.874+0000 7f7027fff640 1 --2- 172.21.15.106:0/420307308 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7028072870 0x7f7028159ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:33.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.874+0000 7f702caea640 1 -- 172.21.15.106:0/420307308 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7028156d70 msgr2=0x7f70281f3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:33.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.874+0000 7f702caea640 1 --2- 172.21.15.106:0/420307308 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7028156d70 0x7f70281f3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:33.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.874+0000 7f702caea640 1 -- 172.21.15.106:0/420307308 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7028072870 msgr2=0x7f7028159ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:33.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.874+0000 7f702caea640 1 --2- 172.21.15.106:0/420307308 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7028072870 0x7f7028159ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:33.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.874+0000 7f702caea640 1 -- 172.21.15.106:0/420307308 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70281f3680 con 0x7f7028157180 2024-04-25T10:33:33.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.874+0000 7f702caea640 1 --2- 172.21.15.106:0/420307308 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7028157180 0x7f702814ae20 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f702000cad0 tx=0x7f702000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:33.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.874+0000 7f7024ff9640 1 -- 172.21.15.106:0/420307308 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7020017070 con 0x7f7028157180 2024-04-25T10:33:33.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.874+0000 7f7024ff9640 1 -- 172.21.15.106:0/420307308 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f702000ac30 con 0x7f7028157180 2024-04-25T10:33:33.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.874+0000 7f7024ff9640 1 -- 172.21.15.106:0/420307308 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7020012400 con 0x7f7028157180 2024-04-25T10:33:33.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.874+0000 7f702e54d640 1 -- 172.21.15.106:0/420307308 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7028149f30 con 0x7f7028157180 2024-04-25T10:33:33.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.874+0000 7f702e54d640 1 -- 172.21.15.106:0/420307308 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f702814a4b0 con 0x7f7028157180 2024-04-25T10:33:33.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.875+0000 7f702e54d640 1 --2- 172.21.15.106:0/420307308 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70281f3850 0x7f702814b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:33.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.875+0000 7f702e54d640 1 -- 172.21.15.106:0/420307308 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7028154c40 con 0x7f70281f3850 2024-04-25T10:33:33.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.875+0000 7f7027fff640 1 --2- 172.21.15.106:0/420307308 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70281f3850 0x7f702814b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:33.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.876+0000 7f7027fff640 1 -- 172.21.15.106:0/420307308 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f702806a3d0 con 0x7f70281f3850 2024-04-25T10:33:33.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.876+0000 7f7027fff640 1 --2- 172.21.15.106:0/420307308 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70281f3850 0x7f702814b360 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7028074970 tx=0x7f70180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:33.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.876+0000 7f7024ff9640 1 -- 172.21.15.106:0/420307308 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7018042070 con 0x7f70281f3850 2024-04-25T10:33:33.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.876+0000 7f7024ff9640 1 -- 172.21.15.106:0/420307308 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f702001a070 con 0x7f7028157180 2024-04-25T10:33:33.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.877+0000 7f7024ff9640 1 --2- 172.21.15.106:0/420307308 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f70140743e0 0x7f70140768f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:33.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.877+0000 7f7024ff9640 1 -- 172.21.15.106:0/420307308 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f7020094f60 con 0x7f7028157180 2024-04-25T10:33:33.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.877+0000 7f70277fe640 1 --2- 172.21.15.106:0/420307308 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f70140743e0 0x7f70140768f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:33.882 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.877+0000 7f7024ff9640 1 -- 172.21.15.106:0/420307308 <== mon.0 v2:172.21.15.106:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f701803e660 con 0x7f70281f3850 2024-04-25T10:33:33.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.877+0000 7f7024ff9640 1 -- 172.21.15.106:0/420307308 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70281f3850 msgr2=0x7f702814b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:33.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.877+0000 7f7024ff9640 1 --2- 172.21.15.106:0/420307308 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70281f3850 0x7f702814b360 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7028074970 tx=0x7f70180046c0 comp rx=0 tx=0).stop 2024-04-25T10:33:33.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.877+0000 7f7024ff9640 1 -- 172.21.15.106:0/420307308 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70281f3850 msgr2=0x7f702814b360 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T10:33:33.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.877+0000 7f7024ff9640 1 --2- 172.21.15.106:0/420307308 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70281f3850 0x7f702814b360 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:33.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.877+0000 7f70277fe640 1 --2- 172.21.15.106:0/420307308 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f70140743e0 0x7f70140768f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7028070070 tx=0x7f701c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:33.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.883+0000 7f702e54d640 1 --2- 172.21.15.106:0/420307308 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f702814a680 0x7f70281f6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:33.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.883+0000 7f702e54d640 1 -- 172.21.15.106:0/420307308 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7028070940 con 0x7f702814a680 2024-04-25T10:33:33.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.883+0000 7f7027fff640 1 --2- 172.21.15.106:0/420307308 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f702814a680 0x7f70281f6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:33.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.884+0000 7f7027fff640 1 -- 172.21.15.106:0/420307308 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f7028070660 con 0x7f702814a680 2024-04-25T10:33:33.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.884+0000 7f7027fff640 1 --2- 172.21.15.106:0/420307308 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f702814a680 0x7f70281f6260 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f701802a250 tx=0x7f7018008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:33.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.884+0000 7f7024ff9640 1 -- 172.21.15.106:0/420307308 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7018037040 con 0x7f702814a680 2024-04-25T10:33:33.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.884+0000 7f7024ff9640 1 -- 172.21.15.106:0/420307308 <== mon.0 v2:172.21.15.106:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5071 (secure 0 0 0) 0x7f70180044a0 con 0x7f702814a680 2024-04-25T10:33:33.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.884+0000 7f7024ff9640 1 -- 172.21.15.106:0/420307308 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f702814a680 msgr2=0x7f70281f6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:33.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.884+0000 7f7024ff9640 1 --2- 172.21.15.106:0/420307308 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f702814a680 0x7f70281f6260 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f701802a250 tx=0x7f7018008030 comp rx=0 tx=0).stop 2024-04-25T10:33:33.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.884+0000 7f7024ff9640 1 -- 172.21.15.106:0/420307308 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f702814a680 msgr2=0x7f70281f6260 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T10:33:33.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.884+0000 7f7024ff9640 1 --2- 172.21.15.106:0/420307308 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f702814a680 0x7f70281f6260 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:33.887 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-25T10:33:33.887 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "a", 2024-04-25T10:33:33.887 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 0, 2024-04-25T10:33:33.888 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "leader", 2024-04-25T10:33:33.888 INFO:teuthology.orchestra.run.smithi106.stdout: "election_epoch": 22, 2024-04-25T10:33:33.888 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 206123, 2024-04-25T10:33:33.888 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum": [ 2024-04-25T10:33:33.888 INFO:teuthology.orchestra.run.smithi106.stdout: 0, 2024-04-25T10:33:33.888 INFO:teuthology.orchestra.run.smithi106.stdout: 1, 2024-04-25T10:33:33.888 INFO:teuthology.orchestra.run.smithi106.stdout: 2 2024-04-25T10:33:33.888 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:33:33.888 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_age": 0, 2024-04-25T10:33:33.888 INFO:teuthology.orchestra.run.smithi106.stdout: "features": { 2024-04-25T10:33:33.888 INFO:teuthology.orchestra.run.smithi106.stdout: "required_con": "2450521708007866372", 2024-04-25T10:33:33.888 INFO:teuthology.orchestra.run.smithi106.stdout: "required_mon": [ 2024-04-25T10:33:33.888 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:33:33.888 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:33:33.888 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:33:33.889 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:33:33.889 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:33:33.889 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:33:33.889 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:33:33.889 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:33:33.889 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:33:33.889 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:33:33.889 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:33:33.889 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:33:33.889 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_con": "4540701547738038271", 2024-04-25T10:33:33.889 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_mon": [ 2024-04-25T10:33:33.889 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:33:33.889 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:33:33.890 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:33:33.890 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:33:33.890 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:33:33.890 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:33:33.890 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:33:33.890 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:33:33.890 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:33:33.890 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:33:33.890 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:33:33.890 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:33:33.890 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:33.890 INFO:teuthology.orchestra.run.smithi106.stdout: "outside_quorum": [], 2024-04-25T10:33:33.890 INFO:teuthology.orchestra.run.smithi106.stdout: "extra_probe_peers": [], 2024-04-25T10:33:33.890 INFO:teuthology.orchestra.run.smithi106.stdout: "sync_provider": [], 2024-04-25T10:33:33.891 INFO:teuthology.orchestra.run.smithi106.stdout: "monmap": { 2024-04-25T10:33:33.891 INFO:teuthology.orchestra.run.smithi106.stdout: "epoch": 1, 2024-04-25T10:33:33.891 INFO:teuthology.orchestra.run.smithi106.stdout: "fsid": "3ee77985-e8e4-41cf-81e9-7c774cf7891f", 2024-04-25T10:33:33.891 INFO:teuthology.orchestra.run.smithi106.stdout: "modified": "2024-04-25T10:29:33.446228Z", 2024-04-25T10:33:33.891 INFO:teuthology.orchestra.run.smithi106.stdout: "created": "2024-04-25T10:29:33.446228Z", 2024-04-25T10:33:33.891 INFO:teuthology.orchestra.run.smithi106.stdout: "min_mon_release": 19, 2024-04-25T10:33:33.891 INFO:teuthology.orchestra.run.smithi106.stdout: "min_mon_release_name": "squid", 2024-04-25T10:33:33.891 INFO:teuthology.orchestra.run.smithi106.stdout: "election_strategy": 1, 2024-04-25T10:33:33.891 INFO:teuthology.orchestra.run.smithi106.stdout: "disallowed_leaders: ": "", 2024-04-25T10:33:33.891 INFO:teuthology.orchestra.run.smithi106.stdout: "stretch_mode": false, 2024-04-25T10:33:33.891 INFO:teuthology.orchestra.run.smithi106.stdout: "tiebreaker_mon": "", 2024-04-25T10:33:33.891 INFO:teuthology.orchestra.run.smithi106.stdout: "removed_ranks: ": "", 2024-04-25T10:33:33.891 INFO:teuthology.orchestra.run.smithi106.stdout: "features": { 2024-04-25T10:33:33.892 INFO:teuthology.orchestra.run.smithi106.stdout: "persistent": [ 2024-04-25T10:33:33.892 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:33:33.892 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:33:33.892 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:33:33.892 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:33:33.892 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:33:33.892 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:33:33.892 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:33:33.892 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:33:33.892 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:33:33.892 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:33:33.892 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:33:33.892 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:33:33.892 INFO:teuthology.orchestra.run.smithi106.stdout: "optional": [] 2024-04-25T10:33:33.892 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:33.893 INFO:teuthology.orchestra.run.smithi106.stdout: "mons": [ 2024-04-25T10:33:33.893 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:33.893 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 0, 2024-04-25T10:33:33.893 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "a", 2024-04-25T10:33:33.893 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:33:33.893 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:33:33.893 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:33.893 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:33:33.893 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:3300", 2024-04-25T10:33:33.893 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:33:33.893 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:33.893 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:33.893 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:33:33.893 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:6789", 2024-04-25T10:33:33.893 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:33:33.894 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:33:33.894 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:33:33.894 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:33.894 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:6789/0", 2024-04-25T10:33:33.894 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.106:6789/0", 2024-04-25T10:33:33.894 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:33:33.894 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:33:33.894 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:33:33.894 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:33.894 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:33.894 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 1, 2024-04-25T10:33:33.894 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "b", 2024-04-25T10:33:33.894 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:33:33.895 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:33:33.895 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:33.895 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:33:33.895 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:3300", 2024-04-25T10:33:33.895 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:33:33.895 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:33.895 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:33.895 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:33:33.895 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6789", 2024-04-25T10:33:33.895 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:33:33.895 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:33:33.895 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:33:33.895 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:33.895 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6789/0", 2024-04-25T10:33:33.896 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.134:6789/0", 2024-04-25T10:33:33.896 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:33:33.896 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:33:33.896 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:33:33.896 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:33.896 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:33.896 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 2, 2024-04-25T10:33:33.896 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "c", 2024-04-25T10:33:33.896 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:33:33.896 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:33:33.896 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:33.896 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:33:33.896 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:3301", 2024-04-25T10:33:33.897 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:33:33.897 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:33.897 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:33.897 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:33:33.897 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6790", 2024-04-25T10:33:33.897 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:33:33.897 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:33:33.897 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:33:33.897 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:33.897 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6790/0", 2024-04-25T10:33:33.897 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.134:6790/0", 2024-04-25T10:33:33.897 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:33:33.897 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:33:33.897 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:33:33.898 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:33:33.898 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:33:33.898 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:33.898 INFO:teuthology.orchestra.run.smithi106.stdout: "feature_map": { 2024-04-25T10:33:33.898 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": [ 2024-04-25T10:33:33.898 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:33.898 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:33:33.898 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:33:33.898 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 1 2024-04-25T10:33:33.898 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:33:33.898 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:33:33.898 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": [ 2024-04-25T10:33:33.899 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:33.899 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:33:33.899 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:33:33.899 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 6 2024-04-25T10:33:33.899 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:33:33.899 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:33:33.899 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": [ 2024-04-25T10:33:33.899 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:33.899 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:33:33.899 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:33:33.899 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 6 2024-04-25T10:33:33.899 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:33:33.899 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:33:33.899 INFO:teuthology.orchestra.run.smithi106.stdout: "client": [ 2024-04-25T10:33:33.900 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:33.900 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:33:33.900 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:33:33.900 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 7 2024-04-25T10:33:33.900 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:33:33.900 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:33:33.900 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": [ 2024-04-25T10:33:33.900 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:33.900 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:33:33.900 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:33:33.900 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 2 2024-04-25T10:33:33.900 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:33:33.900 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:33:33.901 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:33.901 INFO:teuthology.orchestra.run.smithi106.stdout: "stretch_mode": false 2024-04-25T10:33:33.901 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-25T10:33:33.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.884+0000 7f702e54d640 1 -- 172.21.15.106:0/420307308 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f70140743e0 msgr2=0x7f70140768f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:33.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.884+0000 7f702e54d640 1 --2- 172.21.15.106:0/420307308 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f70140743e0 0x7f70140768f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7028070070 tx=0x7f701c008040 comp rx=0 tx=0).stop 2024-04-25T10:33:33.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.884+0000 7f702e54d640 1 -- 172.21.15.106:0/420307308 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7028157180 msgr2=0x7f702814ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:33.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.884+0000 7f702caea640 1 -- 172.21.15.106:0/420307308 reap_dead start 2024-04-25T10:33:33.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.884+0000 7f702e54d640 1 --2- 172.21.15.106:0/420307308 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7028157180 0x7f702814ae20 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f702000cad0 tx=0x7f702000cf90 comp rx=0 tx=0).stop 2024-04-25T10:33:33.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.884+0000 7f702e54d640 1 -- 172.21.15.106:0/420307308 shutdown_connections 2024-04-25T10:33:33.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.884+0000 7f702e54d640 1 --2- 172.21.15.106:0/420307308 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7028157180 0x7f702814ae20 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:33.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.884+0000 7f702e54d640 1 -- 172.21.15.106:0/420307308 >> 172.21.15.106:0/420307308 conn(0x7f702806c9b0 msgr2=0x7f70280c68c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:33.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.885+0000 7f702e54d640 1 -- 172.21.15.106:0/420307308 shutdown_connections 2024-04-25T10:33:33.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:33.885+0000 7f702e54d640 1 -- 172.21.15.106:0/420307308 wait complete. 2024-04-25T10:33:33.903 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T10:33:33.903 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-04-25T10:33:33.903 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-25T10:33:34.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.009+0000 7f6a125d6640 1 Processor -- start 2024-04-25T10:33:34.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.009+0000 7f6a125d6640 1 -- start start 2024-04-25T10:33:34.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.010+0000 7f6a125d6640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f6a0c0ca170 0x7f6a0c156f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:34.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.010+0000 7f6a125d6640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a0c073980 con 0x7f6a0c15d510 2024-04-25T10:33:34.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.010+0000 7f6a125d6640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a0c160ea0 con 0x7f6a0c06c5c0 2024-04-25T10:33:34.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.010+0000 7f6a125d6640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f6a0c073f00 con 0x7f6a0c0ca170 2024-04-25T10:33:34.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.010+0000 7f6a10b73640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f6a0c15d510 0x7f6a0c167680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:46350/0 (socket says 172.21.15.106:46350) 2024-04-25T10:33:34.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.010+0000 7f6a10b73640 1 -- 172.21.15.106:0/3666770890 learned_addr learned my addr 172.21.15.106:0/3666770890 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:33:34.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.010+0000 7f6a0bfff640 1 --2- 172.21.15.106:0/3666770890 >> v2:172.21.15.134:3300/0 conn(0x7f6a0c0ca170 0x7f6a0c156f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:34.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.011+0000 7f6a0affd640 1 -- 172.21.15.106:0/3666770890 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 791875075 0 0) 0x7f6a0c073980 con 0x7f6a0c15d510 2024-04-25T10:33:34.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.011+0000 7f6a0affd640 1 -- 172.21.15.106:0/3666770890 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f69f4003520 con 0x7f6a0c15d510 2024-04-25T10:33:34.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.011+0000 7f6a0bfff640 1 -- 172.21.15.106:0/3666770890 >> v2:172.21.15.134:3300/0 conn(0x7f6a0c0ca170 msgr2=0x7f6a0c156f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T10:33:34.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.011+0000 7f6a0bfff640 1 -- 172.21.15.106:0/3666770890 >> v2:172.21.15.134:3300/0 conn(0x7f6a0c0ca170 msgr2=0x7f6a0c156f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:33:34.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.011+0000 7f6a0bfff640 1 --2- 172.21.15.106:0/3666770890 >> v2:172.21.15.134:3300/0 conn(0x7f6a0c0ca170 0x7f6a0c156f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:33:34.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.011+0000 7f6a0bfff640 1 --2- 172.21.15.106:0/3666770890 >> v2:172.21.15.134:3300/0 conn(0x7f6a0c0ca170 0x7f6a0c156f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:33:34.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.011+0000 7f6a0affd640 1 -- 172.21.15.106:0/3666770890 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 461688004 0 0) 0x7f69f4003520 con 0x7f6a0c15d510 2024-04-25T10:33:34.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.011+0000 7f6a0affd640 1 -- 172.21.15.106:0/3666770890 >> v2:172.21.15.134:3300/0 conn(0x7f6a0c0ca170 msgr2=0x7f6a0c156f70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:33:34.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.011+0000 7f6a0affd640 1 --2- 172.21.15.106:0/3666770890 >> v2:172.21.15.134:3300/0 conn(0x7f6a0c0ca170 0x7f6a0c156f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:34.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.011+0000 7f6a0affd640 1 -- 172.21.15.106:0/3666770890 >> v1:172.21.15.134:6789/0 conn(0x7f6a0c06c5c0 legacy=0x7f6a0c15ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:34.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.011+0000 7f6a0affd640 1 -- 172.21.15.106:0/3666770890 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a0c074640 con 0x7f6a0c15d510 2024-04-25T10:33:34.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.011+0000 7f6a0affd640 1 -- 172.21.15.106:0/3666770890 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3776459350 0 0) 0x7f6a040019f0 con 0x7f6a0c15d510 2024-04-25T10:33:34.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.011+0000 7f6a0affd640 1 -- 172.21.15.106:0/3666770890 >> v1:172.21.15.106:6789/0 conn(0x7f6a0c15d510 legacy=0x7f6a0c167680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:34.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.011+0000 7f6a0affd640 1 --2- 172.21.15.106:0/3666770890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f69f40039b0 0x7f69f4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:34.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.011+0000 7f6a0affd640 1 --2- 172.21.15.106:0/3666770890 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f69f4004800 0x7f69f4004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:34.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.011+0000 7f6a0affd640 1 --2- 172.21.15.106:0/3666770890 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f69f4005270 0x7f69f400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:34.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.011+0000 7f6a0affd640 1 -- 172.21.15.106:0/3666770890 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0c073f00 con 0x7f69f4005270 2024-04-25T10:33:34.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.011+0000 7f6a0affd640 1 -- 172.21.15.106:0/3666770890 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f69f400c1f0 con 0x7f69f40039b0 2024-04-25T10:33:34.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.011+0000 7f6a0affd640 1 -- 172.21.15.106:0/3666770890 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f69f400c3f0 con 0x7f69f4004800 2024-04-25T10:33:34.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.011+0000 7f6a10b73640 1 --2- 172.21.15.106:0/3666770890 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f69f4005270 0x7f69f400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:34.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.011+0000 7f6a0bfff640 1 --2- 172.21.15.106:0/3666770890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f69f40039b0 0x7f69f4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:34.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.011+0000 7f6a0b7fe640 1 --2- 172.21.15.106:0/3666770890 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f69f4004800 0x7f69f4004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:34.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.011+0000 7f6a10b73640 1 -- 172.21.15.106:0/3666770890 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f69f4004800 msgr2=0x7f69f4004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:34.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.011+0000 7f6a10b73640 1 --2- 172.21.15.106:0/3666770890 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f69f4004800 0x7f69f4004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:34.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.011+0000 7f6a10b73640 1 -- 172.21.15.106:0/3666770890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f69f40039b0 msgr2=0x7f69f4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:34.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.011+0000 7f6a10b73640 1 --2- 172.21.15.106:0/3666770890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f69f40039b0 0x7f69f4003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:34.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.011+0000 7f6a10b73640 1 -- 172.21.15.106:0/3666770890 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69f4003520 con 0x7f69f4005270 2024-04-25T10:33:34.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.012+0000 7f6a10b73640 1 -- 172.21.15.106:0/3666770890 reap_dead start 2024-04-25T10:33:34.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.012+0000 7f6a10b73640 1 --2- 172.21.15.106:0/3666770890 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f69f4005270 0x7f69f400b9c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f6a0400a810 tx=0x7f69f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=692fefd9ef9b62c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:34.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.012+0000 7f6a0affd640 1 -- 172.21.15.106:0/3666770890 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a0400a460 con 0x7f69f4005270 2024-04-25T10:33:34.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.012+0000 7f6a0affd640 1 -- 172.21.15.106:0/3666770890 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a0402f070 con 0x7f69f4005270 2024-04-25T10:33:34.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.012+0000 7f6a0affd640 1 -- 172.21.15.106:0/3666770890 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a0402bc70 con 0x7f69f4005270 2024-04-25T10:33:34.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.013+0000 7f6a125d6640 1 -- 172.21.15.106:0/3666770890 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f69f4005270 msgr2=0x7f69f400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:34.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.013+0000 7f6a125d6640 1 --2- 172.21.15.106:0/3666770890 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f69f4005270 0x7f69f400b9c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f6a0400a810 tx=0x7f69f4005830 comp rx=0 tx=0).stop 2024-04-25T10:33:34.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.013+0000 7f6a125d6640 1 -- 172.21.15.106:0/3666770890 shutdown_connections 2024-04-25T10:33:34.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.013+0000 7f6a125d6640 1 --2- 172.21.15.106:0/3666770890 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f69f4005270 0x7f69f400b9c0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:34.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.013+0000 7f6a125d6640 1 -- 172.21.15.106:0/3666770890 >> 172.21.15.106:0/3666770890 conn(0x7f6a0c0cef20 msgr2=0x7f6a0c0cf330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:34.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.013+0000 7f6a125d6640 1 -- 172.21.15.106:0/3666770890 shutdown_connections 2024-04-25T10:33:34.019 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.013+0000 7f6a125d6640 1 -- 172.21.15.106:0/3666770890 wait complete. 2024-04-25T10:33:34.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.015+0000 7f6a125d6640 1 Processor -- start 2024-04-25T10:33:34.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.015+0000 7f6a125d6640 1 -- start start 2024-04-25T10:33:34.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.016+0000 7f6a125d6640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f69f4005270 0x7f6a0c155ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:34.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.016+0000 7f6a125d6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a0c15eb40 0x7f6a0c153ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:34.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.016+0000 7f6a125d6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a0c15bd50 0x7f6a0c14c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:34.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.016+0000 7f6a125d6640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0c161140 con 0x7f69f4005270 2024-04-25T10:33:34.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.016+0000 7f6a125d6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0c074990 con 0x7f6a0c15bd50 2024-04-25T10:33:34.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.016+0000 7f6a125d6640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6a0c0c9cd0 con 0x7f6a0c15eb40 2024-04-25T10:33:34.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.016+0000 7f6a0bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f69f4005270 0x7f6a0c155ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:34.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.016+0000 7f6a0bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f69f4005270 0x7f6a0c155ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51656/0 (socket says 172.21.15.106:51656) 2024-04-25T10:33:34.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.016+0000 7f6a0bfff640 1 -- 172.21.15.106:0/520996920 learned_addr learned my addr 172.21.15.106:0/520996920 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:33:34.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.016+0000 7f6a0b7fe640 1 --2- 172.21.15.106:0/520996920 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a0c15eb40 0x7f6a0c153ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:34.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.016+0000 7f6a10b73640 1 --2- 172.21.15.106:0/520996920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a0c15bd50 0x7f6a0c14c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:34.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.017+0000 7f6a0bfff640 1 -- 172.21.15.106:0/520996920 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a0c15eb40 msgr2=0x7f6a0c153ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:34.023 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.017+0000 7f6a0bfff640 1 --2- 172.21.15.106:0/520996920 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a0c15eb40 0x7f6a0c153ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:34.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.017+0000 7f6a0bfff640 1 -- 172.21.15.106:0/520996920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a0c15bd50 msgr2=0x7f6a0c14c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:34.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.017+0000 7f6a0bfff640 1 --2- 172.21.15.106:0/520996920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a0c15bd50 0x7f6a0c14c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:34.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.017+0000 7f6a0bfff640 1 -- 172.21.15.106:0/520996920 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a0c15a770 con 0x7f69f4005270 2024-04-25T10:33:34.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.017+0000 7f6a0bfff640 1 --2- 172.21.15.106:0/520996920 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f69f4005270 0x7f6a0c155ae0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f69fc0108c0 tx=0x7f69fc010d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:34.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.017+0000 7f6a08ff9640 1 -- 172.21.15.106:0/520996920 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69fc01a020 con 0x7f69f4005270 2024-04-25T10:33:34.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.017+0000 7f6a08ff9640 1 -- 172.21.15.106:0/520996920 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69fc00e9c0 con 0x7f69f4005270 2024-04-25T10:33:34.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.017+0000 7f6a08ff9640 1 -- 172.21.15.106:0/520996920 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69fc00edb0 con 0x7f69f4005270 2024-04-25T10:33:34.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.017+0000 7f6a125d6640 1 -- 172.21.15.106:0/520996920 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a0c1f3ed0 con 0x7f69f4005270 2024-04-25T10:33:34.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.017+0000 7f6a125d6640 1 -- 172.21.15.106:0/520996920 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a0c15e060 con 0x7f69f4005270 2024-04-25T10:33:34.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.018+0000 7f6a125d6640 1 --2- 172.21.15.106:0/520996920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a0c15c160 0x7f6a0c15e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:34.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.018+0000 7f6a125d6640 1 -- 172.21.15.106:0/520996920 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0c1562d0 con 0x7f6a0c15c160 2024-04-25T10:33:34.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.018+0000 7f6a0b7fe640 1 --2- 172.21.15.106:0/520996920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a0c15c160 0x7f6a0c15e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:34.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.018+0000 7f6a0b7fe640 1 -- 172.21.15.106:0/520996920 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6a0c1f40a0 con 0x7f6a0c15c160 2024-04-25T10:33:34.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.019+0000 7f6a0b7fe640 1 --2- 172.21.15.106:0/520996920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a0c15c160 0x7f6a0c15e2a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6a00000c00 tx=0x7f6a00012ac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:34.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.019+0000 7f6a08ff9640 1 -- 172.21.15.106:0/520996920 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a00019020 con 0x7f6a0c15c160 2024-04-25T10:33:34.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.019+0000 7f6a08ff9640 1 -- 172.21.15.106:0/520996920 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f69fc01d020 con 0x7f69f4005270 2024-04-25T10:33:34.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.019+0000 7f6a08ff9640 1 --2- 172.21.15.106:0/520996920 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f69f80743b0 0x7f69f80768c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:34.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.019+0000 7f6a10b73640 1 --2- 172.21.15.106:0/520996920 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f69f80743b0 0x7f69f80768c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:34.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.019+0000 7f6a08ff9640 1 -- 172.21.15.106:0/520996920 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f69fc095ae0 con 0x7f69f4005270 2024-04-25T10:33:34.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.020+0000 7f6a08ff9640 1 -- 172.21.15.106:0/520996920 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f6a000053a0 con 0x7f6a0c15c160 2024-04-25T10:33:34.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.020+0000 7f6a08ff9640 1 -- 172.21.15.106:0/520996920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a0c15c160 msgr2=0x7f6a0c15e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:34.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.020+0000 7f6a08ff9640 1 --2- 172.21.15.106:0/520996920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a0c15c160 0x7f6a0c15e2a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6a00000c00 tx=0x7f6a00012ac0 comp rx=0 tx=0).stop 2024-04-25T10:33:34.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.020+0000 7f6a08ff9640 1 -- 172.21.15.106:0/520996920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a0c15c160 msgr2=0x7f6a0c15e2a0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T10:33:34.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.020+0000 7f6a08ff9640 1 --2- 172.21.15.106:0/520996920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a0c15c160 0x7f6a0c15e2a0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:34.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.020+0000 7f6a10b73640 1 --2- 172.21.15.106:0/520996920 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f69f80743b0 0x7f69f80768c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f69f400c1c0 tx=0x7f6a04007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:34.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.026+0000 7f6a125d6640 1 --2- 172.21.15.106:0/520996920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a0c14cc70 0x7f6a0c1f5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:34.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.026+0000 7f6a125d6640 1 -- 172.21.15.106:0/520996920 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0c15e7e0 con 0x7f6a0c14cc70 2024-04-25T10:33:34.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.026+0000 7f6a0b7fe640 1 --2- 172.21.15.106:0/520996920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a0c14cc70 0x7f6a0c1f5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:34.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.027+0000 7f6a0b7fe640 1 -- 172.21.15.106:0/520996920 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f6a0c06a3d0 con 0x7f6a0c14cc70 2024-04-25T10:33:34.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.027+0000 7f6a0b7fe640 1 --2- 172.21.15.106:0/520996920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a0c14cc70 0x7f6a0c1f5fd0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6a0001a000 tx=0x7f6a00001120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:34.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.027+0000 7f6a08ff9640 1 -- 172.21.15.106:0/520996920 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a00003bc0 con 0x7f6a0c14cc70 2024-04-25T10:33:34.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.027+0000 7f6a08ff9640 1 -- 172.21.15.106:0/520996920 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4556 (secure 0 0 0) 0x7f6a00003bc0 con 0x7f6a0c14cc70 2024-04-25T10:33:34.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.027+0000 7f6a08ff9640 1 -- 172.21.15.106:0/520996920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a0c14cc70 msgr2=0x7f6a0c1f5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:34.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.027+0000 7f6a08ff9640 1 --2- 172.21.15.106:0/520996920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a0c14cc70 0x7f6a0c1f5fd0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6a0001a000 tx=0x7f6a00001120 comp rx=0 tx=0).stop 2024-04-25T10:33:34.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.027+0000 7f6a08ff9640 1 -- 172.21.15.106:0/520996920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a0c14cc70 msgr2=0x7f6a0c1f5fd0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T10:33:34.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.027+0000 7f6a08ff9640 1 --2- 172.21.15.106:0/520996920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a0c14cc70 0x7f6a0c1f5fd0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:34.030 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-25T10:33:34.030 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "b", 2024-04-25T10:33:34.030 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 1, 2024-04-25T10:33:34.031 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "peon", 2024-04-25T10:33:34.031 INFO:teuthology.orchestra.run.smithi106.stdout: "election_epoch": 22, 2024-04-25T10:33:34.031 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 552, 2024-04-25T10:33:34.031 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum": [ 2024-04-25T10:33:34.031 INFO:teuthology.orchestra.run.smithi106.stdout: 0, 2024-04-25T10:33:34.031 INFO:teuthology.orchestra.run.smithi106.stdout: 1, 2024-04-25T10:33:34.031 INFO:teuthology.orchestra.run.smithi106.stdout: 2 2024-04-25T10:33:34.031 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:33:34.031 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_age": 0, 2024-04-25T10:33:34.031 INFO:teuthology.orchestra.run.smithi106.stdout: "features": { 2024-04-25T10:33:34.031 INFO:teuthology.orchestra.run.smithi106.stdout: "required_con": "2450521708007866372", 2024-04-25T10:33:34.031 INFO:teuthology.orchestra.run.smithi106.stdout: "required_mon": [ 2024-04-25T10:33:34.031 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:33:34.031 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:33:34.031 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:33:34.032 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:33:34.032 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:33:34.032 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:33:34.032 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:33:34.032 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:33:34.032 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:33:34.032 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:33:34.032 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:33:34.032 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:33:34.032 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_con": "4540701547738038271", 2024-04-25T10:33:34.032 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_mon": [ 2024-04-25T10:33:34.032 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:33:34.032 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:33:34.032 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:33:34.033 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:33:34.033 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:33:34.033 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:33:34.033 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:33:34.033 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:33:34.033 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:33:34.033 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:33:34.033 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:33:34.033 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:33:34.033 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:34.033 INFO:teuthology.orchestra.run.smithi106.stdout: "outside_quorum": [], 2024-04-25T10:33:34.033 INFO:teuthology.orchestra.run.smithi106.stdout: "extra_probe_peers": [], 2024-04-25T10:33:34.033 INFO:teuthology.orchestra.run.smithi106.stdout: "sync_provider": [], 2024-04-25T10:33:34.033 INFO:teuthology.orchestra.run.smithi106.stdout: "monmap": { 2024-04-25T10:33:34.034 INFO:teuthology.orchestra.run.smithi106.stdout: "epoch": 1, 2024-04-25T10:33:34.034 INFO:teuthology.orchestra.run.smithi106.stdout: "fsid": "3ee77985-e8e4-41cf-81e9-7c774cf7891f", 2024-04-25T10:33:34.034 INFO:teuthology.orchestra.run.smithi106.stdout: "modified": "2024-04-25T10:29:33.446228Z", 2024-04-25T10:33:34.034 INFO:teuthology.orchestra.run.smithi106.stdout: "created": "2024-04-25T10:29:33.446228Z", 2024-04-25T10:33:34.034 INFO:teuthology.orchestra.run.smithi106.stdout: "min_mon_release": 19, 2024-04-25T10:33:34.034 INFO:teuthology.orchestra.run.smithi106.stdout: "min_mon_release_name": "squid", 2024-04-25T10:33:34.034 INFO:teuthology.orchestra.run.smithi106.stdout: "election_strategy": 1, 2024-04-25T10:33:34.034 INFO:teuthology.orchestra.run.smithi106.stdout: "disallowed_leaders: ": "", 2024-04-25T10:33:34.034 INFO:teuthology.orchestra.run.smithi106.stdout: "stretch_mode": false, 2024-04-25T10:33:34.034 INFO:teuthology.orchestra.run.smithi106.stdout: "tiebreaker_mon": "", 2024-04-25T10:33:34.034 INFO:teuthology.orchestra.run.smithi106.stdout: "removed_ranks: ": "", 2024-04-25T10:33:34.034 INFO:teuthology.orchestra.run.smithi106.stdout: "features": { 2024-04-25T10:33:34.034 INFO:teuthology.orchestra.run.smithi106.stdout: "persistent": [ 2024-04-25T10:33:34.034 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:33:34.035 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:33:34.035 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:33:34.035 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:33:34.035 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:33:34.035 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:33:34.035 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:33:34.035 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:33:34.035 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:33:34.035 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:33:34.035 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:33:34.035 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:33:34.035 INFO:teuthology.orchestra.run.smithi106.stdout: "optional": [] 2024-04-25T10:33:34.035 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:34.035 INFO:teuthology.orchestra.run.smithi106.stdout: "mons": [ 2024-04-25T10:33:34.036 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:34.036 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 0, 2024-04-25T10:33:34.036 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "a", 2024-04-25T10:33:34.036 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:33:34.036 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:33:34.036 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:34.036 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:33:34.036 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:3300", 2024-04-25T10:33:34.036 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:33:34.036 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:34.036 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:34.036 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:33:34.036 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:6789", 2024-04-25T10:33:34.036 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:33:34.036 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:33:34.036 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:33:34.037 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:34.037 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:6789/0", 2024-04-25T10:33:34.037 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.106:6789/0", 2024-04-25T10:33:34.037 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:33:34.037 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:33:34.037 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:33:34.037 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:34.037 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:34.037 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 1, 2024-04-25T10:33:34.037 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "b", 2024-04-25T10:33:34.037 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:33:34.037 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:33:34.037 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:34.037 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:33:34.037 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:3300", 2024-04-25T10:33:34.037 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:33:34.038 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:34.038 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:34.038 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:33:34.038 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6789", 2024-04-25T10:33:34.038 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:33:34.038 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:33:34.038 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:33:34.038 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:34.038 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6789/0", 2024-04-25T10:33:34.038 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.134:6789/0", 2024-04-25T10:33:34.038 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:33:34.038 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:33:34.038 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:33:34.038 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:34.039 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:34.039 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 2, 2024-04-25T10:33:34.039 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "c", 2024-04-25T10:33:34.039 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:33:34.039 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:33:34.039 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:34.039 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:33:34.039 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:3301", 2024-04-25T10:33:34.039 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:33:34.039 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:34.039 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:34.039 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:33:34.039 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6790", 2024-04-25T10:33:34.039 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:33:34.039 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:33:34.040 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:33:34.040 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:34.040 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6790/0", 2024-04-25T10:33:34.040 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.134:6790/0", 2024-04-25T10:33:34.040 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:33:34.040 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:33:34.040 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:33:34.040 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:33:34.040 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:33:34.040 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:34.040 INFO:teuthology.orchestra.run.smithi106.stdout: "feature_map": { 2024-04-25T10:33:34.040 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": [ 2024-04-25T10:33:34.040 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:34.041 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:33:34.041 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:33:34.041 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 1 2024-04-25T10:33:34.041 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:33:34.041 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:33:34.041 INFO:teuthology.orchestra.run.smithi106.stdout: "client": [ 2024-04-25T10:33:34.041 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:34.041 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:33:34.041 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:33:34.041 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 1 2024-04-25T10:33:34.041 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:33:34.041 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:33:34.041 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:34.041 INFO:teuthology.orchestra.run.smithi106.stdout: "stretch_mode": false 2024-04-25T10:33:34.041 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-25T10:33:34.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.027+0000 7f6a125d6640 1 -- 172.21.15.106:0/520996920 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f69f80743b0 msgr2=0x7f69f80768c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:34.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.027+0000 7f6a125d6640 1 --2- 172.21.15.106:0/520996920 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f69f80743b0 0x7f69f80768c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f69f400c1c0 tx=0x7f6a04007000 comp rx=0 tx=0).stop 2024-04-25T10:33:34.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.028+0000 7f6a10b73640 1 -- 172.21.15.106:0/520996920 reap_dead start 2024-04-25T10:33:34.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.028+0000 7f6a125d6640 1 -- 172.21.15.106:0/520996920 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f69f4005270 msgr2=0x7f6a0c155ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:34.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.028+0000 7f6a125d6640 1 --2- 172.21.15.106:0/520996920 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f69f4005270 0x7f6a0c155ae0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f69fc0108c0 tx=0x7f69fc010d80 comp rx=0 tx=0).stop 2024-04-25T10:33:34.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.028+0000 7f6a125d6640 1 -- 172.21.15.106:0/520996920 shutdown_connections 2024-04-25T10:33:34.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.028+0000 7f6a125d6640 1 --2- 172.21.15.106:0/520996920 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f69f4005270 0x7f6a0c155ae0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:34.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.028+0000 7f6a125d6640 1 -- 172.21.15.106:0/520996920 >> 172.21.15.106:0/520996920 conn(0x7f6a0c0cef20 msgr2=0x7f6a0c06c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:34.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.028+0000 7f6a125d6640 1 -- 172.21.15.106:0/520996920 shutdown_connections 2024-04-25T10:33:34.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.028+0000 7f6a125d6640 1 -- 172.21.15.106:0/520996920 wait complete. 2024-04-25T10:33:34.044 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T10:33:34.044 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-04-25T10:33:34.044 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-25T10:33:34.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.150+0000 7fad99e84640 1 Processor -- start 2024-04-25T10:33:34.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.150+0000 7fad99e84640 1 -- start start 2024-04-25T10:33:34.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.151+0000 7fad99e84640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fad940c7e30 0x7fad940c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:34.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.151+0000 7fad99e84640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad94156ef0 0x7fad940b4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:34.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.151+0000 7fad99e84640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad940ce630 con 0x7fad9406d000 2024-04-25T10:33:34.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.151+0000 7fad99e84640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fad940c7c60 con 0x7fad940c7e30 2024-04-25T10:33:34.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.151+0000 7fad99e84640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fad940c7980 con 0x7fad94156ef0 2024-04-25T10:33:34.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.151+0000 7fad937fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fad940c7e30 0x7fad940c8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:34.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.152+0000 7fad937fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fad940c7e30 0x7fad940c8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51670/0 (socket says 172.21.15.106:51670) 2024-04-25T10:33:34.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.152+0000 7fad937fe640 1 -- 172.21.15.106:0/1836840556 learned_addr learned my addr 172.21.15.106:0/1836840556 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:33:34.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.152+0000 7fad93fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fad9406d000 0x7fad9415ce30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:46360/0 (socket says 172.21.15.106:46360) 2024-04-25T10:33:34.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.152+0000 7fad92ffd640 1 --2- 172.21.15.106:0/1836840556 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad94156ef0 0x7fad940b4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:34.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.152+0000 7fad937fe640 1 -- 172.21.15.106:0/1836840556 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad94156ef0 msgr2=0x7fad940b4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:34.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.152+0000 7fad937fe640 1 --2- 172.21.15.106:0/1836840556 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad94156ef0 0x7fad940b4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:34.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.152+0000 7fad937fe640 1 -- 172.21.15.106:0/1836840556 >> v1:172.21.15.106:6789/0 conn(0x7fad9406d000 legacy=0x7fad9415ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:34.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.152+0000 7fad937fe640 1 -- 172.21.15.106:0/1836840556 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad940c7010 con 0x7fad940c7e30 2024-04-25T10:33:34.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.152+0000 7fad927fc640 1 -- 172.21.15.106:0/1836840556 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 11745288 0 0) 0x7fad940ce630 con 0x7fad9406d000 2024-04-25T10:33:34.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.152+0000 7fad937fe640 1 --2- 172.21.15.106:0/1836840556 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad940c7e30 0x7fad940c8d90 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fad940c7de0 tx=0x7fad8402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da7a753bd16707f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:34.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.153+0000 7fad927fc640 1 -- 172.21.15.106:0/1836840556 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad84037070 con 0x7fad940c7e30 2024-04-25T10:33:34.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.153+0000 7fad927fc640 1 -- 172.21.15.106:0/1836840556 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad84032440 con 0x7fad940c7e30 2024-04-25T10:33:34.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.153+0000 7fad927fc640 1 -- 172.21.15.106:0/1836840556 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad84032810 con 0x7fad940c7e30 2024-04-25T10:33:34.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.153+0000 7fad99e84640 1 -- 172.21.15.106:0/1836840556 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad940c7e30 msgr2=0x7fad940c8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:34.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.153+0000 7fad99e84640 1 --2- 172.21.15.106:0/1836840556 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad940c7e30 0x7fad940c8d90 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fad940c7de0 tx=0x7fad8402a1c0 comp rx=0 tx=0).stop 2024-04-25T10:33:34.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.153+0000 7fad99e84640 1 -- 172.21.15.106:0/1836840556 shutdown_connections 2024-04-25T10:33:34.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.153+0000 7fad99e84640 1 --2- 172.21.15.106:0/1836840556 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad94156ef0 0x7fad940b4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:34.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.153+0000 7fad99e84640 1 --2- 172.21.15.106:0/1836840556 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad940c7e30 0x7fad940c8d90 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:34.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.153+0000 7fad99e84640 1 -- 172.21.15.106:0/1836840556 >> 172.21.15.106:0/1836840556 conn(0x7fad94068b10 msgr2=0x7fad940d1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:34.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.153+0000 7fad99e84640 1 -- 172.21.15.106:0/1836840556 shutdown_connections 2024-04-25T10:33:34.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.153+0000 7fad99e84640 1 -- 172.21.15.106:0/1836840556 wait complete. 2024-04-25T10:33:34.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.155+0000 7fad99e84640 1 Processor -- start 2024-04-25T10:33:34.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.155+0000 7fad99e84640 1 -- start start 2024-04-25T10:33:34.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.156+0000 7fad99e84640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad9406d000 0x7fad9415c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:34.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.156+0000 7fad99e84640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad940c7e30 0x7fad941e9330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:34.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.156+0000 7fad99e84640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad94156ef0 0x7fad941f6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:34.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.156+0000 7fad99e84640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fad9415d6c0 con 0x7fad94156ef0 2024-04-25T10:33:34.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.157+0000 7fad99e84640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fad9415d540 con 0x7fad940c7e30 2024-04-25T10:33:34.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.157+0000 7fad99e84640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fad941f7150 con 0x7fad9406d000 2024-04-25T10:33:34.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.157+0000 7fad93fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad94156ef0 0x7fad941f6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:34.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.157+0000 7fad93fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad94156ef0 0x7fad941f6200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51680/0 (socket says 172.21.15.106:51680) 2024-04-25T10:33:34.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.157+0000 7fad93fff640 1 -- 172.21.15.106:0/3266791729 learned_addr learned my addr 172.21.15.106:0/3266791729 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:33:34.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.157+0000 7fad937fe640 1 --2- 172.21.15.106:0/3266791729 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad9406d000 0x7fad9415c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:34.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.157+0000 7fad92ffd640 1 --2- 172.21.15.106:0/3266791729 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad940c7e30 0x7fad941e9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:34.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.157+0000 7fad93fff640 1 -- 172.21.15.106:0/3266791729 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad9406d000 msgr2=0x7fad9415c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:34.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.157+0000 7fad93fff640 1 --2- 172.21.15.106:0/3266791729 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad9406d000 0x7fad9415c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:34.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.157+0000 7fad93fff640 1 -- 172.21.15.106:0/3266791729 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad940c7e30 msgr2=0x7fad941e9330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:34.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.157+0000 7fad93fff640 1 --2- 172.21.15.106:0/3266791729 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad940c7e30 0x7fad941e9330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:34.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.157+0000 7fad93fff640 1 -- 172.21.15.106:0/3266791729 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad9415ca70 con 0x7fad94156ef0 2024-04-25T10:33:34.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.157+0000 7fad93fff640 1 --2- 172.21.15.106:0/3266791729 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad94156ef0 0x7fad941f6200 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fad8c00c700 tx=0x7fad8c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:34.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.157+0000 7fad7bfff640 1 -- 172.21.15.106:0/3266791729 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad8c018070 con 0x7fad94156ef0 2024-04-25T10:33:34.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.157+0000 7fad7bfff640 1 -- 172.21.15.106:0/3266791729 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad8c00ce10 con 0x7fad94156ef0 2024-04-25T10:33:34.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.157+0000 7fad99e84640 1 -- 172.21.15.106:0/3266791729 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad9415c5a0 con 0x7fad94156ef0 2024-04-25T10:33:34.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.157+0000 7fad7bfff640 1 -- 172.21.15.106:0/3266791729 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad8c00a880 con 0x7fad94156ef0 2024-04-25T10:33:34.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.157+0000 7fad99e84640 1 -- 172.21.15.106:0/3266791729 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad941e9e00 con 0x7fad94156ef0 2024-04-25T10:33:34.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.158+0000 7fad99e84640 1 --2- 172.21.15.106:0/3266791729 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad940d0810 0x7fad9415b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:34.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.158+0000 7fad99e84640 1 -- 172.21.15.106:0/3266791729 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fad9415d840 con 0x7fad940d0810 2024-04-25T10:33:34.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.158+0000 7fad937fe640 1 --2- 172.21.15.106:0/3266791729 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad940d0810 0x7fad9415b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:34.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.159+0000 7fad937fe640 1 -- 172.21.15.106:0/3266791729 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fad9415b9d0 con 0x7fad940d0810 2024-04-25T10:33:34.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.159+0000 7fad937fe640 1 --2- 172.21.15.106:0/3266791729 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad940d0810 0x7fad9415b490 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fad94090190 tx=0x7fad840046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:34.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.159+0000 7fad7bfff640 1 -- 172.21.15.106:0/3266791729 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fad8c01b070 con 0x7fad94156ef0 2024-04-25T10:33:34.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.159+0000 7fad7bfff640 1 --2- 172.21.15.106:0/3266791729 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fad800744b0 0x7fad800769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:34.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.160+0000 7fad92ffd640 1 --2- 172.21.15.106:0/3266791729 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fad800744b0 0x7fad800769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:34.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.160+0000 7fad7bfff640 1 -- 172.21.15.106:0/3266791729 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad84042070 con 0x7fad940d0810 2024-04-25T10:33:34.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.160+0000 7fad7bfff640 1 -- 172.21.15.106:0/3266791729 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fad8c094780 con 0x7fad94156ef0 2024-04-25T10:33:34.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.160+0000 7fad7bfff640 1 -- 172.21.15.106:0/3266791729 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fad8403e660 con 0x7fad940d0810 2024-04-25T10:33:34.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.160+0000 7fad7bfff640 1 -- 172.21.15.106:0/3266791729 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad940d0810 msgr2=0x7fad9415b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:34.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.160+0000 7fad7bfff640 1 --2- 172.21.15.106:0/3266791729 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad940d0810 0x7fad9415b490 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fad94090190 tx=0x7fad840046c0 comp rx=0 tx=0).stop 2024-04-25T10:33:34.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.160+0000 7fad7bfff640 1 -- 172.21.15.106:0/3266791729 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad940d0810 msgr2=0x7fad9415b490 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T10:33:34.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.160+0000 7fad7bfff640 1 --2- 172.21.15.106:0/3266791729 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad940d0810 0x7fad9415b490 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:34.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.160+0000 7fad92ffd640 1 --2- 172.21.15.106:0/3266791729 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fad800744b0 0x7fad800769c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fad9414e5f0 tx=0x7fad88008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:34.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.166+0000 7fad99e84640 1 --2- 172.21.15.106:0/3266791729 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad94149cb0 0x7fad9414a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:34.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.166+0000 7fad99e84640 1 -- 172.21.15.106:0/3266791729 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fad94150380 con 0x7fad94149cb0 2024-04-25T10:33:34.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.166+0000 7fad937fe640 1 --2- 172.21.15.106:0/3266791729 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad94149cb0 0x7fad9414a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:34.169 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.167+0000 7fad937fe640 1 -- 172.21.15.106:0/3266791729 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fad9415a5a0 con 0x7fad94149cb0 2024-04-25T10:33:34.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.167+0000 7fad937fe640 1 --2- 172.21.15.106:0/3266791729 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad94149cb0 0x7fad9414a0c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fad9415a570 tx=0x7fad84009240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:34.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.167+0000 7fad7bfff640 1 -- 172.21.15.106:0/3266791729 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad8404e070 con 0x7fad94149cb0 2024-04-25T10:33:34.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.167+0000 7fad7bfff640 1 -- 172.21.15.106:0/3266791729 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4728 (secure 0 0 0) 0x7fad8403c710 con 0x7fad94149cb0 2024-04-25T10:33:34.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.167+0000 7fad7bfff640 1 -- 172.21.15.106:0/3266791729 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad94149cb0 msgr2=0x7fad9414a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:34.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.167+0000 7fad7bfff640 1 --2- 172.21.15.106:0/3266791729 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad94149cb0 0x7fad9414a0c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fad9415a570 tx=0x7fad84009240 comp rx=0 tx=0).stop 2024-04-25T10:33:34.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.167+0000 7fad7bfff640 1 -- 172.21.15.106:0/3266791729 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad94149cb0 msgr2=0x7fad9414a0c0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T10:33:34.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.167+0000 7fad7bfff640 1 --2- 172.21.15.106:0/3266791729 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad94149cb0 0x7fad9414a0c0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:34.170 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-25T10:33:34.171 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "c", 2024-04-25T10:33:34.171 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 2, 2024-04-25T10:33:34.171 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "peon", 2024-04-25T10:33:34.171 INFO:teuthology.orchestra.run.smithi106.stdout: "election_epoch": 22, 2024-04-25T10:33:34.171 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 53091, 2024-04-25T10:33:34.171 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum": [ 2024-04-25T10:33:34.171 INFO:teuthology.orchestra.run.smithi106.stdout: 0, 2024-04-25T10:33:34.171 INFO:teuthology.orchestra.run.smithi106.stdout: 1, 2024-04-25T10:33:34.171 INFO:teuthology.orchestra.run.smithi106.stdout: 2 2024-04-25T10:33:34.171 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:33:34.171 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_age": 0, 2024-04-25T10:33:34.171 INFO:teuthology.orchestra.run.smithi106.stdout: "features": { 2024-04-25T10:33:34.171 INFO:teuthology.orchestra.run.smithi106.stdout: "required_con": "2450521708007866372", 2024-04-25T10:33:34.171 INFO:teuthology.orchestra.run.smithi106.stdout: "required_mon": [ 2024-04-25T10:33:34.172 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:33:34.172 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:33:34.172 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:33:34.172 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:33:34.172 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:33:34.172 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:33:34.172 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:33:34.172 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:33:34.172 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:33:34.172 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:33:34.172 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:33:34.172 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:33:34.172 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_con": "4540701547738038271", 2024-04-25T10:33:34.172 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_mon": [ 2024-04-25T10:33:34.173 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:33:34.173 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:33:34.173 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:33:34.173 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:33:34.173 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:33:34.173 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:33:34.173 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:33:34.173 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:33:34.173 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:33:34.173 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:33:34.173 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:33:34.173 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:33:34.173 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:34.173 INFO:teuthology.orchestra.run.smithi106.stdout: "outside_quorum": [], 2024-04-25T10:33:34.174 INFO:teuthology.orchestra.run.smithi106.stdout: "extra_probe_peers": [], 2024-04-25T10:33:34.174 INFO:teuthology.orchestra.run.smithi106.stdout: "sync_provider": [], 2024-04-25T10:33:34.174 INFO:teuthology.orchestra.run.smithi106.stdout: "monmap": { 2024-04-25T10:33:34.174 INFO:teuthology.orchestra.run.smithi106.stdout: "epoch": 1, 2024-04-25T10:33:34.174 INFO:teuthology.orchestra.run.smithi106.stdout: "fsid": "3ee77985-e8e4-41cf-81e9-7c774cf7891f", 2024-04-25T10:33:34.174 INFO:teuthology.orchestra.run.smithi106.stdout: "modified": "2024-04-25T10:29:33.446228Z", 2024-04-25T10:33:34.174 INFO:teuthology.orchestra.run.smithi106.stdout: "created": "2024-04-25T10:29:33.446228Z", 2024-04-25T10:33:34.174 INFO:teuthology.orchestra.run.smithi106.stdout: "min_mon_release": 19, 2024-04-25T10:33:34.174 INFO:teuthology.orchestra.run.smithi106.stdout: "min_mon_release_name": "squid", 2024-04-25T10:33:34.174 INFO:teuthology.orchestra.run.smithi106.stdout: "election_strategy": 1, 2024-04-25T10:33:34.174 INFO:teuthology.orchestra.run.smithi106.stdout: "disallowed_leaders: ": "", 2024-04-25T10:33:34.174 INFO:teuthology.orchestra.run.smithi106.stdout: "stretch_mode": false, 2024-04-25T10:33:34.174 INFO:teuthology.orchestra.run.smithi106.stdout: "tiebreaker_mon": "", 2024-04-25T10:33:34.175 INFO:teuthology.orchestra.run.smithi106.stdout: "removed_ranks: ": "", 2024-04-25T10:33:34.175 INFO:teuthology.orchestra.run.smithi106.stdout: "features": { 2024-04-25T10:33:34.175 INFO:teuthology.orchestra.run.smithi106.stdout: "persistent": [ 2024-04-25T10:33:34.175 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:33:34.175 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:33:34.175 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:33:34.175 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:33:34.175 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:33:34.175 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:33:34.176 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:33:34.176 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:33:34.176 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:33:34.176 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:33:34.176 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:33:34.176 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:33:34.176 INFO:teuthology.orchestra.run.smithi106.stdout: "optional": [] 2024-04-25T10:33:34.176 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:34.176 INFO:teuthology.orchestra.run.smithi106.stdout: "mons": [ 2024-04-25T10:33:34.176 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:34.176 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 0, 2024-04-25T10:33:34.176 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "a", 2024-04-25T10:33:34.176 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:33:34.176 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:33:34.177 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:34.177 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:33:34.177 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:3300", 2024-04-25T10:33:34.177 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:33:34.177 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:34.177 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:34.177 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:33:34.177 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:6789", 2024-04-25T10:33:34.177 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:33:34.177 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:33:34.177 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:33:34.177 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:34.177 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:6789/0", 2024-04-25T10:33:34.178 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.106:6789/0", 2024-04-25T10:33:34.178 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:33:34.178 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:33:34.178 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:33:34.178 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:34.178 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:34.178 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 1, 2024-04-25T10:33:34.178 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "b", 2024-04-25T10:33:34.178 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:33:34.178 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:33:34.178 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:34.178 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:33:34.178 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:3300", 2024-04-25T10:33:34.178 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:33:34.179 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:34.179 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:34.179 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:33:34.179 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6789", 2024-04-25T10:33:34.179 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:33:34.179 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:33:34.179 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:33:34.179 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:34.179 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6789/0", 2024-04-25T10:33:34.179 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.134:6789/0", 2024-04-25T10:33:34.179 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:33:34.179 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:33:34.179 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:33:34.179 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:34.180 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:34.180 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 2, 2024-04-25T10:33:34.180 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "c", 2024-04-25T10:33:34.180 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:33:34.180 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:33:34.180 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:34.180 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:33:34.180 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:3301", 2024-04-25T10:33:34.180 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:33:34.180 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:34.180 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:34.180 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:33:34.180 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6790", 2024-04-25T10:33:34.180 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:33:34.180 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:33:34.181 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:33:34.181 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:34.181 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6790/0", 2024-04-25T10:33:34.181 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.134:6790/0", 2024-04-25T10:33:34.181 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:33:34.181 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:33:34.181 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:33:34.181 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:33:34.181 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:33:34.181 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:34.181 INFO:teuthology.orchestra.run.smithi106.stdout: "feature_map": { 2024-04-25T10:33:34.181 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": [ 2024-04-25T10:33:34.181 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:34.181 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:33:34.181 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:33:34.181 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 1 2024-04-25T10:33:34.182 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:33:34.182 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:33:34.182 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": [ 2024-04-25T10:33:34.182 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:34.182 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:33:34.182 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:33:34.182 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 2 2024-04-25T10:33:34.182 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:33:34.182 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:33:34.182 INFO:teuthology.orchestra.run.smithi106.stdout: "client": [ 2024-04-25T10:33:34.182 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:34.182 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:33:34.182 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:33:34.182 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 1 2024-04-25T10:33:34.183 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:33:34.183 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:33:34.183 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:34.183 INFO:teuthology.orchestra.run.smithi106.stdout: "stretch_mode": false 2024-04-25T10:33:34.183 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-25T10:33:34.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.168+0000 7fad99e84640 1 -- 172.21.15.106:0/3266791729 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fad800744b0 msgr2=0x7fad800769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:34.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.168+0000 7fad99e84640 1 --2- 172.21.15.106:0/3266791729 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fad800744b0 0x7fad800769c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fad9414e5f0 tx=0x7fad88008040 comp rx=0 tx=0).stop 2024-04-25T10:33:34.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.168+0000 7fad99e84640 1 -- 172.21.15.106:0/3266791729 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad94156ef0 msgr2=0x7fad941f6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:34.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.168+0000 7fad99e84640 1 --2- 172.21.15.106:0/3266791729 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad94156ef0 0x7fad941f6200 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fad8c00c700 tx=0x7fad8c00cbc0 comp rx=0 tx=0).stop 2024-04-25T10:33:34.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.168+0000 7fad93fff640 1 -- 172.21.15.106:0/3266791729 reap_dead start 2024-04-25T10:33:34.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.168+0000 7fad93fff640 1 -- 172.21.15.106:0/3266791729 reap_dead start 2024-04-25T10:33:34.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.168+0000 7fad99e84640 1 -- 172.21.15.106:0/3266791729 shutdown_connections 2024-04-25T10:33:34.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.168+0000 7fad99e84640 1 -- 172.21.15.106:0/3266791729 >> 172.21.15.106:0/3266791729 conn(0x7fad94068b10 msgr2=0x7fad94075c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:34.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.168+0000 7fad99e84640 1 -- 172.21.15.106:0/3266791729 shutdown_connections 2024-04-25T10:33:34.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.168+0000 7fad99e84640 1 -- 172.21.15.106:0/3266791729 wait complete. 2024-04-25T10:33:34.185 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T10:33:34.185 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-04-25T10:33:34.185 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-04-25T10:33:34.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.293+0000 7fa6c16bd640 1 Processor -- start 2024-04-25T10:33:34.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.293+0000 7fa6c16bd640 1 -- start start 2024-04-25T10:33:34.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.294+0000 7fa6c16bd640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fa6bc0c9920 0x7fa6bc156c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:34.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.294+0000 7fa6c16bd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa6bc15d650 0x7fa6bc0c7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:34.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.294+0000 7fa6c16bd640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6bc160340 con 0x7fa6bc074630 2024-04-25T10:33:34.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.294+0000 7fa6c16bd640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fa6bc15dbe0 con 0x7fa6bc15d650 2024-04-25T10:33:34.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.294+0000 7fa6c16bd640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fa6bc160620 con 0x7fa6bc0c9920 2024-04-25T10:33:34.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.294+0000 7fa6bb7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa6bc15d650 0x7fa6bc0c7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:34.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.294+0000 7fa6bb7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa6bc15d650 0x7fa6bc0c7a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51684/0 (socket says 172.21.15.106:51684) 2024-04-25T10:33:34.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.294+0000 7fa6bb7fe640 1 -- 172.21.15.106:0/2151678272 learned_addr learned my addr 172.21.15.106:0/2151678272 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:33:34.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.294+0000 7fa6baffd640 1 --2- 172.21.15.106:0/2151678272 >> v2:172.21.15.134:3300/0 conn(0x7fa6bc0c9920 0x7fa6bc156c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:34.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.295+0000 7fa6bb7fe640 1 -- 172.21.15.106:0/2151678272 >> v2:172.21.15.134:3300/0 conn(0x7fa6bc0c9920 msgr2=0x7fa6bc156c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:34.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.295+0000 7fa6bb7fe640 1 --2- 172.21.15.106:0/2151678272 >> v2:172.21.15.134:3300/0 conn(0x7fa6bc0c9920 0x7fa6bc156c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:34.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.295+0000 7fa6bb7fe640 1 -- 172.21.15.106:0/2151678272 >> v1:172.21.15.134:6789/0 conn(0x7fa6bc074630 legacy=0x7fa6bc0738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:34.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.295+0000 7fa6bb7fe640 1 -- 172.21.15.106:0/2151678272 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6bc160900 con 0x7fa6bc15d650 2024-04-25T10:33:34.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.295+0000 7fa6bb7fe640 1 --2- 172.21.15.106:0/2151678272 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa6bc15d650 0x7fa6bc0c7a80 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc06e4d0 tx=0x7fa6b402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=165d8642d2bd70f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:34.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.296+0000 7fa6b9ffb640 1 -- 172.21.15.106:0/2151678272 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b4036070 con 0x7fa6bc15d650 2024-04-25T10:33:34.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.296+0000 7fa6b9ffb640 1 -- 172.21.15.106:0/2151678272 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6b4031600 con 0x7fa6bc15d650 2024-04-25T10:33:34.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.296+0000 7fa6b9ffb640 1 -- 172.21.15.106:0/2151678272 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b40319d0 con 0x7fa6bc15d650 2024-04-25T10:33:34.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.296+0000 7fa6c16bd640 1 -- 172.21.15.106:0/2151678272 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa6bc15d650 msgr2=0x7fa6bc0c7a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:34.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.296+0000 7fa6c16bd640 1 --2- 172.21.15.106:0/2151678272 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa6bc15d650 0x7fa6bc0c7a80 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc06e4d0 tx=0x7fa6b402a6a0 comp rx=0 tx=0).stop 2024-04-25T10:33:34.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.296+0000 7fa6c16bd640 1 -- 172.21.15.106:0/2151678272 shutdown_connections 2024-04-25T10:33:34.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.296+0000 7fa6c16bd640 1 --2- 172.21.15.106:0/2151678272 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa6bc15d650 0x7fa6bc0c7a80 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:34.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.296+0000 7fa6c16bd640 1 --2- 172.21.15.106:0/2151678272 >> v2:172.21.15.134:3300/0 conn(0x7fa6bc0c9920 0x7fa6bc156c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:34.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.296+0000 7fa6c16bd640 1 -- 172.21.15.106:0/2151678272 >> 172.21.15.106:0/2151678272 conn(0x7fa6bc06d6d0 msgr2=0x7fa6bc06dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:34.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.296+0000 7fa6c16bd640 1 -- 172.21.15.106:0/2151678272 shutdown_connections 2024-04-25T10:33:34.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.296+0000 7fa6c16bd640 1 -- 172.21.15.106:0/2151678272 wait complete. 2024-04-25T10:33:34.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.298+0000 7fa6c16bd640 1 Processor -- start 2024-04-25T10:33:34.300 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.298+0000 7fa6c16bd640 1 -- start start 2024-04-25T10:33:34.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.299+0000 7fa6c16bd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa6bc074630 0x7fa6bc0cb230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:34.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.299+0000 7fa6c16bd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa6bc0c9920 0x7fa6bc0ca610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:34.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.299+0000 7fa6c16bd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa6bc15d650 0x7fa6bc14fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:34.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.299+0000 7fa6c16bd640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa6bc073fe0 con 0x7fa6bc0c9920 2024-04-25T10:33:34.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.299+0000 7fa6c16bd640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa6bc06e280 con 0x7fa6bc15d650 2024-04-25T10:33:34.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.299+0000 7fa6c16bd640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa6bc074160 con 0x7fa6bc074630 2024-04-25T10:33:34.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.300+0000 7fa6ba7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa6bc0c9920 0x7fa6bc0ca610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:34.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.300+0000 7fa6ba7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa6bc0c9920 0x7fa6bc0ca610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51694/0 (socket says 172.21.15.106:51694) 2024-04-25T10:33:34.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.300+0000 7fa6ba7fc640 1 -- 172.21.15.106:0/3142631227 learned_addr learned my addr 172.21.15.106:0/3142631227 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:33:34.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.300+0000 7fa6bb7fe640 1 --2- 172.21.15.106:0/3142631227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa6bc15d650 0x7fa6bc14fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:34.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.300+0000 7fa6baffd640 1 --2- 172.21.15.106:0/3142631227 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa6bc074630 0x7fa6bc0cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:34.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.300+0000 7fa6bb7fe640 1 -- 172.21.15.106:0/3142631227 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa6bc074630 msgr2=0x7fa6bc0cb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:34.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.300+0000 7fa6bb7fe640 1 --2- 172.21.15.106:0/3142631227 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa6bc074630 0x7fa6bc0cb230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:34.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.300+0000 7fa6bb7fe640 1 -- 172.21.15.106:0/3142631227 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa6bc0c9920 msgr2=0x7fa6bc0ca610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:34.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.300+0000 7fa6bb7fe640 1 --2- 172.21.15.106:0/3142631227 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa6bc0c9920 0x7fa6bc0ca610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:34.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.300+0000 7fa6bb7fe640 1 -- 172.21.15.106:0/3142631227 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6bc0cb770 con 0x7fa6bc15d650 2024-04-25T10:33:34.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.300+0000 7fa6ba7fc640 1 --2- 172.21.15.106:0/3142631227 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa6bc0c9920 0x7fa6bc0ca610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:33:34.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.300+0000 7fa6bb7fe640 1 --2- 172.21.15.106:0/3142631227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa6bc15d650 0x7fa6bc14fa70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc069410 tx=0x7fa6b4004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:34.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.300+0000 7fa6a37fe640 1 -- 172.21.15.106:0/3142631227 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b4036030 con 0x7fa6bc15d650 2024-04-25T10:33:34.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.300+0000 7fa6a37fe640 1 -- 172.21.15.106:0/3142631227 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6b4040070 con 0x7fa6bc15d650 2024-04-25T10:33:34.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.300+0000 7fa6a37fe640 1 -- 172.21.15.106:0/3142631227 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b4009510 con 0x7fa6bc15d650 2024-04-25T10:33:34.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.300+0000 7fa6c16bd640 1 -- 172.21.15.106:0/3142631227 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6bc0c5d20 con 0x7fa6bc15d650 2024-04-25T10:33:34.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.300+0000 7fa6c16bd640 1 -- 172.21.15.106:0/3142631227 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6bc1f3f10 con 0x7fa6bc15d650 2024-04-25T10:33:34.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.302+0000 7fa6c16bd640 1 -- 172.21.15.106:0/3142631227 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6bc06a3d0 con 0x7fa6bc15d650 2024-04-25T10:33:34.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.302+0000 7fa6a37fe640 1 -- 172.21.15.106:0/3142631227 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa6b4007bf0 con 0x7fa6bc15d650 2024-04-25T10:33:34.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.302+0000 7fa6a37fe640 1 --2- 172.21.15.106:0/3142631227 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fa6a80744b0 0x7fa6a80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:34.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.302+0000 7fa6a37fe640 1 -- 172.21.15.106:0/3142631227 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fa6b40b5910 con 0x7fa6bc15d650 2024-04-25T10:33:34.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.302+0000 7fa6baffd640 1 --2- 172.21.15.106:0/3142631227 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fa6a80744b0 0x7fa6a80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:34.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.303+0000 7fa6baffd640 1 --2- 172.21.15.106:0/3142631227 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fa6a80744b0 0x7fa6a80769c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac009aa0 tx=0x7fa6ac009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:34.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.307+0000 7fa6a37fe640 1 -- 172.21.15.106:0/3142631227 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa6b407f9d0 con 0x7fa6bc15d650 2024-04-25T10:33:34.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.556+0000 7fa6c16bd640 1 -- 172.21.15.106:0/3142631227 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7fa6bc0c5fc0 con 0x7fa6bc15d650 2024-04-25T10:33:34.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.557+0000 7fa6a37fe640 1 -- 172.21.15.106:0/3142631227 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7fa6b4083cd0 con 0x7fa6bc15d650 2024-04-25T10:33:34.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.559+0000 7fa6c16bd640 1 -- 172.21.15.106:0/3142631227 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fa6a80744b0 msgr2=0x7fa6a80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:34.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.559+0000 7fa6c16bd640 1 --2- 172.21.15.106:0/3142631227 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fa6a80744b0 0x7fa6a80769c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac009aa0 tx=0x7fa6ac009190 comp rx=0 tx=0).stop 2024-04-25T10:33:34.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.559+0000 7fa6c16bd640 1 -- 172.21.15.106:0/3142631227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa6bc15d650 msgr2=0x7fa6bc14fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:34.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.559+0000 7fa6c16bd640 1 --2- 172.21.15.106:0/3142631227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa6bc15d650 0x7fa6bc14fa70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc069410 tx=0x7fa6b4004a40 comp rx=0 tx=0).stop 2024-04-25T10:33:34.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.559+0000 7fa6c16bd640 1 -- 172.21.15.106:0/3142631227 shutdown_connections 2024-04-25T10:33:34.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.559+0000 7fa6c16bd640 1 --2- 172.21.15.106:0/3142631227 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fa6a80744b0 0x7fa6a80769c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:34.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.559+0000 7fa6c16bd640 1 --2- 172.21.15.106:0/3142631227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa6bc15d650 0x7fa6bc14fa70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:34.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.559+0000 7fa6c16bd640 1 --2- 172.21.15.106:0/3142631227 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa6bc0c9920 0x7fa6bc0ca610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:34.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.559+0000 7fa6c16bd640 1 --2- 172.21.15.106:0/3142631227 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa6bc074630 0x7fa6bc0cb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:34.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.559+0000 7fa6c16bd640 1 -- 172.21.15.106:0/3142631227 >> 172.21.15.106:0/3142631227 conn(0x7fa6bc06d6d0 msgr2=0x7fa6bc1e6070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:34.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.559+0000 7fa6c16bd640 1 -- 172.21.15.106:0/3142631227 shutdown_connections 2024-04-25T10:33:34.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:34.559+0000 7fa6c16bd640 1 -- 172.21.15.106:0/3142631227 wait complete. 2024-04-25T10:33:34.573 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-04-25T10:33:36.587 INFO:tasks.workunit.client.0.smithi106.stdout: 2024-04-25T10:33:36.587 INFO:tasks.workunit.client.0.smithi106.stdout: 2024-04-25T10:33:36.587 INFO:tasks.workunit.client.0.smithi106.stdout: 2024-04-25T10:33:36.587 INFO:tasks.workunit.client.0.smithi106.stdout:iozone test complete. 2024-04-25T10:33:36.588 INFO:tasks.workunit.client.0.smithi106.stderr:+ iozone -c -e -s 10240M -r 1M -t 1 -F f3 -i 0 -i 1 2024-04-25T10:33:36.654 INFO:tasks.workunit.client.0.smithi106.stdout: Iozone: Performance Test of File I/O 2024-04-25T10:33:36.654 INFO:tasks.workunit.client.0.smithi106.stdout: Version $Revision: 3.494 $ 2024-04-25T10:33:36.654 INFO:tasks.workunit.client.0.smithi106.stdout: Compiled for 64 bit mode. 2024-04-25T10:33:36.654 INFO:tasks.workunit.client.0.smithi106.stdout: Build: linux-AMD64 2024-04-25T10:33:36.654 INFO:tasks.workunit.client.0.smithi106.stdout: 2024-04-25T10:33:36.654 INFO:tasks.workunit.client.0.smithi106.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-04-25T10:33:36.654 INFO:tasks.workunit.client.0.smithi106.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-04-25T10:33:36.654 INFO:tasks.workunit.client.0.smithi106.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-04-25T10:33:36.654 INFO:tasks.workunit.client.0.smithi106.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-04-25T10:33:36.654 INFO:tasks.workunit.client.0.smithi106.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-04-25T10:33:36.654 INFO:tasks.workunit.client.0.smithi106.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-04-25T10:33:36.655 INFO:tasks.workunit.client.0.smithi106.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-04-25T10:33:36.655 INFO:tasks.workunit.client.0.smithi106.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-04-25T10:33:36.655 INFO:tasks.workunit.client.0.smithi106.stdout: Alexey Skidanov, Sudhir Kumar. 2024-04-25T10:33:36.655 INFO:tasks.workunit.client.0.smithi106.stdout: 2024-04-25T10:33:36.655 INFO:tasks.workunit.client.0.smithi106.stdout: Run began: Thu Apr 25 10:33:36 2024 2024-04-25T10:33:36.655 INFO:tasks.workunit.client.0.smithi106.stdout: 2024-04-25T10:33:36.655 INFO:tasks.workunit.client.0.smithi106.stdout: Include close in write timing 2024-04-25T10:33:36.655 INFO:tasks.workunit.client.0.smithi106.stdout: Include fsync in write timing 2024-04-25T10:33:36.655 INFO:tasks.workunit.client.0.smithi106.stdout: File size set to 10485760 kB 2024-04-25T10:33:36.655 INFO:tasks.workunit.client.0.smithi106.stdout: Record Size 1024 kB 2024-04-25T10:33:36.655 INFO:tasks.workunit.client.0.smithi106.stdout: Command line used: iozone -c -e -s 10240M -r 1M -t 1 -F f3 -i 0 -i 1 2024-04-25T10:33:36.655 INFO:tasks.workunit.client.0.smithi106.stdout: Output is in kBytes/sec 2024-04-25T10:33:36.655 INFO:tasks.workunit.client.0.smithi106.stdout: Time Resolution = 0.000001 seconds. 2024-04-25T10:33:36.655 INFO:tasks.workunit.client.0.smithi106.stdout: Processor cache size set to 1024 kBytes. 2024-04-25T10:33:36.655 INFO:tasks.workunit.client.0.smithi106.stdout: Processor cache line size set to 32 bytes. 2024-04-25T10:33:36.656 INFO:tasks.workunit.client.0.smithi106.stdout: File stride size set to 17 * record size. 2024-04-25T10:33:36.656 INFO:tasks.workunit.client.0.smithi106.stdout: Throughput test with 1 process 2024-04-25T10:33:36.656 INFO:tasks.workunit.client.0.smithi106.stdout: Each process writes a 10485760 kByte file in 1024 kByte records 2024-04-25T10:33:44.575 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-04-25T10:33:44.575 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T10:33:44.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:44.725+0000 7f54ea234640 1 Processor -- start 2024-04-25T10:33:44.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:44.725+0000 7f54ea234640 1 -- start start 2024-04-25T10:33:44.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:44.727+0000 7f54ea234640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f54e4074910 0x7f54e40c36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:44.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:44.727+0000 7f54ea234640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f54e4153c50 0x7f54e40c4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:44.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:44.727+0000 7f54ea234640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54e40c90c0 con 0x7f54e40c5450 2024-04-25T10:33:44.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:44.727+0000 7f54ea234640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f54e415a400 con 0x7f54e4074910 2024-04-25T10:33:44.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:44.727+0000 7f54ea234640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f54e415a630 con 0x7f54e4153c50 2024-04-25T10:33:44.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:44.727+0000 7f54e37fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f54e4074910 0x7f54e40c36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:44.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:44.727+0000 7f54e3fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f54e40c5450 0x7f54e40739c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:59780/0 (socket says 172.21.15.106:59780) 2024-04-25T10:33:44.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:44.727+0000 7f54e3fff640 1 -- 172.21.15.106:0/2093371023 learned_addr learned my addr 172.21.15.106:0/2093371023 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:33:44.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:44.727+0000 7f54e2ffd640 1 --2- 172.21.15.106:0/2093371023 >> v2:172.21.15.134:3300/0 conn(0x7f54e4153c50 0x7f54e40c4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:44.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:44.728+0000 7f54e37fe640 1 -- 172.21.15.106:0/2093371023 >> v2:172.21.15.134:3300/0 conn(0x7f54e4153c50 msgr2=0x7f54e40c4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:44.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:44.728+0000 7f54e37fe640 1 --2- 172.21.15.106:0/2093371023 >> v2:172.21.15.134:3300/0 conn(0x7f54e4153c50 0x7f54e40c4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:44.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:44.728+0000 7f54e37fe640 1 -- 172.21.15.106:0/2093371023 >> v1:172.21.15.106:6789/0 conn(0x7f54e40c5450 legacy=0x7f54e40739c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:44.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:44.728+0000 7f54e27fc640 1 -- 172.21.15.106:0/2093371023 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1975114221 0 0) 0x7f54e40c90c0 con 0x7f54e40c5450 2024-04-25T10:33:44.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:44.728+0000 7f54e37fe640 1 -- 172.21.15.106:0/2093371023 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54e40731c0 con 0x7f54e4074910 2024-04-25T10:33:44.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:44.729+0000 7f54e37fe640 1 --2- 172.21.15.106:0/2093371023 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54e4074910 0x7f54e40c36e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f54e4071eb0 tx=0x7f54d402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c56b654d05ffb5b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:44.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:44.729+0000 7f54e27fc640 1 -- 172.21.15.106:0/2093371023 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d4037070 con 0x7f54e4074910 2024-04-25T10:33:44.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:44.729+0000 7f54e27fc640 1 -- 172.21.15.106:0/2093371023 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54d4032440 con 0x7f54e4074910 2024-04-25T10:33:44.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:44.729+0000 7f54e27fc640 1 -- 172.21.15.106:0/2093371023 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d4032810 con 0x7f54e4074910 2024-04-25T10:33:44.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:44.730+0000 7f54ea234640 1 -- 172.21.15.106:0/2093371023 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54e4074910 msgr2=0x7f54e40c36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:44.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:44.730+0000 7f54ea234640 1 --2- 172.21.15.106:0/2093371023 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54e4074910 0x7f54e40c36e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f54e4071eb0 tx=0x7f54d402a1a0 comp rx=0 tx=0).stop 2024-04-25T10:33:44.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:44.730+0000 7f54ea234640 1 -- 172.21.15.106:0/2093371023 shutdown_connections 2024-04-25T10:33:44.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:44.730+0000 7f54ea234640 1 --2- 172.21.15.106:0/2093371023 >> v2:172.21.15.134:3300/0 conn(0x7f54e4153c50 0x7f54e40c4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:44.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:44.730+0000 7f54ea234640 1 --2- 172.21.15.106:0/2093371023 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54e4074910 0x7f54e40c36e0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:44.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:44.730+0000 7f54ea234640 1 -- 172.21.15.106:0/2093371023 >> 172.21.15.106:0/2093371023 conn(0x7f54e4070b30 msgr2=0x7f54e4070f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:44.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:44.730+0000 7f54ea234640 1 -- 172.21.15.106:0/2093371023 shutdown_connections 2024-04-25T10:33:44.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:44.730+0000 7f54ea234640 1 -- 172.21.15.106:0/2093371023 wait complete. 2024-04-25T10:33:44.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:44.732+0000 7f54ea234640 1 Processor -- start 2024-04-25T10:33:44.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:44.732+0000 7f54ea234640 1 -- start start 2024-04-25T10:33:44.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:44.734+0000 7f54ea234640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f54e4074910 0x7f54e4141580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:44.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:44.734+0000 7f54ea234640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54e40c5450 0x7f54e4142c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:44.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:44.734+0000 7f54ea234640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54e4153c50 0x7f54e4158040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:44.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:44.734+0000 7f54ea234640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f54e40c9360 con 0x7f54e40c5450 2024-04-25T10:33:44.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:44.734+0000 7f54ea234640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f54e40688e0 con 0x7f54e4153c50 2024-04-25T10:33:44.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:44.734+0000 7f54ea234640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f54e40c4670 con 0x7f54e4074910 2024-04-25T10:33:44.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:44.734+0000 7f54e2ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54e40c5450 0x7f54e4142c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:44.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:44.734+0000 7f54e2ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54e40c5450 0x7f54e4142c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:39438/0 (socket says 172.21.15.106:39438) 2024-04-25T10:33:44.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:44.734+0000 7f54e2ffd640 1 -- 172.21.15.106:0/1089232520 learned_addr learned my addr 172.21.15.106:0/1089232520 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:33:44.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:44.734+0000 7f54e37fe640 1 --2- 172.21.15.106:0/1089232520 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f54e4074910 0x7f54e4141580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:44.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:44.734+0000 7f54e2ffd640 1 -- 172.21.15.106:0/1089232520 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f54e4074910 msgr2=0x7f54e4141580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:44.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:44.734+0000 7f54e2ffd640 1 --2- 172.21.15.106:0/1089232520 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f54e4074910 0x7f54e4141580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:44.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:44.734+0000 7f54e2ffd640 1 -- 172.21.15.106:0/1089232520 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54e4153c50 msgr2=0x7f54e4158040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:44.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:44.734+0000 7f54e2ffd640 1 --2- 172.21.15.106:0/1089232520 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54e4153c50 0x7f54e4158040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:44.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:44.734+0000 7f54e2ffd640 1 -- 172.21.15.106:0/1089232520 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54e4140870 con 0x7f54e40c5450 2024-04-25T10:33:44.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:44.734+0000 7f54e37fe640 1 --2- 172.21.15.106:0/1089232520 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f54e4074910 0x7f54e4141580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:33:44.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:44.735+0000 7f54e2ffd640 1 --2- 172.21.15.106:0/1089232520 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54e40c5450 0x7f54e4142c70 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f54d800e870 tx=0x7f54d800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:44.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:44.735+0000 7f54cbfff640 1 -- 172.21.15.106:0/1089232520 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d8017070 con 0x7f54e40c5450 2024-04-25T10:33:44.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:44.735+0000 7f54ea234640 1 -- 172.21.15.106:0/1089232520 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54e4140b50 con 0x7f54e40c5450 2024-04-25T10:33:44.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:44.735+0000 7f54cbfff640 1 -- 172.21.15.106:0/1089232520 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54d800ee60 con 0x7f54e40c5450 2024-04-25T10:33:44.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:44.735+0000 7f54ea234640 1 -- 172.21.15.106:0/1089232520 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54e414e5b0 con 0x7f54e40c5450 2024-04-25T10:33:44.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:44.735+0000 7f54cbfff640 1 -- 172.21.15.106:0/1089232520 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d800cae0 con 0x7f54e40c5450 2024-04-25T10:33:44.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:44.736+0000 7f54ea234640 1 -- 172.21.15.106:0/1089232520 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54c4005290 con 0x7f54e40c5450 2024-04-25T10:33:44.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:44.738+0000 7f54cbfff640 1 -- 172.21.15.106:0/1089232520 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f54d801a070 con 0x7f54e40c5450 2024-04-25T10:33:44.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:44.738+0000 7f54cbfff640 1 --2- 172.21.15.106:0/1089232520 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f54d00744e0 0x7f54d00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:44.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:44.738+0000 7f54e37fe640 1 --2- 172.21.15.106:0/1089232520 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f54d00744e0 0x7f54d00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:44.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:44.739+0000 7f54cbfff640 1 -- 172.21.15.106:0/1089232520 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f54d8095430 con 0x7f54e40c5450 2024-04-25T10:33:44.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:44.739+0000 7f54e37fe640 1 --2- 172.21.15.106:0/1089232520 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f54d00744e0 0x7f54d00769f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f54d4034000 tx=0x7f54d402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:44.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:44.745+0000 7f54cbfff640 1 -- 172.21.15.106:0/1089232520 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f54d805fb50 con 0x7f54e40c5450 2024-04-25T10:33:45.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.117+0000 7f54ea234640 1 -- 172.21.15.106:0/1089232520 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f54c4005850 con 0x7f54e40c5450 2024-04-25T10:33:45.120 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.118+0000 7f54cbfff640 1 -- 172.21.15.106:0/1089232520 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1569 (secure 0 0 0) 0x7f54d805f450 con 0x7f54e40c5450 2024-04-25T10:33:45.120 INFO:teuthology.orchestra.run.smithi106.stdout:{"election_epoch":22,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"3ee77985-e8e4-41cf-81e9-7c774cf7891f","modified":"2024-04-25T10:29:33.446228Z","created":"2024-04-25T10:29:33.446228Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:3300","nonce":0},{"type":"v1","addr":"172.21.15.106:6789","nonce":0}]},"addr":"172.21.15.106:6789/0","public_addr":"172.21.15.106:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T10:33:45.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.122+0000 7f54ea234640 1 -- 172.21.15.106:0/1089232520 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f54d00744e0 msgr2=0x7f54d00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:45.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.122+0000 7f54ea234640 1 --2- 172.21.15.106:0/1089232520 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f54d00744e0 0x7f54d00769f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f54d4034000 tx=0x7f54d402e000 comp rx=0 tx=0).stop 2024-04-25T10:33:45.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.122+0000 7f54ea234640 1 -- 172.21.15.106:0/1089232520 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54e40c5450 msgr2=0x7f54e4142c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:45.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.122+0000 7f54ea234640 1 --2- 172.21.15.106:0/1089232520 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54e40c5450 0x7f54e4142c70 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f54d800e870 tx=0x7f54d800ed30 comp rx=0 tx=0).stop 2024-04-25T10:33:45.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.122+0000 7f54ea234640 1 -- 172.21.15.106:0/1089232520 shutdown_connections 2024-04-25T10:33:45.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.122+0000 7f54ea234640 1 --2- 172.21.15.106:0/1089232520 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f54d00744e0 0x7f54d00769f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:45.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.122+0000 7f54ea234640 1 --2- 172.21.15.106:0/1089232520 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54e4153c50 0x7f54e4158040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:45.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.122+0000 7f54ea234640 1 --2- 172.21.15.106:0/1089232520 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54e40c5450 0x7f54e4142c70 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:45.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.122+0000 7f54ea234640 1 --2- 172.21.15.106:0/1089232520 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f54e4074910 0x7f54e4141580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:45.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.122+0000 7f54ea234640 1 -- 172.21.15.106:0/1089232520 >> 172.21.15.106:0/1089232520 conn(0x7f54e4070b30 msgr2=0x7f54e414ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:45.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.122+0000 7f54ea234640 1 -- 172.21.15.106:0/1089232520 shutdown_connections 2024-04-25T10:33:45.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.122+0000 7f54ea234640 1 -- 172.21.15.106:0/1089232520 wait complete. 2024-04-25T10:33:45.141 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-04-25T10:33:45.142 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-04-25T10:33:45.142 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-04-25T10:33:45.142 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-25T10:33:45.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.304+0000 7fc5b12aa640 1 Processor -- start 2024-04-25T10:33:45.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.304+0000 7fc5b12aa640 1 -- start start 2024-04-25T10:33:45.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.306+0000 7fc5b12aa640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc5ac160440 0x7fc5ac160850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:45.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.306+0000 7fc5b12aa640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5ac15cf80 con 0x7fc5ac0c9820 2024-04-25T10:33:45.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.306+0000 7fc5b12aa640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5ac15d220 con 0x7fc5ac0cb3d0 2024-04-25T10:33:45.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.306+0000 7fc5b12aa640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fc5ac0ca6c0 con 0x7fc5ac160440 2024-04-25T10:33:45.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.306+0000 7fc5ab7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc5ac160440 0x7fc5ac160850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:45.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.306+0000 7fc5aa7fc640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fc5ac0c9820 0x7fc5ac156b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:59790/0 (socket says 172.21.15.106:59790) 2024-04-25T10:33:45.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.306+0000 7fc5aa7fc640 1 -- 172.21.15.106:0/4170958694 learned_addr learned my addr 172.21.15.106:0/4170958694 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:33:45.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.306+0000 7fc5ab7fe640 1 -- 172.21.15.106:0/4170958694 >> v1:172.21.15.134:6789/0 conn(0x7fc5ac0cb3d0 legacy=0x7fc5ac0cf2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:45.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.306+0000 7fc5ab7fe640 1 -- 172.21.15.106:0/4170958694 >> v1:172.21.15.106:6789/0 conn(0x7fc5ac0c9820 legacy=0x7fc5ac156b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:45.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.306+0000 7fc5a9ffb640 1 -- 172.21.15.106:0/4170958694 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2032857503 0 0) 0x7fc5ac15cf80 con 0x7fc5ac0c9820 2024-04-25T10:33:45.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.306+0000 7fc5ab7fe640 1 -- 172.21.15.106:0/4170958694 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5ac15d500 con 0x7fc5ac160440 2024-04-25T10:33:45.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.307+0000 7fc5ab7fe640 1 --2- 172.21.15.106:0/4170958694 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc5ac160440 0x7fc5ac160850 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4009d90 tx=0x7fc5a402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd888be9e914a35e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:45.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.307+0000 7fc5a9ffb640 1 -- 172.21.15.106:0/4170958694 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a4007740 con 0x7fc5ac160440 2024-04-25T10:33:45.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.307+0000 7fc5a9ffb640 1 -- 172.21.15.106:0/4170958694 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5a40078e0 con 0x7fc5ac160440 2024-04-25T10:33:45.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.307+0000 7fc5a9ffb640 1 -- 172.21.15.106:0/4170958694 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a4007cb0 con 0x7fc5ac160440 2024-04-25T10:33:45.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.308+0000 7fc5b12aa640 1 -- 172.21.15.106:0/4170958694 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc5ac160440 msgr2=0x7fc5ac160850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:45.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.308+0000 7fc5b12aa640 1 --2- 172.21.15.106:0/4170958694 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc5ac160440 0x7fc5ac160850 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4009d90 tx=0x7fc5a402a730 comp rx=0 tx=0).stop 2024-04-25T10:33:45.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.309+0000 7fc5b12aa640 1 -- 172.21.15.106:0/4170958694 shutdown_connections 2024-04-25T10:33:45.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.309+0000 7fc5b12aa640 1 --2- 172.21.15.106:0/4170958694 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc5ac160440 0x7fc5ac160850 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:45.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.309+0000 7fc5b12aa640 1 -- 172.21.15.106:0/4170958694 >> 172.21.15.106:0/4170958694 conn(0x7fc5ac06d8b0 msgr2=0x7fc5ac06dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:45.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.309+0000 7fc5b12aa640 1 -- 172.21.15.106:0/4170958694 shutdown_connections 2024-04-25T10:33:45.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.309+0000 7fc5b12aa640 1 -- 172.21.15.106:0/4170958694 wait complete. 2024-04-25T10:33:45.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.311+0000 7fc5b12aa640 1 Processor -- start 2024-04-25T10:33:45.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.311+0000 7fc5b12aa640 1 -- start start 2024-04-25T10:33:45.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.313+0000 7fc5b12aa640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc5ac0c9820 0x7fc5ac14a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:45.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.313+0000 7fc5b12aa640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5ac0cb3d0 0x7fc5ac15f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:45.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.313+0000 7fc5b12aa640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc5ac160440 0x7fc5ac148240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:45.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.313+0000 7fc5b12aa640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ac15db50 con 0x7fc5ac160440 2024-04-25T10:33:45.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.313+0000 7fc5b12aa640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ac0cfa00 con 0x7fc5ac0cb3d0 2024-04-25T10:33:45.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.313+0000 7fc5b12aa640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc5ac160d90 con 0x7fc5ac0c9820 2024-04-25T10:33:45.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.313+0000 7fc5ab7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc5ac160440 0x7fc5ac148240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:45.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.313+0000 7fc5ab7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc5ac160440 0x7fc5ac148240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:39462/0 (socket says 172.21.15.106:39462) 2024-04-25T10:33:45.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.313+0000 7fc5ab7fe640 1 -- 172.21.15.106:0/566013328 learned_addr learned my addr 172.21.15.106:0/566013328 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:33:45.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.313+0000 7fc5aaffd640 1 --2- 172.21.15.106:0/566013328 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc5ac0c9820 0x7fc5ac14a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:45.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.313+0000 7fc5aa7fc640 1 --2- 172.21.15.106:0/566013328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5ac0cb3d0 0x7fc5ac15f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:45.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.313+0000 7fc5ab7fe640 1 -- 172.21.15.106:0/566013328 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc5ac0c9820 msgr2=0x7fc5ac14a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:45.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.313+0000 7fc5ab7fe640 1 --2- 172.21.15.106:0/566013328 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc5ac0c9820 0x7fc5ac14a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:45.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.313+0000 7fc5ab7fe640 1 -- 172.21.15.106:0/566013328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5ac0cb3d0 msgr2=0x7fc5ac15f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:45.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.313+0000 7fc5ab7fe640 1 --2- 172.21.15.106:0/566013328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5ac0cb3d0 0x7fc5ac15f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:45.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.313+0000 7fc5ab7fe640 1 -- 172.21.15.106:0/566013328 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5ac14ac10 con 0x7fc5ac160440 2024-04-25T10:33:45.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.314+0000 7fc5ab7fe640 1 --2- 172.21.15.106:0/566013328 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc5ac160440 0x7fc5ac148240 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4009eb0 tx=0x7fc5a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:45.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.314+0000 7fc5937fe640 1 -- 172.21.15.106:0/566013328 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a4040070 con 0x7fc5ac160440 2024-04-25T10:33:45.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.314+0000 7fc5937fe640 1 -- 172.21.15.106:0/566013328 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5a4031c40 con 0x7fc5ac160440 2024-04-25T10:33:45.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.314+0000 7fc5937fe640 1 -- 172.21.15.106:0/566013328 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a4003990 con 0x7fc5ac160440 2024-04-25T10:33:45.318 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.314+0000 7fc5b12aa640 1 -- 172.21.15.106:0/566013328 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5ac14fca0 con 0x7fc5ac160440 2024-04-25T10:33:45.318 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.314+0000 7fc5b12aa640 1 -- 172.21.15.106:0/566013328 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5ac150260 con 0x7fc5ac160440 2024-04-25T10:33:45.318 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.315+0000 7fc5b12aa640 1 --2- 172.21.15.106:0/566013328 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc5ac149690 0x7fc5ac148780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:45.318 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.316+0000 7fc5aaffd640 1 --2- 172.21.15.106:0/566013328 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc5ac149690 0x7fc5ac148780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:45.319 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.316+0000 7fc5b12aa640 1 -- 172.21.15.106:0/566013328 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ac14a9a0 con 0x7fc5ac149690 2024-04-25T10:33:45.319 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.316+0000 7fc5aaffd640 1 -- 172.21.15.106:0/566013328 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc5ac14fad0 con 0x7fc5ac149690 2024-04-25T10:33:45.320 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.317+0000 7fc5937fe640 1 -- 172.21.15.106:0/566013328 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc5a4039040 con 0x7fc5ac160440 2024-04-25T10:33:45.320 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.317+0000 7fc5aaffd640 1 --2- 172.21.15.106:0/566013328 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc5ac149690 0x7fc5ac148780 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fc59c00e770 tx=0x7fc59c00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:45.320 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.317+0000 7fc5937fe640 1 --2- 172.21.15.106:0/566013328 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fc5980744e0 0x7fc5980769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:45.320 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.317+0000 7fc5937fe640 1 -- 172.21.15.106:0/566013328 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fc5a40b6460 con 0x7fc5ac160440 2024-04-25T10:33:45.320 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.318+0000 7fc5aa7fc640 1 --2- 172.21.15.106:0/566013328 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fc5980744e0 0x7fc5980769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:45.320 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.318+0000 7fc5937fe640 1 -- 172.21.15.106:0/566013328 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59c017070 con 0x7fc5ac149690 2024-04-25T10:33:45.320 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.318+0000 7fc5937fe640 1 -- 172.21.15.106:0/566013328 <== mon.0 v2:172.21.15.106:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fc59c00c7a0 con 0x7fc5ac149690 2024-04-25T10:33:45.320 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.318+0000 7fc5937fe640 1 -- 172.21.15.106:0/566013328 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc5ac149690 msgr2=0x7fc5ac148780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:45.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.318+0000 7fc5937fe640 1 --2- 172.21.15.106:0/566013328 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc5ac149690 0x7fc5ac148780 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fc59c00e770 tx=0x7fc59c00ec30 comp rx=0 tx=0).stop 2024-04-25T10:33:45.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.318+0000 7fc5937fe640 1 -- 172.21.15.106:0/566013328 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc5ac149690 msgr2=0x7fc5ac148780 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T10:33:45.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.318+0000 7fc5937fe640 1 --2- 172.21.15.106:0/566013328 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc5ac149690 0x7fc5ac148780 secure :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fc59c00e770 tx=0x7fc59c00ec30 comp rx=0 tx=0).stop 2024-04-25T10:33:45.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.318+0000 7fc5aa7fc640 1 --2- 172.21.15.106:0/566013328 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fc5980744e0 0x7fc5980769f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac15ceb0 tx=0x7fc5a0007e40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:45.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.330+0000 7fc5b12aa640 1 --2- 172.21.15.106:0/566013328 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc5ac15df10 0x7fc5ac0ccfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:45.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.330+0000 7fc5b12aa640 1 -- 172.21.15.106:0/566013328 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ac15f3d0 con 0x7fc5ac15df10 2024-04-25T10:33:45.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.330+0000 7fc5aaffd640 1 --2- 172.21.15.106:0/566013328 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc5ac15df10 0x7fc5ac0ccfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:45.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.331+0000 7fc5aaffd640 1 -- 172.21.15.106:0/566013328 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fc5ac070e30 con 0x7fc5ac15df10 2024-04-25T10:33:45.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.331+0000 7fc5aaffd640 1 --2- 172.21.15.106:0/566013328 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc5ac15df10 0x7fc5ac0ccfb0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fc59c00e770 tx=0x7fc59c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:45.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.331+0000 7fc5937fe640 1 -- 172.21.15.106:0/566013328 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59c017070 con 0x7fc5ac15df10 2024-04-25T10:33:45.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.331+0000 7fc5937fe640 1 -- 172.21.15.106:0/566013328 <== mon.0 v2:172.21.15.106:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5072 (secure 0 0 0) 0x7fc59c006250 con 0x7fc5ac15df10 2024-04-25T10:33:45.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.332+0000 7fc5937fe640 1 -- 172.21.15.106:0/566013328 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc5ac15df10 msgr2=0x7fc5ac0ccfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:45.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.332+0000 7fc5937fe640 1 --2- 172.21.15.106:0/566013328 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc5ac15df10 0x7fc5ac0ccfb0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fc59c00e770 tx=0x7fc59c0046b0 comp rx=0 tx=0).stop 2024-04-25T10:33:45.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.332+0000 7fc5937fe640 1 -- 172.21.15.106:0/566013328 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc5ac15df10 msgr2=0x7fc5ac0ccfb0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T10:33:45.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.332+0000 7fc5937fe640 1 --2- 172.21.15.106:0/566013328 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc5ac15df10 0x7fc5ac0ccfb0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:45.335 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-25T10:33:45.335 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "a", 2024-04-25T10:33:45.335 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 0, 2024-04-25T10:33:45.335 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "leader", 2024-04-25T10:33:45.335 INFO:teuthology.orchestra.run.smithi106.stdout: "election_epoch": 22, 2024-04-25T10:33:45.335 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 217570, 2024-04-25T10:33:45.335 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum": [ 2024-04-25T10:33:45.335 INFO:teuthology.orchestra.run.smithi106.stdout: 0, 2024-04-25T10:33:45.335 INFO:teuthology.orchestra.run.smithi106.stdout: 1, 2024-04-25T10:33:45.335 INFO:teuthology.orchestra.run.smithi106.stdout: 2 2024-04-25T10:33:45.335 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:33:45.335 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_age": 11, 2024-04-25T10:33:45.336 INFO:teuthology.orchestra.run.smithi106.stdout: "features": { 2024-04-25T10:33:45.336 INFO:teuthology.orchestra.run.smithi106.stdout: "required_con": "2450521708007866372", 2024-04-25T10:33:45.336 INFO:teuthology.orchestra.run.smithi106.stdout: "required_mon": [ 2024-04-25T10:33:45.336 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:33:45.336 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:33:45.336 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:33:45.336 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:33:45.336 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:33:45.336 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:33:45.336 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:33:45.336 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:33:45.336 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:33:45.336 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:33:45.336 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:33:45.336 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:33:45.336 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_con": "4540701547738038271", 2024-04-25T10:33:45.337 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_mon": [ 2024-04-25T10:33:45.337 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:33:45.337 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:33:45.337 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:33:45.337 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:33:45.337 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:33:45.337 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:33:45.337 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:33:45.337 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:33:45.337 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:33:45.337 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:33:45.337 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:33:45.337 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:33:45.338 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:45.338 INFO:teuthology.orchestra.run.smithi106.stdout: "outside_quorum": [], 2024-04-25T10:33:45.338 INFO:teuthology.orchestra.run.smithi106.stdout: "extra_probe_peers": [], 2024-04-25T10:33:45.338 INFO:teuthology.orchestra.run.smithi106.stdout: "sync_provider": [], 2024-04-25T10:33:45.338 INFO:teuthology.orchestra.run.smithi106.stdout: "monmap": { 2024-04-25T10:33:45.338 INFO:teuthology.orchestra.run.smithi106.stdout: "epoch": 1, 2024-04-25T10:33:45.338 INFO:teuthology.orchestra.run.smithi106.stdout: "fsid": "3ee77985-e8e4-41cf-81e9-7c774cf7891f", 2024-04-25T10:33:45.338 INFO:teuthology.orchestra.run.smithi106.stdout: "modified": "2024-04-25T10:29:33.446228Z", 2024-04-25T10:33:45.338 INFO:teuthology.orchestra.run.smithi106.stdout: "created": "2024-04-25T10:29:33.446228Z", 2024-04-25T10:33:45.338 INFO:teuthology.orchestra.run.smithi106.stdout: "min_mon_release": 19, 2024-04-25T10:33:45.338 INFO:teuthology.orchestra.run.smithi106.stdout: "min_mon_release_name": "squid", 2024-04-25T10:33:45.339 INFO:teuthology.orchestra.run.smithi106.stdout: "election_strategy": 1, 2024-04-25T10:33:45.339 INFO:teuthology.orchestra.run.smithi106.stdout: "disallowed_leaders: ": "", 2024-04-25T10:33:45.339 INFO:teuthology.orchestra.run.smithi106.stdout: "stretch_mode": false, 2024-04-25T10:33:45.339 INFO:teuthology.orchestra.run.smithi106.stdout: "tiebreaker_mon": "", 2024-04-25T10:33:45.339 INFO:teuthology.orchestra.run.smithi106.stdout: "removed_ranks: ": "", 2024-04-25T10:33:45.339 INFO:teuthology.orchestra.run.smithi106.stdout: "features": { 2024-04-25T10:33:45.339 INFO:teuthology.orchestra.run.smithi106.stdout: "persistent": [ 2024-04-25T10:33:45.339 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:33:45.339 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:33:45.339 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:33:45.339 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:33:45.339 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:33:45.339 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:33:45.339 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:33:45.339 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:33:45.339 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:33:45.339 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:33:45.340 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:33:45.340 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:33:45.340 INFO:teuthology.orchestra.run.smithi106.stdout: "optional": [] 2024-04-25T10:33:45.340 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:45.340 INFO:teuthology.orchestra.run.smithi106.stdout: "mons": [ 2024-04-25T10:33:45.340 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:45.340 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 0, 2024-04-25T10:33:45.340 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "a", 2024-04-25T10:33:45.340 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:33:45.340 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:33:45.340 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:45.340 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:33:45.340 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:3300", 2024-04-25T10:33:45.340 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:33:45.340 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:45.340 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:45.341 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:33:45.341 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:6789", 2024-04-25T10:33:45.341 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:33:45.341 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:33:45.341 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:33:45.341 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:45.341 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:6789/0", 2024-04-25T10:33:45.341 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.106:6789/0", 2024-04-25T10:33:45.341 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:33:45.341 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:33:45.341 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:33:45.341 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:45.341 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:45.341 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 1, 2024-04-25T10:33:45.341 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "b", 2024-04-25T10:33:45.342 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:33:45.342 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:33:45.342 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:45.342 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:33:45.342 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:3300", 2024-04-25T10:33:45.342 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:33:45.342 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:45.342 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:45.342 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:33:45.342 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6789", 2024-04-25T10:33:45.342 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:33:45.342 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:33:45.342 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:33:45.342 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:45.343 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6789/0", 2024-04-25T10:33:45.343 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.134:6789/0", 2024-04-25T10:33:45.343 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:33:45.343 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:33:45.343 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:33:45.343 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:45.343 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:45.343 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 2, 2024-04-25T10:33:45.343 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "c", 2024-04-25T10:33:45.343 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:33:45.343 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:33:45.343 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:45.343 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:33:45.343 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:3301", 2024-04-25T10:33:45.343 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:33:45.344 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:45.344 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:45.344 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:33:45.344 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6790", 2024-04-25T10:33:45.344 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:33:45.344 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:33:45.344 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:33:45.344 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:45.344 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6790/0", 2024-04-25T10:33:45.344 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.134:6790/0", 2024-04-25T10:33:45.344 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:33:45.344 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:33:45.344 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:33:45.344 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:33:45.344 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:33:45.345 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:45.345 INFO:teuthology.orchestra.run.smithi106.stdout: "feature_map": { 2024-04-25T10:33:45.345 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": [ 2024-04-25T10:33:45.345 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:45.345 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:33:45.345 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:33:45.345 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 1 2024-04-25T10:33:45.345 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:33:45.345 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:33:45.345 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": [ 2024-04-25T10:33:45.345 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:45.345 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:33:45.345 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:33:45.345 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 6 2024-04-25T10:33:45.345 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:33:45.346 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:33:45.346 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": [ 2024-04-25T10:33:45.346 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:45.346 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:33:45.346 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:33:45.346 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 6 2024-04-25T10:33:45.346 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:33:45.346 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:33:45.346 INFO:teuthology.orchestra.run.smithi106.stdout: "client": [ 2024-04-25T10:33:45.346 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:45.346 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:33:45.346 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:33:45.346 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 7 2024-04-25T10:33:45.346 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:33:45.347 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:33:45.347 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": [ 2024-04-25T10:33:45.347 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:45.347 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:33:45.347 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:33:45.347 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 2 2024-04-25T10:33:45.347 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:33:45.347 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:33:45.347 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:45.347 INFO:teuthology.orchestra.run.smithi106.stdout: "stretch_mode": false 2024-04-25T10:33:45.347 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-25T10:33:45.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.332+0000 7fc5b12aa640 1 -- 172.21.15.106:0/566013328 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fc5980744e0 msgr2=0x7fc5980769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:45.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.332+0000 7fc5b12aa640 1 --2- 172.21.15.106:0/566013328 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fc5980744e0 0x7fc5980769f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac15ceb0 tx=0x7fc5a0007e40 comp rx=0 tx=0).stop 2024-04-25T10:33:45.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.332+0000 7fc5b12aa640 1 -- 172.21.15.106:0/566013328 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc5ac160440 msgr2=0x7fc5ac148240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:45.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.332+0000 7fc5ab7fe640 1 -- 172.21.15.106:0/566013328 reap_dead start 2024-04-25T10:33:45.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.332+0000 7fc5b12aa640 1 --2- 172.21.15.106:0/566013328 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc5ac160440 0x7fc5ac148240 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4009eb0 tx=0x7fc5a4004a40 comp rx=0 tx=0).stop 2024-04-25T10:33:45.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.332+0000 7fc5b12aa640 1 -- 172.21.15.106:0/566013328 shutdown_connections 2024-04-25T10:33:45.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.332+0000 7fc5b12aa640 1 --2- 172.21.15.106:0/566013328 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc5ac160440 0x7fc5ac148240 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:45.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.332+0000 7fc5b12aa640 1 -- 172.21.15.106:0/566013328 >> 172.21.15.106:0/566013328 conn(0x7fc5ac06d8b0 msgr2=0x7fc5ac06dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:45.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.332+0000 7fc5b12aa640 1 -- 172.21.15.106:0/566013328 shutdown_connections 2024-04-25T10:33:45.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.332+0000 7fc5b12aa640 1 -- 172.21.15.106:0/566013328 wait complete. 2024-04-25T10:33:45.353 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T10:33:45.353 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-04-25T10:33:45.354 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-25T10:33:45.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.505+0000 7f157f900640 1 Processor -- start 2024-04-25T10:33:45.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.506+0000 7f157f900640 1 -- start start 2024-04-25T10:33:45.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.508+0000 7f157f900640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15780ca0e0 0x7f15780728b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:45.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.508+0000 7f157f900640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1578072df0 0x7f157815e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:45.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.508+0000 7f157f900640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15780cad70 con 0x7f15780ca860 2024-04-25T10:33:45.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.508+0000 7f157f900640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f15781550d0 con 0x7f1578072df0 2024-04-25T10:33:45.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.508+0000 7f157f900640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f15780cc8e0 con 0x7f15780ca0e0 2024-04-25T10:33:45.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.508+0000 7f157d69c640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f15780ca860 0x7f15780cbee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:59792/0 (socket says 172.21.15.106:59792) 2024-04-25T10:33:45.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.508+0000 7f157d69c640 1 -- 172.21.15.106:0/894898943 learned_addr learned my addr 172.21.15.106:0/894898943 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:33:45.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.508+0000 7f157ce9b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15780ca0e0 0x7f15780728b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:45.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.508+0000 7f157de9d640 1 --2- 172.21.15.106:0/894898943 >> v2:172.21.15.106:3300/0 conn(0x7f1578072df0 0x7f157815e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:45.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.509+0000 7f156ffff640 1 -- 172.21.15.106:0/894898943 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1281037932 0 0) 0x7f15780cad70 con 0x7f15780ca860 2024-04-25T10:33:45.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.509+0000 7f157ce9b640 1 -- 172.21.15.106:0/894898943 >> v2:172.21.15.106:3300/0 conn(0x7f1578072df0 msgr2=0x7f157815e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:45.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.509+0000 7f157ce9b640 1 --2- 172.21.15.106:0/894898943 >> v2:172.21.15.106:3300/0 conn(0x7f1578072df0 0x7f157815e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:45.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.509+0000 7f157ce9b640 1 -- 172.21.15.106:0/894898943 >> v1:172.21.15.106:6789/0 conn(0x7f15780ca860 legacy=0x7f15780cbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:45.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.509+0000 7f157ce9b640 1 -- 172.21.15.106:0/894898943 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15780cb010 con 0x7f15780ca0e0 2024-04-25T10:33:45.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.509+0000 7f157de9d640 1 --2- 172.21.15.106:0/894898943 >> v2:172.21.15.106:3300/0 conn(0x7f1578072df0 0x7f157815e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:33:45.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.510+0000 7f157ce9b640 1 --2- 172.21.15.106:0/894898943 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15780ca0e0 0x7f15780728b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f15680097a0 tx=0x7f156802a230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=897aad2d04166e92 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:45.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.510+0000 7f156ffff640 1 -- 172.21.15.106:0/894898943 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1568037070 con 0x7f15780ca0e0 2024-04-25T10:33:45.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.510+0000 7f156ffff640 1 -- 172.21.15.106:0/894898943 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1568032440 con 0x7f15780ca0e0 2024-04-25T10:33:45.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.510+0000 7f156ffff640 1 -- 172.21.15.106:0/894898943 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1568032810 con 0x7f15780ca0e0 2024-04-25T10:33:45.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.511+0000 7f157f900640 1 -- 172.21.15.106:0/894898943 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15780ca0e0 msgr2=0x7f15780728b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:45.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.511+0000 7f157f900640 1 --2- 172.21.15.106:0/894898943 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15780ca0e0 0x7f15780728b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f15680097a0 tx=0x7f156802a230 comp rx=0 tx=0).stop 2024-04-25T10:33:45.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.511+0000 7f157f900640 1 -- 172.21.15.106:0/894898943 shutdown_connections 2024-04-25T10:33:45.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.511+0000 7f157f900640 1 --2- 172.21.15.106:0/894898943 >> v2:172.21.15.106:3300/0 conn(0x7f1578072df0 0x7f157815e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:45.514 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.511+0000 7f157f900640 1 --2- 172.21.15.106:0/894898943 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15780ca0e0 0x7f15780728b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:45.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.511+0000 7f157f900640 1 -- 172.21.15.106:0/894898943 >> 172.21.15.106:0/894898943 conn(0x7f157806c5c0 msgr2=0x7f157806c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:45.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.511+0000 7f157f900640 1 -- 172.21.15.106:0/894898943 shutdown_connections 2024-04-25T10:33:45.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.511+0000 7f157f900640 1 -- 172.21.15.106:0/894898943 wait complete. 2024-04-25T10:33:45.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.514+0000 7f157f900640 1 Processor -- start 2024-04-25T10:33:45.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.514+0000 7f157f900640 1 -- start start 2024-04-25T10:33:45.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.515+0000 7f157f900640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1578072df0 0x7f1578158170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:45.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.515+0000 7f157f900640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15780ca860 0x7f1578159c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:45.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.515+0000 7f157f900640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1578150300 0x7f157814e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:45.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.515+0000 7f157f900640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f157815b4c0 con 0x7f1578072df0 2024-04-25T10:33:45.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.516+0000 7f157f900640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f157815b1e0 con 0x7f15780ca860 2024-04-25T10:33:45.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.516+0000 7f157f900640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f157815ab80 con 0x7f1578150300 2024-04-25T10:33:45.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.516+0000 7f157d69c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1578072df0 0x7f1578158170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:45.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.516+0000 7f157d69c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1578072df0 0x7f1578158170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:39494/0 (socket says 172.21.15.106:39494) 2024-04-25T10:33:45.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.516+0000 7f157d69c640 1 -- 172.21.15.106:0/3535400257 learned_addr learned my addr 172.21.15.106:0/3535400257 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:33:45.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.516+0000 7f157ce9b640 1 --2- 172.21.15.106:0/3535400257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15780ca860 0x7f1578159c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:45.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.516+0000 7f157de9d640 1 --2- 172.21.15.106:0/3535400257 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1578150300 0x7f157814e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:45.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.516+0000 7f157d69c640 1 -- 172.21.15.106:0/3535400257 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1578150300 msgr2=0x7f157814e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:45.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.516+0000 7f157d69c640 1 --2- 172.21.15.106:0/3535400257 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1578150300 0x7f157814e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:45.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.516+0000 7f157d69c640 1 -- 172.21.15.106:0/3535400257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15780ca860 msgr2=0x7f1578159c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:45.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.516+0000 7f157d69c640 1 --2- 172.21.15.106:0/3535400257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15780ca860 0x7f1578159c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:45.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.516+0000 7f157d69c640 1 -- 172.21.15.106:0/3535400257 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15781f0e70 con 0x7f1578072df0 2024-04-25T10:33:45.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.516+0000 7f157ce9b640 1 --2- 172.21.15.106:0/3535400257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15780ca860 0x7f1578159c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:33:45.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.516+0000 7f157de9d640 1 --2- 172.21.15.106:0/3535400257 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1578150300 0x7f157814e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:33:45.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.516+0000 7f157d69c640 1 --2- 172.21.15.106:0/3535400257 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1578072df0 0x7f1578158170 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f157400c430 tx=0x7f157400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:45.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.517+0000 7f156dffb640 1 -- 172.21.15.106:0/3535400257 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1574017070 con 0x7f1578072df0 2024-04-25T10:33:45.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.517+0000 7f156dffb640 1 -- 172.21.15.106:0/3535400257 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1574005ce0 con 0x7f1578072df0 2024-04-25T10:33:45.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.517+0000 7f156dffb640 1 -- 172.21.15.106:0/3535400257 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1574007310 con 0x7f1578072df0 2024-04-25T10:33:45.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.517+0000 7f157f900640 1 -- 172.21.15.106:0/3535400257 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15781f1110 con 0x7f1578072df0 2024-04-25T10:33:45.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.517+0000 7f157f900640 1 -- 172.21.15.106:0/3535400257 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15781f5160 con 0x7f1578072df0 2024-04-25T10:33:45.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.518+0000 7f157f900640 1 --2- 172.21.15.106:0/3535400257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1578150710 0x7f157806f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:45.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.519+0000 7f157f900640 1 -- 172.21.15.106:0/3535400257 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f15781f4f10 con 0x7f1578150710 2024-04-25T10:33:45.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.519+0000 7f157ce9b640 1 --2- 172.21.15.106:0/3535400257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1578150710 0x7f157806f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:45.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.519+0000 7f156dffb640 1 -- 172.21.15.106:0/3535400257 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f157401a070 con 0x7f1578072df0 2024-04-25T10:33:45.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.519+0000 7f157ce9b640 1 -- 172.21.15.106:0/3535400257 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f157806a3d0 con 0x7f1578150710 2024-04-25T10:33:45.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.520+0000 7f156dffb640 1 --2- 172.21.15.106:0/3535400257 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f155c0788f0 0x7f155c07ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:45.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.520+0000 7f156dffb640 1 -- 172.21.15.106:0/3535400257 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f1574094650 con 0x7f1578072df0 2024-04-25T10:33:45.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.520+0000 7f157de9d640 1 --2- 172.21.15.106:0/3535400257 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f155c0788f0 0x7f155c07ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:45.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.520+0000 7f157ce9b640 1 --2- 172.21.15.106:0/3535400257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1578150710 0x7f157806f5d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f15780cd050 tx=0x7f15680046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:45.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.520+0000 7f156dffb640 1 -- 172.21.15.106:0/3535400257 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1568041070 con 0x7f1578150710 2024-04-25T10:33:45.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.521+0000 7f156dffb640 1 -- 172.21.15.106:0/3535400257 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f156803c660 con 0x7f1578150710 2024-04-25T10:33:45.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.521+0000 7f156dffb640 1 -- 172.21.15.106:0/3535400257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1578150710 msgr2=0x7f157806f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:45.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.521+0000 7f156dffb640 1 --2- 172.21.15.106:0/3535400257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1578150710 0x7f157806f5d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f15780cd050 tx=0x7f15680046c0 comp rx=0 tx=0).stop 2024-04-25T10:33:45.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.521+0000 7f156dffb640 1 -- 172.21.15.106:0/3535400257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1578150710 msgr2=0x7f157806f5d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T10:33:45.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.521+0000 7f156dffb640 1 --2- 172.21.15.106:0/3535400257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1578150710 0x7f157806f5d0 secure :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f15780cd050 tx=0x7f15680046c0 comp rx=0 tx=0).stop 2024-04-25T10:33:45.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.521+0000 7f157de9d640 1 --2- 172.21.15.106:0/3535400257 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f155c0788f0 0x7f155c07ae00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f15781586b0 tx=0x7f1570003a30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:45.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.530+0000 7f157f900640 1 --2- 172.21.15.106:0/3535400257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1578153d50 0x7f157814e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:45.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.530+0000 7f157f900640 1 -- 172.21.15.106:0/3535400257 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f15780b4370 con 0x7f1578153d50 2024-04-25T10:33:45.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.531+0000 7f157ce9b640 1 --2- 172.21.15.106:0/3535400257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1578153d50 0x7f157814e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:45.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.531+0000 7f157ce9b640 1 -- 172.21.15.106:0/3535400257 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f15780c92c0 con 0x7f1578153d50 2024-04-25T10:33:45.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.531+0000 7f157ce9b640 1 --2- 172.21.15.106:0/3535400257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1578153d50 0x7f157814e960 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f15680097a0 tx=0x7f1568008030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:45.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.532+0000 7f156dffb640 1 -- 172.21.15.106:0/3535400257 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1568037040 con 0x7f1578153d50 2024-04-25T10:33:45.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.532+0000 7f156dffb640 1 -- 172.21.15.106:0/3535400257 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4559 (secure 0 0 0) 0x7f156803ce60 con 0x7f1578153d50 2024-04-25T10:33:45.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.532+0000 7f156dffb640 1 -- 172.21.15.106:0/3535400257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1578153d50 msgr2=0x7f157814e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:45.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.532+0000 7f156dffb640 1 --2- 172.21.15.106:0/3535400257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1578153d50 0x7f157814e960 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f15680097a0 tx=0x7f1568008030 comp rx=0 tx=0).stop 2024-04-25T10:33:45.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.532+0000 7f156dffb640 1 -- 172.21.15.106:0/3535400257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1578153d50 msgr2=0x7f157814e960 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T10:33:45.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.532+0000 7f156dffb640 1 --2- 172.21.15.106:0/3535400257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1578153d50 0x7f157814e960 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:45.535 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-25T10:33:45.535 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "b", 2024-04-25T10:33:45.535 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 1, 2024-04-25T10:33:45.535 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "peon", 2024-04-25T10:33:45.535 INFO:teuthology.orchestra.run.smithi106.stdout: "election_epoch": 22, 2024-04-25T10:33:45.535 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 12056, 2024-04-25T10:33:45.536 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum": [ 2024-04-25T10:33:45.536 INFO:teuthology.orchestra.run.smithi106.stdout: 0, 2024-04-25T10:33:45.536 INFO:teuthology.orchestra.run.smithi106.stdout: 1, 2024-04-25T10:33:45.536 INFO:teuthology.orchestra.run.smithi106.stdout: 2 2024-04-25T10:33:45.536 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:33:45.536 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_age": 11, 2024-04-25T10:33:45.536 INFO:teuthology.orchestra.run.smithi106.stdout: "features": { 2024-04-25T10:33:45.536 INFO:teuthology.orchestra.run.smithi106.stdout: "required_con": "2450521708007866372", 2024-04-25T10:33:45.536 INFO:teuthology.orchestra.run.smithi106.stdout: "required_mon": [ 2024-04-25T10:33:45.536 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:33:45.536 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:33:45.536 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:33:45.536 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:33:45.536 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:33:45.536 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:33:45.537 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:33:45.537 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:33:45.537 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:33:45.537 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:33:45.537 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:33:45.537 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:33:45.537 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_con": "4540701547738038271", 2024-04-25T10:33:45.537 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_mon": [ 2024-04-25T10:33:45.537 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:33:45.537 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:33:45.537 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:33:45.537 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:33:45.537 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:33:45.537 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:33:45.537 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:33:45.538 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:33:45.538 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:33:45.538 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:33:45.538 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:33:45.538 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:33:45.538 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:45.538 INFO:teuthology.orchestra.run.smithi106.stdout: "outside_quorum": [], 2024-04-25T10:33:45.538 INFO:teuthology.orchestra.run.smithi106.stdout: "extra_probe_peers": [], 2024-04-25T10:33:45.538 INFO:teuthology.orchestra.run.smithi106.stdout: "sync_provider": [], 2024-04-25T10:33:45.538 INFO:teuthology.orchestra.run.smithi106.stdout: "monmap": { 2024-04-25T10:33:45.538 INFO:teuthology.orchestra.run.smithi106.stdout: "epoch": 1, 2024-04-25T10:33:45.538 INFO:teuthology.orchestra.run.smithi106.stdout: "fsid": "3ee77985-e8e4-41cf-81e9-7c774cf7891f", 2024-04-25T10:33:45.539 INFO:teuthology.orchestra.run.smithi106.stdout: "modified": "2024-04-25T10:29:33.446228Z", 2024-04-25T10:33:45.539 INFO:teuthology.orchestra.run.smithi106.stdout: "created": "2024-04-25T10:29:33.446228Z", 2024-04-25T10:33:45.539 INFO:teuthology.orchestra.run.smithi106.stdout: "min_mon_release": 19, 2024-04-25T10:33:45.539 INFO:teuthology.orchestra.run.smithi106.stdout: "min_mon_release_name": "squid", 2024-04-25T10:33:45.539 INFO:teuthology.orchestra.run.smithi106.stdout: "election_strategy": 1, 2024-04-25T10:33:45.539 INFO:teuthology.orchestra.run.smithi106.stdout: "disallowed_leaders: ": "", 2024-04-25T10:33:45.539 INFO:teuthology.orchestra.run.smithi106.stdout: "stretch_mode": false, 2024-04-25T10:33:45.539 INFO:teuthology.orchestra.run.smithi106.stdout: "tiebreaker_mon": "", 2024-04-25T10:33:45.539 INFO:teuthology.orchestra.run.smithi106.stdout: "removed_ranks: ": "", 2024-04-25T10:33:45.539 INFO:teuthology.orchestra.run.smithi106.stdout: "features": { 2024-04-25T10:33:45.539 INFO:teuthology.orchestra.run.smithi106.stdout: "persistent": [ 2024-04-25T10:33:45.539 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:33:45.539 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:33:45.540 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:33:45.540 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:33:45.540 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:33:45.540 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:33:45.540 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:33:45.540 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:33:45.540 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:33:45.540 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:33:45.540 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:33:45.540 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:33:45.540 INFO:teuthology.orchestra.run.smithi106.stdout: "optional": [] 2024-04-25T10:33:45.540 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:45.540 INFO:teuthology.orchestra.run.smithi106.stdout: "mons": [ 2024-04-25T10:33:45.540 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:45.541 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 0, 2024-04-25T10:33:45.541 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "a", 2024-04-25T10:33:45.541 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:33:45.541 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:33:45.541 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:45.541 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:33:45.541 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:3300", 2024-04-25T10:33:45.541 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:33:45.541 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:45.541 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:45.541 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:33:45.541 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:6789", 2024-04-25T10:33:45.541 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:33:45.541 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:33:45.542 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:33:45.542 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:45.542 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:6789/0", 2024-04-25T10:33:45.542 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.106:6789/0", 2024-04-25T10:33:45.542 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:33:45.542 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:33:45.542 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:33:45.542 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:45.542 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:45.542 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 1, 2024-04-25T10:33:45.542 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "b", 2024-04-25T10:33:45.542 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:33:45.543 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:33:45.543 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:45.543 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:33:45.543 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:3300", 2024-04-25T10:33:45.543 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:33:45.543 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:45.543 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:45.543 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:33:45.543 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6789", 2024-04-25T10:33:45.543 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:33:45.543 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:33:45.543 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:33:45.543 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:45.543 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6789/0", 2024-04-25T10:33:45.544 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.134:6789/0", 2024-04-25T10:33:45.544 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:33:45.544 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:33:45.544 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:33:45.544 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:45.544 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:45.544 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 2, 2024-04-25T10:33:45.544 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "c", 2024-04-25T10:33:45.544 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:33:45.544 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:33:45.544 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:45.544 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:33:45.544 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:3301", 2024-04-25T10:33:45.544 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:33:45.544 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:45.544 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:45.545 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:33:45.545 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6790", 2024-04-25T10:33:45.545 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:33:45.545 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:33:45.545 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:33:45.545 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:45.545 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6790/0", 2024-04-25T10:33:45.545 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.134:6790/0", 2024-04-25T10:33:45.545 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:33:45.545 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:33:45.545 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:33:45.545 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:33:45.545 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:33:45.545 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:45.545 INFO:teuthology.orchestra.run.smithi106.stdout: "feature_map": { 2024-04-25T10:33:45.545 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": [ 2024-04-25T10:33:45.546 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:45.546 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:33:45.546 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:33:45.546 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 1 2024-04-25T10:33:45.546 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:33:45.546 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:33:45.546 INFO:teuthology.orchestra.run.smithi106.stdout: "client": [ 2024-04-25T10:33:45.546 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:45.546 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:33:45.546 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:33:45.546 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 1 2024-04-25T10:33:45.546 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:33:45.546 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:33:45.546 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:45.546 INFO:teuthology.orchestra.run.smithi106.stdout: "stretch_mode": false 2024-04-25T10:33:45.547 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-25T10:33:45.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.533+0000 7f157f900640 1 -- 172.21.15.106:0/3535400257 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f155c0788f0 msgr2=0x7f155c07ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:45.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.533+0000 7f157f900640 1 --2- 172.21.15.106:0/3535400257 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f155c0788f0 0x7f155c07ae00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f15781586b0 tx=0x7f1570003a30 comp rx=0 tx=0).stop 2024-04-25T10:33:45.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.533+0000 7f157f900640 1 -- 172.21.15.106:0/3535400257 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1578072df0 msgr2=0x7f1578158170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:45.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.533+0000 7f157f900640 1 --2- 172.21.15.106:0/3535400257 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1578072df0 0x7f1578158170 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f157400c430 tx=0x7f157400c8f0 comp rx=0 tx=0).stop 2024-04-25T10:33:45.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.533+0000 7f157de9d640 1 -- 172.21.15.106:0/3535400257 reap_dead start 2024-04-25T10:33:45.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.533+0000 7f157de9d640 1 -- 172.21.15.106:0/3535400257 reap_dead start 2024-04-25T10:33:45.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.533+0000 7f157f900640 1 -- 172.21.15.106:0/3535400257 shutdown_connections 2024-04-25T10:33:45.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.533+0000 7f157f900640 1 -- 172.21.15.106:0/3535400257 >> 172.21.15.106:0/3535400257 conn(0x7f157806c5c0 msgr2=0x7f1578075c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:45.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.533+0000 7f157f900640 1 -- 172.21.15.106:0/3535400257 shutdown_connections 2024-04-25T10:33:45.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.533+0000 7f157f900640 1 -- 172.21.15.106:0/3535400257 wait complete. 2024-04-25T10:33:45.551 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T10:33:45.551 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-04-25T10:33:45.552 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-25T10:33:45.710 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.708+0000 7faa77ce2640 1 Processor -- start 2024-04-25T10:33:45.710 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.708+0000 7faa77ce2640 1 -- start start 2024-04-25T10:33:45.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.710+0000 7faa77ce2640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7faa700741a0 0x7faa700745b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:45.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.710+0000 7faa77ce2640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa700cafc0 con 0x7faa70153bc0 2024-04-25T10:33:45.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.710+0000 7faa77ce2640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa700c44b0 con 0x7faa7006fe80 2024-04-25T10:33:45.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.710+0000 7faa77ce2640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7faa700c36e0 con 0x7faa700741a0 2024-04-25T10:33:45.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.710+0000 7faa7527d640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7faa70153bc0 0x7faa700ca8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:59798/0 (socket says 172.21.15.106:59798) 2024-04-25T10:33:45.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.710+0000 7faa7527d640 1 -- 172.21.15.106:0/6705671 learned_addr learned my addr 172.21.15.106:0/6705671 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:33:45.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.710+0000 7faa7627f640 1 --2- 172.21.15.106:0/6705671 >> v2:172.21.15.134:3300/0 conn(0x7faa700741a0 0x7faa700745b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:45.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.710+0000 7faa74a7c640 1 -- 172.21.15.106:0/6705671 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4110397060 0 0) 0x7faa700cafc0 con 0x7faa70153bc0 2024-04-25T10:33:45.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.710+0000 7faa74a7c640 1 -- 172.21.15.106:0/6705671 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faa5c003520 con 0x7faa70153bc0 2024-04-25T10:33:45.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.711+0000 7faa74a7c640 1 -- 172.21.15.106:0/6705671 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 430053504 0 0) 0x7faa700c44b0 con 0x7faa7006fe80 2024-04-25T10:33:45.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.711+0000 7faa74a7c640 1 -- 172.21.15.106:0/6705671 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faa700cafc0 con 0x7faa7006fe80 2024-04-25T10:33:45.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.711+0000 7faa74a7c640 1 -- 172.21.15.106:0/6705671 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1624249044 0 0) 0x7faa5c003520 con 0x7faa70153bc0 2024-04-25T10:33:45.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.711+0000 7faa74a7c640 1 -- 172.21.15.106:0/6705671 >> v2:172.21.15.134:3300/0 conn(0x7faa700741a0 msgr2=0x7faa700745b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:45.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.711+0000 7faa74a7c640 1 --2- 172.21.15.106:0/6705671 >> v2:172.21.15.134:3300/0 conn(0x7faa700741a0 0x7faa700745b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:45.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.711+0000 7faa74a7c640 1 -- 172.21.15.106:0/6705671 >> v1:172.21.15.134:6789/0 conn(0x7faa7006fe80 legacy=0x7faa700b42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:45.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.711+0000 7faa74a7c640 1 -- 172.21.15.106:0/6705671 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa700c3930 con 0x7faa70153bc0 2024-04-25T10:33:45.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.711+0000 7faa74a7c640 1 -- 172.21.15.106:0/6705671 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3776459350 0 0) 0x7faa60001670 con 0x7faa70153bc0 2024-04-25T10:33:45.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.711+0000 7faa7627f640 1 --2- 172.21.15.106:0/6705671 >> v2:172.21.15.134:3300/0 conn(0x7faa700741a0 0x7faa700745b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:33:45.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.711+0000 7faa74a7c640 1 -- 172.21.15.106:0/6705671 >> v1:172.21.15.106:6789/0 conn(0x7faa70153bc0 legacy=0x7faa700ca8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:45.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.711+0000 7faa74a7c640 1 --2- 172.21.15.106:0/6705671 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa5c003b40 0x7faa5c003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:45.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.711+0000 7faa74a7c640 1 --2- 172.21.15.106:0/6705671 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa5c004800 0x7faa5c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:45.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.711+0000 7faa74a7c640 1 --2- 172.21.15.106:0/6705671 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faa5c0051c0 0x7faa5c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:45.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.711+0000 7faa74a7c640 1 -- 172.21.15.106:0/6705671 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7faa700c36e0 con 0x7faa5c0051c0 2024-04-25T10:33:45.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.711+0000 7faa74a7c640 1 -- 172.21.15.106:0/6705671 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7faa5c00c0c0 con 0x7faa5c003b40 2024-04-25T10:33:45.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.711+0000 7faa74a7c640 1 -- 172.21.15.106:0/6705671 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7faa5c00c2c0 con 0x7faa5c004800 2024-04-25T10:33:45.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.711+0000 7faa7627f640 1 --2- 172.21.15.106:0/6705671 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faa5c0051c0 0x7faa5c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:45.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.711+0000 7faa7627f640 1 -- 172.21.15.106:0/6705671 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa5c004800 msgr2=0x7faa5c004c10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T10:33:45.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.711+0000 7faa7627f640 1 --2- 172.21.15.106:0/6705671 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa5c004800 0x7faa5c004c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:45.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.711+0000 7faa7627f640 1 -- 172.21.15.106:0/6705671 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa5c003b40 msgr2=0x7faa5c003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:45.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.711+0000 7faa7627f640 1 --2- 172.21.15.106:0/6705671 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa5c003b40 0x7faa5c003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:45.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.711+0000 7faa7627f640 1 -- 172.21.15.106:0/6705671 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa5c003520 con 0x7faa5c0051c0 2024-04-25T10:33:45.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.712+0000 7faa7627f640 1 -- 172.21.15.106:0/6705671 reap_dead start 2024-04-25T10:33:45.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.712+0000 7faa7627f640 1 --2- 172.21.15.106:0/6705671 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faa5c0051c0 0x7faa5c00b9c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7faa7015aa80 tx=0x7faa5c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ea9ff26fab67008 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:45.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.712+0000 7faa74a7c640 1 -- 172.21.15.106:0/6705671 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa68009de0 con 0x7faa5c0051c0 2024-04-25T10:33:45.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.712+0000 7faa74a7c640 1 -- 172.21.15.106:0/6705671 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa68003b90 con 0x7faa5c0051c0 2024-04-25T10:33:45.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.712+0000 7faa74a7c640 1 -- 172.21.15.106:0/6705671 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa68004020 con 0x7faa5c0051c0 2024-04-25T10:33:45.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.713+0000 7faa77ce2640 1 -- 172.21.15.106:0/6705671 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faa5c0051c0 msgr2=0x7faa5c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:45.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.713+0000 7faa77ce2640 1 --2- 172.21.15.106:0/6705671 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faa5c0051c0 0x7faa5c00b9c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7faa7015aa80 tx=0x7faa5c005830 comp rx=0 tx=0).stop 2024-04-25T10:33:45.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.714+0000 7faa77ce2640 1 -- 172.21.15.106:0/6705671 shutdown_connections 2024-04-25T10:33:45.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.714+0000 7faa77ce2640 1 --2- 172.21.15.106:0/6705671 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faa5c0051c0 0x7faa5c00b9c0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:45.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.714+0000 7faa77ce2640 1 -- 172.21.15.106:0/6705671 >> 172.21.15.106:0/6705671 conn(0x7faa7006c5c0 msgr2=0x7faa7006c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:45.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.714+0000 7faa77ce2640 1 -- 172.21.15.106:0/6705671 shutdown_connections 2024-04-25T10:33:45.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.714+0000 7faa77ce2640 1 -- 172.21.15.106:0/6705671 wait complete. 2024-04-25T10:33:45.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.717+0000 7faa77ce2640 1 Processor -- start 2024-04-25T10:33:45.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.717+0000 7faa77ce2640 1 -- start start 2024-04-25T10:33:45.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.719+0000 7faa77ce2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa5c0051c0 0x7faa700cc860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:45.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.719+0000 7faa77ce2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faa701437e0 0x7faa701f4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:45.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.720+0000 7faa77ce2640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa701f49b0 0x7faa70150e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:45.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.720+0000 7faa77ce2640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7faa700c41d0 con 0x7faa701437e0 2024-04-25T10:33:45.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.720+0000 7faa77ce2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7faa700c3ef0 con 0x7faa5c0051c0 2024-04-25T10:33:45.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.720+0000 7faa77ce2640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7faa7015cee0 con 0x7faa701f49b0 2024-04-25T10:33:45.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.720+0000 7faa7527d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faa701437e0 0x7faa701f4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:45.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.720+0000 7faa7527d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faa701437e0 0x7faa701f4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:39524/0 (socket says 172.21.15.106:39524) 2024-04-25T10:33:45.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.720+0000 7faa7527d640 1 -- 172.21.15.106:0/2545365653 learned_addr learned my addr 172.21.15.106:0/2545365653 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:33:45.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.720+0000 7faa75a7e640 1 --2- 172.21.15.106:0/2545365653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa5c0051c0 0x7faa700cc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:45.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.720+0000 7faa7627f640 1 --2- 172.21.15.106:0/2545365653 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa701f49b0 0x7faa70150e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:45.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.720+0000 7faa7527d640 1 -- 172.21.15.106:0/2545365653 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa701f49b0 msgr2=0x7faa70150e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:45.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.720+0000 7faa7527d640 1 --2- 172.21.15.106:0/2545365653 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa701f49b0 0x7faa70150e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:45.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.720+0000 7faa7527d640 1 -- 172.21.15.106:0/2545365653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa5c0051c0 msgr2=0x7faa700cc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:45.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.720+0000 7faa7527d640 1 --2- 172.21.15.106:0/2545365653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa5c0051c0 0x7faa700cc860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:45.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.720+0000 7faa7527d640 1 -- 172.21.15.106:0/2545365653 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa700ccda0 con 0x7faa701437e0 2024-04-25T10:33:45.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.721+0000 7faa7527d640 1 --2- 172.21.15.106:0/2545365653 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faa701437e0 0x7faa701f4470 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7faa6000c730 tx=0x7faa6000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:45.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.721+0000 7faa667fc640 1 -- 172.21.15.106:0/2545365653 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa60018070 con 0x7faa701437e0 2024-04-25T10:33:45.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.721+0000 7faa667fc640 1 -- 172.21.15.106:0/2545365653 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa600072f0 con 0x7faa701437e0 2024-04-25T10:33:45.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.721+0000 7faa77ce2640 1 -- 172.21.15.106:0/2545365653 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa70138730 con 0x7faa701437e0 2024-04-25T10:33:45.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.721+0000 7faa667fc640 1 -- 172.21.15.106:0/2545365653 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa600047e0 con 0x7faa701437e0 2024-04-25T10:33:45.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.721+0000 7faa77ce2640 1 -- 172.21.15.106:0/2545365653 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa70144c40 con 0x7faa701437e0 2024-04-25T10:33:45.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.722+0000 7faa77ce2640 1 --2- 172.21.15.106:0/2545365653 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa70143290 0x7faa70144e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:45.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.722+0000 7faa77ce2640 1 -- 172.21.15.106:0/2545365653 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7faa700cd270 con 0x7faa70143290 2024-04-25T10:33:45.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.723+0000 7faa75a7e640 1 --2- 172.21.15.106:0/2545365653 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa70143290 0x7faa70144e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:45.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.723+0000 7faa667fc640 1 -- 172.21.15.106:0/2545365653 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7faa60004980 con 0x7faa701437e0 2024-04-25T10:33:45.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.723+0000 7faa75a7e640 1 -- 172.21.15.106:0/2545365653 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7faa701f51a0 con 0x7faa70143290 2024-04-25T10:33:45.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.724+0000 7faa75a7e640 1 --2- 172.21.15.106:0/2545365653 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa70143290 0x7faa70144e10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7faa6c00e700 tx=0x7faa6c00ebc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:45.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.724+0000 7faa667fc640 1 --2- 172.21.15.106:0/2545365653 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7faa540744b0 0x7faa540769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:45.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.724+0000 7faa667fc640 1 -- 172.21.15.106:0/2545365653 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7faa60094880 con 0x7faa701437e0 2024-04-25T10:33:45.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.724+0000 7faa7627f640 1 --2- 172.21.15.106:0/2545365653 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7faa540744b0 0x7faa540769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:45.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.724+0000 7faa667fc640 1 -- 172.21.15.106:0/2545365653 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa6c00c6e0 con 0x7faa70143290 2024-04-25T10:33:45.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.724+0000 7faa667fc640 1 -- 172.21.15.106:0/2545365653 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7faa6c01a070 con 0x7faa70143290 2024-04-25T10:33:45.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.724+0000 7faa667fc640 1 -- 172.21.15.106:0/2545365653 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa70143290 msgr2=0x7faa70144e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:45.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.724+0000 7faa667fc640 1 --2- 172.21.15.106:0/2545365653 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa70143290 0x7faa70144e10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7faa6c00e700 tx=0x7faa6c00ebc0 comp rx=0 tx=0).stop 2024-04-25T10:33:45.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.724+0000 7faa667fc640 1 -- 172.21.15.106:0/2545365653 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa70143290 msgr2=0x7faa70144e10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T10:33:45.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.724+0000 7faa667fc640 1 --2- 172.21.15.106:0/2545365653 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa70143290 0x7faa70144e10 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:45.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.725+0000 7faa7627f640 1 --2- 172.21.15.106:0/2545365653 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7faa540744b0 0x7faa540769c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7faa5c00c090 tx=0x7faa6802f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:45.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.733+0000 7faa77ce2640 1 --2- 172.21.15.106:0/2545365653 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa701513c0 0x7faa70136810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:45.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.733+0000 7faa77ce2640 1 -- 172.21.15.106:0/2545365653 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7faa701523f0 con 0x7faa701513c0 2024-04-25T10:33:45.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.734+0000 7faa75a7e640 1 --2- 172.21.15.106:0/2545365653 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa701513c0 0x7faa70136810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:45.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.735+0000 7faa75a7e640 1 -- 172.21.15.106:0/2545365653 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7faa701692b0 con 0x7faa701513c0 2024-04-25T10:33:45.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.736+0000 7faa75a7e640 1 --2- 172.21.15.106:0/2545365653 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa701513c0 0x7faa70136810 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7faa6c00e700 tx=0x7faa6c000ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:45.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.736+0000 7faa667fc640 1 -- 172.21.15.106:0/2545365653 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa6c025070 con 0x7faa701513c0 2024-04-25T10:33:45.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.736+0000 7faa667fc640 1 -- 172.21.15.106:0/2545365653 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4729 (secure 0 0 0) 0x7faa6c015430 con 0x7faa701513c0 2024-04-25T10:33:45.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.736+0000 7faa667fc640 1 -- 172.21.15.106:0/2545365653 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa701513c0 msgr2=0x7faa70136810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:45.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.736+0000 7faa667fc640 1 --2- 172.21.15.106:0/2545365653 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa701513c0 0x7faa70136810 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7faa6c00e700 tx=0x7faa6c000ed0 comp rx=0 tx=0).stop 2024-04-25T10:33:45.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.736+0000 7faa667fc640 1 -- 172.21.15.106:0/2545365653 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa701513c0 msgr2=0x7faa70136810 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T10:33:45.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.737+0000 7faa667fc640 1 --2- 172.21.15.106:0/2545365653 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa701513c0 0x7faa70136810 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:45.739 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-25T10:33:45.740 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "c", 2024-04-25T10:33:45.740 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 2, 2024-04-25T10:33:45.740 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "peon", 2024-04-25T10:33:45.740 INFO:teuthology.orchestra.run.smithi106.stdout: "election_epoch": 22, 2024-04-25T10:33:45.740 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 64660, 2024-04-25T10:33:45.740 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum": [ 2024-04-25T10:33:45.740 INFO:teuthology.orchestra.run.smithi106.stdout: 0, 2024-04-25T10:33:45.740 INFO:teuthology.orchestra.run.smithi106.stdout: 1, 2024-04-25T10:33:45.740 INFO:teuthology.orchestra.run.smithi106.stdout: 2 2024-04-25T10:33:45.740 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:33:45.740 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_age": 12, 2024-04-25T10:33:45.740 INFO:teuthology.orchestra.run.smithi106.stdout: "features": { 2024-04-25T10:33:45.740 INFO:teuthology.orchestra.run.smithi106.stdout: "required_con": "2450521708007866372", 2024-04-25T10:33:45.740 INFO:teuthology.orchestra.run.smithi106.stdout: "required_mon": [ 2024-04-25T10:33:45.740 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:33:45.741 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:33:45.741 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:33:45.741 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:33:45.741 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:33:45.741 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:33:45.741 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:33:45.741 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:33:45.741 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:33:45.741 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:33:45.741 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:33:45.741 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:33:45.741 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_con": "4540701547738038271", 2024-04-25T10:33:45.741 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_mon": [ 2024-04-25T10:33:45.741 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:33:45.741 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:33:45.742 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:33:45.742 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:33:45.742 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:33:45.742 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:33:45.742 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:33:45.742 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:33:45.742 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:33:45.742 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:33:45.742 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:33:45.742 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:33:45.742 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:45.742 INFO:teuthology.orchestra.run.smithi106.stdout: "outside_quorum": [], 2024-04-25T10:33:45.742 INFO:teuthology.orchestra.run.smithi106.stdout: "extra_probe_peers": [], 2024-04-25T10:33:45.742 INFO:teuthology.orchestra.run.smithi106.stdout: "sync_provider": [], 2024-04-25T10:33:45.743 INFO:teuthology.orchestra.run.smithi106.stdout: "monmap": { 2024-04-25T10:33:45.743 INFO:teuthology.orchestra.run.smithi106.stdout: "epoch": 1, 2024-04-25T10:33:45.743 INFO:teuthology.orchestra.run.smithi106.stdout: "fsid": "3ee77985-e8e4-41cf-81e9-7c774cf7891f", 2024-04-25T10:33:45.743 INFO:teuthology.orchestra.run.smithi106.stdout: "modified": "2024-04-25T10:29:33.446228Z", 2024-04-25T10:33:45.743 INFO:teuthology.orchestra.run.smithi106.stdout: "created": "2024-04-25T10:29:33.446228Z", 2024-04-25T10:33:45.743 INFO:teuthology.orchestra.run.smithi106.stdout: "min_mon_release": 19, 2024-04-25T10:33:45.743 INFO:teuthology.orchestra.run.smithi106.stdout: "min_mon_release_name": "squid", 2024-04-25T10:33:45.743 INFO:teuthology.orchestra.run.smithi106.stdout: "election_strategy": 1, 2024-04-25T10:33:45.743 INFO:teuthology.orchestra.run.smithi106.stdout: "disallowed_leaders: ": "", 2024-04-25T10:33:45.743 INFO:teuthology.orchestra.run.smithi106.stdout: "stretch_mode": false, 2024-04-25T10:33:45.743 INFO:teuthology.orchestra.run.smithi106.stdout: "tiebreaker_mon": "", 2024-04-25T10:33:45.743 INFO:teuthology.orchestra.run.smithi106.stdout: "removed_ranks: ": "", 2024-04-25T10:33:45.743 INFO:teuthology.orchestra.run.smithi106.stdout: "features": { 2024-04-25T10:33:45.743 INFO:teuthology.orchestra.run.smithi106.stdout: "persistent": [ 2024-04-25T10:33:45.744 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:33:45.744 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:33:45.744 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:33:45.744 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:33:45.744 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:33:45.744 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:33:45.744 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:33:45.744 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:33:45.744 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:33:45.744 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:33:45.744 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:33:45.744 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:33:45.744 INFO:teuthology.orchestra.run.smithi106.stdout: "optional": [] 2024-04-25T10:33:45.744 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:45.744 INFO:teuthology.orchestra.run.smithi106.stdout: "mons": [ 2024-04-25T10:33:45.745 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:45.745 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 0, 2024-04-25T10:33:45.745 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "a", 2024-04-25T10:33:45.745 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:33:45.745 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:33:45.745 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:45.745 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:33:45.745 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:3300", 2024-04-25T10:33:45.745 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:33:45.745 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:45.745 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:45.745 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:33:45.745 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:6789", 2024-04-25T10:33:45.745 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:33:45.745 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:33:45.746 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:33:45.746 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:45.746 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:6789/0", 2024-04-25T10:33:45.746 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.106:6789/0", 2024-04-25T10:33:45.746 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:33:45.746 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:33:45.746 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:33:45.746 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:45.746 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:45.746 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 1, 2024-04-25T10:33:45.746 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "b", 2024-04-25T10:33:45.746 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:33:45.746 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:33:45.746 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:45.747 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:33:45.747 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:3300", 2024-04-25T10:33:45.747 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:33:45.747 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:45.747 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:45.747 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:33:45.747 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6789", 2024-04-25T10:33:45.747 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:33:45.747 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:33:45.747 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:33:45.747 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:45.747 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6789/0", 2024-04-25T10:33:45.747 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.134:6789/0", 2024-04-25T10:33:45.747 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:33:45.748 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:33:45.748 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:33:45.748 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:45.748 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:45.748 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 2, 2024-04-25T10:33:45.748 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "c", 2024-04-25T10:33:45.748 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:33:45.748 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:33:45.748 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:45.748 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:33:45.748 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:3301", 2024-04-25T10:33:45.748 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:33:45.748 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:45.748 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:45.748 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:33:45.749 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6790", 2024-04-25T10:33:45.749 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:33:45.749 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:33:45.749 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:33:45.749 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:45.749 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6790/0", 2024-04-25T10:33:45.749 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.134:6790/0", 2024-04-25T10:33:45.749 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:33:45.749 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:33:45.749 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:33:45.749 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:33:45.749 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:33:45.749 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:45.749 INFO:teuthology.orchestra.run.smithi106.stdout: "feature_map": { 2024-04-25T10:33:45.749 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": [ 2024-04-25T10:33:45.750 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:45.750 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:33:45.750 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:33:45.750 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 1 2024-04-25T10:33:45.750 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:33:45.750 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:33:45.750 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": [ 2024-04-25T10:33:45.750 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:45.750 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:33:45.750 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:33:45.750 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 2 2024-04-25T10:33:45.750 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:33:45.750 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:33:45.750 INFO:teuthology.orchestra.run.smithi106.stdout: "client": [ 2024-04-25T10:33:45.750 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:33:45.751 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:33:45.751 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:33:45.751 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 1 2024-04-25T10:33:45.751 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:33:45.751 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:33:45.751 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:33:45.751 INFO:teuthology.orchestra.run.smithi106.stdout: "stretch_mode": false 2024-04-25T10:33:45.751 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-25T10:33:45.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.737+0000 7faa77ce2640 1 -- 172.21.15.106:0/2545365653 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7faa540744b0 msgr2=0x7faa540769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:45.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.737+0000 7faa77ce2640 1 --2- 172.21.15.106:0/2545365653 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7faa540744b0 0x7faa540769c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7faa5c00c090 tx=0x7faa6802f000 comp rx=0 tx=0).stop 2024-04-25T10:33:45.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.737+0000 7faa7627f640 1 -- 172.21.15.106:0/2545365653 reap_dead start 2024-04-25T10:33:45.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.737+0000 7faa77ce2640 1 -- 172.21.15.106:0/2545365653 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faa701437e0 msgr2=0x7faa701f4470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:45.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.737+0000 7faa77ce2640 1 --2- 172.21.15.106:0/2545365653 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faa701437e0 0x7faa701f4470 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7faa6000c730 tx=0x7faa6000cbf0 comp rx=0 tx=0).stop 2024-04-25T10:33:45.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.737+0000 7faa77ce2640 1 -- 172.21.15.106:0/2545365653 shutdown_connections 2024-04-25T10:33:45.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.737+0000 7faa77ce2640 1 --2- 172.21.15.106:0/2545365653 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faa701437e0 0x7faa701f4470 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:45.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.737+0000 7faa77ce2640 1 -- 172.21.15.106:0/2545365653 >> 172.21.15.106:0/2545365653 conn(0x7faa7006c5c0 msgr2=0x7faa7015d180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:45.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.737+0000 7faa77ce2640 1 -- 172.21.15.106:0/2545365653 shutdown_connections 2024-04-25T10:33:45.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.737+0000 7faa77ce2640 1 -- 172.21.15.106:0/2545365653 wait complete. 2024-04-25T10:33:45.756 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T10:33:45.756 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2024-04-25T10:33:45.756 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-04-25T10:33:45.756 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2024-04-25T10:33:45.756 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-04-25T10:33:45.756 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-25T10:33:45.756 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:33:45.777 INFO:tasks.ceph.mon.a:Stopped 2024-04-25T10:33:45.777 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-04-25T10:33:45.777 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T10:33:45.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.970+0000 7efe0a91f640 1 Processor -- start 2024-04-25T10:33:45.972 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.970+0000 7efe0a91f640 1 -- start start 2024-04-25T10:33:45.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.972+0000 7efe0a91f640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7efe040ca170 0x7efe04154960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:45.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.972+0000 7efe0a91f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe04073000 0x7efe040caed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:45.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.972+0000 7efe0a91f640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe040cb820 con 0x7efe040cb410 2024-04-25T10:33:45.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.972+0000 7efe0a91f640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7efe0415df50 con 0x7efe040ca170 2024-04-25T10:33:45.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.972+0000 7efe0a91f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7efe0415ac00 con 0x7efe04073000 2024-04-25T10:33:45.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.972+0000 7efe08ebc640 1 -- >> v1:172.21.15.106:6789/0 conn(0x7efe040cb410 legacy=0x7efe0415e3e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.106:6789/0 2024-04-25T10:33:45.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.973+0000 7efe03fff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7efe040ca170 0x7efe04154960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:45.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.973+0000 7efe03fff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7efe040ca170 0x7efe04154960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.106:48366/0 (socket says 172.21.15.106:48366) 2024-04-25T10:33:45.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.973+0000 7efe03fff640 1 -- 172.21.15.106:0/193913905 learned_addr learned my addr 172.21.15.106:0/193913905 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:33:45.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.973+0000 7efe037fe640 1 --2- 172.21.15.106:0/193913905 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe04073000 0x7efe040caed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:45.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.974+0000 7efe03fff640 1 -- 172.21.15.106:0/193913905 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe04073000 msgr2=0x7efe040caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:45.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.974+0000 7efe03fff640 1 --2- 172.21.15.106:0/193913905 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe04073000 0x7efe040caed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:45.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.974+0000 7efe03fff640 1 -- 172.21.15.106:0/193913905 >> v1:172.21.15.106:6789/0 conn(0x7efe040cb410 legacy=0x7efe0415e3e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:33:45.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.974+0000 7efe03fff640 1 -- 172.21.15.106:0/193913905 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe0415aee0 con 0x7efe040ca170 2024-04-25T10:33:45.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.974+0000 7efe037fe640 1 --2- 172.21.15.106:0/193913905 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe04073000 0x7efe040caed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:33:45.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.975+0000 7efe03fff640 1 --2- 172.21.15.106:0/193913905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe040ca170 0x7efe04154960 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7efdf4009870 tx=0x7efdf402a2a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4563b80a16019eaa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:45.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.975+0000 7efe02ffd640 1 -- 172.21.15.106:0/193913905 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf4037070 con 0x7efe040ca170 2024-04-25T10:33:45.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.975+0000 7efe02ffd640 1 -- 172.21.15.106:0/193913905 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdf4032440 con 0x7efe040ca170 2024-04-25T10:33:45.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.975+0000 7efe02ffd640 1 -- 172.21.15.106:0/193913905 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf4032810 con 0x7efe040ca170 2024-04-25T10:33:45.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.977+0000 7efe0a91f640 1 -- 172.21.15.106:0/193913905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe040ca170 msgr2=0x7efe04154960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:45.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.977+0000 7efe0a91f640 1 --2- 172.21.15.106:0/193913905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe040ca170 0x7efe04154960 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7efdf4009870 tx=0x7efdf402a2a0 comp rx=0 tx=0).stop 2024-04-25T10:33:45.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.977+0000 7efe0a91f640 1 -- 172.21.15.106:0/193913905 shutdown_connections 2024-04-25T10:33:45.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.977+0000 7efe0a91f640 1 --2- 172.21.15.106:0/193913905 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe04073000 0x7efe040caed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:45.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.977+0000 7efe0a91f640 1 --2- 172.21.15.106:0/193913905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe040ca170 0x7efe04154960 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:45.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.977+0000 7efe0a91f640 1 -- 172.21.15.106:0/193913905 >> 172.21.15.106:0/193913905 conn(0x7efe0406c5c0 msgr2=0x7efe0406c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:45.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.978+0000 7efe0a91f640 1 -- 172.21.15.106:0/193913905 shutdown_connections 2024-04-25T10:33:45.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.978+0000 7efe0a91f640 1 -- 172.21.15.106:0/193913905 wait complete. 2024-04-25T10:33:45.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.981+0000 7efe0a91f640 1 Processor -- start 2024-04-25T10:33:45.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.981+0000 7efe0a91f640 1 -- start start 2024-04-25T10:33:45.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.983+0000 7efe0a91f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe04073000 0x7efe040c6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:45.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.983+0000 7efe0a91f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe040ca170 0x7efe0415d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:45.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.983+0000 7efe037fe640 1 -- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe040ca170 msgr2=0x7efe0415d190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.106:3300/0 2024-04-25T10:33:45.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.983+0000 7efe037fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe040ca170 0x7efe0415d190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:33:45.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.983+0000 7efe0a91f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe040cb410 0x7efe04153ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:45.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.983+0000 7efe0a91f640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7efe0415b780 con 0x7efe040ca170 2024-04-25T10:33:45.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.983+0000 7efe0a91f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efe0415b4a0 con 0x7efe040cb410 2024-04-25T10:33:45.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.983+0000 7efe0a91f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7efe04068b10 con 0x7efe04073000 2024-04-25T10:33:45.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.984+0000 7efe08ebc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe040cb410 0x7efe04153ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:45.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.984+0000 7efe03fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe04073000 0x7efe040c6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:45.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.984+0000 7efe08ebc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe040cb410 0x7efe04153ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.106:48378/0 (socket says 172.21.15.106:48378) 2024-04-25T10:33:45.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.984+0000 7efe08ebc640 1 -- 172.21.15.106:0/4144404201 learned_addr learned my addr 172.21.15.106:0/4144404201 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:33:45.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.984+0000 7efe08ebc640 1 -- 172.21.15.106:0/4144404201 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe04073000 msgr2=0x7efe040c6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:45.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.984+0000 7efe08ebc640 1 --2- 172.21.15.106:0/4144404201 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe04073000 0x7efe040c6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:45.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.984+0000 7efe08ebc640 1 -- 172.21.15.106:0/4144404201 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe040ca170 msgr2=0x7efe0415d190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:33:45.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.984+0000 7efe08ebc640 1 --2- 172.21.15.106:0/4144404201 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe040ca170 0x7efe0415d190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:45.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.984+0000 7efe08ebc640 1 -- 172.21.15.106:0/4144404201 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe040c6f60 con 0x7efe040cb410 2024-04-25T10:33:45.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.984+0000 7efe03fff640 1 --2- 172.21.15.106:0/4144404201 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe04073000 0x7efe040c6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:33:45.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.985+0000 7efe08ebc640 1 --2- 172.21.15.106:0/4144404201 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe040cb410 0x7efe04153ac0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7efdfc00be60 tx=0x7efdfc00bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:45.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.985+0000 7efe00ff9640 1 -- 172.21.15.106:0/4144404201 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdfc00cb50 con 0x7efe040cb410 2024-04-25T10:33:45.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.985+0000 7efe00ff9640 1 -- 172.21.15.106:0/4144404201 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdfc010030 con 0x7efe040cb410 2024-04-25T10:33:45.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.985+0000 7efe00ff9640 1 -- 172.21.15.106:0/4144404201 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdfc016430 con 0x7efe040cb410 2024-04-25T10:33:45.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.985+0000 7efe0a91f640 1 -- 172.21.15.106:0/4144404201 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe041f1550 con 0x7efe040cb410 2024-04-25T10:33:45.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.985+0000 7efe0a91f640 1 -- 172.21.15.106:0/4144404201 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe041f1b10 con 0x7efe040cb410 2024-04-25T10:33:45.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.987+0000 7efe0a91f640 1 -- 172.21.15.106:0/4144404201 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe041f12c0 con 0x7efe040cb410 2024-04-25T10:33:45.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.988+0000 7efe00ff9640 1 -- 172.21.15.106:0/4144404201 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7efdfc008070 con 0x7efe040cb410 2024-04-25T10:33:45.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.988+0000 7efe00ff9640 1 --2- 172.21.15.106:0/4144404201 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7efdf0074490 0x7efdf00769a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:45.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.988+0000 7efe00ff9640 1 -- 172.21.15.106:0/4144404201 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7efdfc036e00 con 0x7efe040cb410 2024-04-25T10:33:45.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.988+0000 7efe03fff640 1 --2- 172.21.15.106:0/4144404201 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7efdf0074490 0x7efdf00769a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:45.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.989+0000 7efe03fff640 1 --2- 172.21.15.106:0/4144404201 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7efdf0074490 0x7efdf00769a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7efdf4034000 tx=0x7efdf402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:45.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:45.992+0000 7efe00ff9640 1 -- 172.21.15.106:0/4144404201 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efdfc05f510 con 0x7efe040cb410 2024-04-25T10:33:46.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:46.347+0000 7efe0a91f640 1 -- 172.21.15.106:0/4144404201 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7efe04073410 con 0x7efe040cb410 2024-04-25T10:33:46.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:46.348+0000 7efe00ff9640 1 -- 172.21.15.106:0/4144404201 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1569 (secure 0 0 0) 0x7efe04073410 con 0x7efe040cb410 2024-04-25T10:33:46.350 INFO:teuthology.orchestra.run.smithi106.stdout:{"election_epoch":22,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"3ee77985-e8e4-41cf-81e9-7c774cf7891f","modified":"2024-04-25T10:29:33.446228Z","created":"2024-04-25T10:29:33.446228Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:3300","nonce":0},{"type":"v1","addr":"172.21.15.106:6789","nonce":0}]},"addr":"172.21.15.106:6789/0","public_addr":"172.21.15.106:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T10:33:46.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:46.350+0000 7efe0a91f640 1 -- 172.21.15.106:0/4144404201 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7efdf0074490 msgr2=0x7efdf00769a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:46.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:46.350+0000 7efe0a91f640 1 --2- 172.21.15.106:0/4144404201 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7efdf0074490 0x7efdf00769a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7efdf4034000 tx=0x7efdf402e000 comp rx=0 tx=0).stop 2024-04-25T10:33:46.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:46.350+0000 7efe0a91f640 1 -- 172.21.15.106:0/4144404201 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe040cb410 msgr2=0x7efe04153ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:46.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:46.350+0000 7efe0a91f640 1 --2- 172.21.15.106:0/4144404201 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe040cb410 0x7efe04153ac0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7efdfc00be60 tx=0x7efdfc00bf50 comp rx=0 tx=0).stop 2024-04-25T10:33:46.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:46.350+0000 7efe0a91f640 1 -- 172.21.15.106:0/4144404201 shutdown_connections 2024-04-25T10:33:46.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:46.350+0000 7efe0a91f640 1 --2- 172.21.15.106:0/4144404201 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7efdf0074490 0x7efdf00769a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:46.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:46.350+0000 7efe0a91f640 1 --2- 172.21.15.106:0/4144404201 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe040cb410 0x7efe04153ac0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:46.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:46.350+0000 7efe0a91f640 1 --2- 172.21.15.106:0/4144404201 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe040ca170 0x7efe0415d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:46.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:46.350+0000 7efe0a91f640 1 --2- 172.21.15.106:0/4144404201 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe04073000 0x7efe040c6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:46.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:46.350+0000 7efe0a91f640 1 -- 172.21.15.106:0/4144404201 >> 172.21.15.106:0/4144404201 conn(0x7efe0406c5c0 msgr2=0x7efe040706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:46.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:46.351+0000 7efe0a91f640 1 -- 172.21.15.106:0/4144404201 shutdown_connections 2024-04-25T10:33:46.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:46.351+0000 7efe0a91f640 1 -- 172.21.15.106:0/4144404201 wait complete. 2024-04-25T10:33:49.374 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T10:33:49.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.536+0000 7ff3bc8e7640 1 Processor -- start 2024-04-25T10:33:49.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.536+0000 7ff3bc8e7640 1 -- start start 2024-04-25T10:33:49.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.538+0000 7ff3bc8e7640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff3b8074790 0x7ff3b80ca4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:49.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.538+0000 7ff3bc8e7640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff3b80c8fe0 0x7ff3b80c38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:49.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.539+0000 7ff3b5d9b640 1 -- >> v2:172.21.15.106:3300/0 conn(0x7ff3b80c8fe0 msgr2=0x7ff3b80c38a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.106:3300/0 2024-04-25T10:33:49.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.539+0000 7ff3b5d9b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff3b80c8fe0 0x7ff3b80c38a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:33:49.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.539+0000 7ff3b6d9d640 1 -- >> v1:172.21.15.106:6789/0 conn(0x7ff3b8153a80 legacy=0x7ff3b8159800 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.106:6789/0 2024-04-25T10:33:49.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.539+0000 7ff3bc8e7640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3b80c93f0 con 0x7ff3b8153a80 2024-04-25T10:33:49.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.539+0000 7ff3bc8e7640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7ff3b80c8520 con 0x7ff3b80c8fe0 2024-04-25T10:33:49.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.539+0000 7ff3bc8e7640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7ff3b815a740 con 0x7ff3b8074790 2024-04-25T10:33:49.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.539+0000 7ff3b659c640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff3b8074790 0x7ff3b80ca4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:49.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.539+0000 7ff3b659c640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff3b8074790 0x7ff3b80ca4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.106:48386/0 (socket says 172.21.15.106:48386) 2024-04-25T10:33:49.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.539+0000 7ff3b659c640 1 -- 172.21.15.106:0/1279802201 learned_addr learned my addr 172.21.15.106:0/1279802201 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:33:49.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.539+0000 7ff3b659c640 1 -- 172.21.15.106:0/1279802201 >> v2:172.21.15.106:3300/0 conn(0x7ff3b80c8fe0 msgr2=0x7ff3b80c38a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:33:49.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.539+0000 7ff3b659c640 1 --2- 172.21.15.106:0/1279802201 >> v2:172.21.15.106:3300/0 conn(0x7ff3b80c8fe0 0x7ff3b80c38a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:49.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.539+0000 7ff3b659c640 1 -- 172.21.15.106:0/1279802201 >> v1:172.21.15.106:6789/0 conn(0x7ff3b8153a80 legacy=0x7ff3b8159800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:33:49.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.539+0000 7ff3b659c640 1 -- 172.21.15.106:0/1279802201 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3b80c5330 con 0x7ff3b8074790 2024-04-25T10:33:49.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.540+0000 7ff3b659c640 1 --2- 172.21.15.106:0/1279802201 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3b8074790 0x7ff3b80ca4f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff3b80c86a0 tx=0x7ff3a802c870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d18c4d975e68fa5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:49.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.541+0000 7ff3b559a640 1 -- 172.21.15.106:0/1279802201 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a8034620 con 0x7ff3b8074790 2024-04-25T10:33:49.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.541+0000 7ff3b559a640 1 -- 172.21.15.106:0/1279802201 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3a80347c0 con 0x7ff3b8074790 2024-04-25T10:33:49.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.541+0000 7ff3b559a640 1 -- 172.21.15.106:0/1279802201 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a8034b90 con 0x7ff3b8074790 2024-04-25T10:33:49.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.542+0000 7ff3bc8e7640 1 -- 172.21.15.106:0/1279802201 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3b8074790 msgr2=0x7ff3b80ca4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:49.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.542+0000 7ff3bc8e7640 1 --2- 172.21.15.106:0/1279802201 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3b8074790 0x7ff3b80ca4f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff3b80c86a0 tx=0x7ff3a802c870 comp rx=0 tx=0).stop 2024-04-25T10:33:49.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.542+0000 7ff3bc8e7640 1 -- 172.21.15.106:0/1279802201 shutdown_connections 2024-04-25T10:33:49.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.543+0000 7ff3bc8e7640 1 --2- 172.21.15.106:0/1279802201 >> v2:172.21.15.106:3300/0 conn(0x7ff3b80c8fe0 0x7ff3b80c38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:49.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.543+0000 7ff3bc8e7640 1 --2- 172.21.15.106:0/1279802201 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3b8074790 0x7ff3b80ca4f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:49.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.543+0000 7ff3bc8e7640 1 -- 172.21.15.106:0/1279802201 >> 172.21.15.106:0/1279802201 conn(0x7ff3b806c5c0 msgr2=0x7ff3b806c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:49.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.543+0000 7ff3bc8e7640 1 -- 172.21.15.106:0/1279802201 shutdown_connections 2024-04-25T10:33:49.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.543+0000 7ff3bc8e7640 1 -- 172.21.15.106:0/1279802201 wait complete. 2024-04-25T10:33:49.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.546+0000 7ff3bc8e7640 1 Processor -- start 2024-04-25T10:33:49.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.546+0000 7ff3bc8e7640 1 -- start start 2024-04-25T10:33:49.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.548+0000 7ff3bc8e7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3b8074790 0x7ff3b81561c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:49.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.548+0000 7ff3bc8e7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff3b80c8fe0 0x7ff3b8154760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:49.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.548+0000 7ff3bc8e7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3b8153a80 0x7ff3b8154ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:49.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.548+0000 7ff3bc8e7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b80c8a50 con 0x7ff3b80c8fe0 2024-04-25T10:33:49.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.548+0000 7ff3bc8e7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b80c8770 con 0x7ff3b8074790 2024-04-25T10:33:49.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.548+0000 7ff3bc8e7640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff3b8153e90 con 0x7ff3b8153a80 2024-04-25T10:33:49.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.548+0000 7ff3b5d9b640 1 -- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff3b80c8fe0 msgr2=0x7ff3b8154760 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.106:3300/0 2024-04-25T10:33:49.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.548+0000 7ff3b5d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff3b80c8fe0 0x7ff3b8154760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:33:49.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.548+0000 7ff3b659c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3b8074790 0x7ff3b81561c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:49.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.548+0000 7ff3b659c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3b8074790 0x7ff3b81561c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.106:48396/0 (socket says 172.21.15.106:48396) 2024-04-25T10:33:49.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.548+0000 7ff3b659c640 1 -- 172.21.15.106:0/211393658 learned_addr learned my addr 172.21.15.106:0/211393658 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:33:49.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.548+0000 7ff3b6d9d640 1 --2- 172.21.15.106:0/211393658 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3b8153a80 0x7ff3b8154ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:49.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.549+0000 7ff3b659c640 1 -- 172.21.15.106:0/211393658 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3b8153a80 msgr2=0x7ff3b8154ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:49.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.549+0000 7ff3b659c640 1 --2- 172.21.15.106:0/211393658 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3b8153a80 0x7ff3b8154ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:49.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.549+0000 7ff3b659c640 1 -- 172.21.15.106:0/211393658 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff3b80c8fe0 msgr2=0x7ff3b8154760 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:33:49.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.549+0000 7ff3b659c640 1 --2- 172.21.15.106:0/211393658 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff3b80c8fe0 0x7ff3b8154760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:49.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.549+0000 7ff3b659c640 1 -- 172.21.15.106:0/211393658 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3b80c7a90 con 0x7ff3b8074790 2024-04-25T10:33:49.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.549+0000 7ff3b659c640 1 --2- 172.21.15.106:0/211393658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3b8074790 0x7ff3b81561c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff3a8000c00 tx=0x7ff3a8004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:49.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.550+0000 7ff3a6ffd640 1 -- 172.21.15.106:0/211393658 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a8043020 con 0x7ff3b8074790 2024-04-25T10:33:49.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.550+0000 7ff3a6ffd640 1 -- 172.21.15.106:0/211393658 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3a8033c00 con 0x7ff3b8074790 2024-04-25T10:33:49.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.550+0000 7ff3a6ffd640 1 -- 172.21.15.106:0/211393658 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a803f480 con 0x7ff3b8074790 2024-04-25T10:33:49.553 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.550+0000 7ff3bc8e7640 1 -- 172.21.15.106:0/211393658 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3b8156ba0 con 0x7ff3b8074790 2024-04-25T10:33:49.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.550+0000 7ff3bc8e7640 1 -- 172.21.15.106:0/211393658 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3b8153090 con 0x7ff3b8074790 2024-04-25T10:33:49.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.551+0000 7ff3bc8e7640 1 -- 172.21.15.106:0/211393658 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3b806a3d0 con 0x7ff3b8074790 2024-04-25T10:33:49.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.553+0000 7ff3a6ffd640 1 -- 172.21.15.106:0/211393658 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff3a8046020 con 0x7ff3b8074790 2024-04-25T10:33:49.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.553+0000 7ff3a6ffd640 1 --2- 172.21.15.106:0/211393658 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7ff398074490 0x7ff3980769a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:49.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.553+0000 7ff3a6ffd640 1 -- 172.21.15.106:0/211393658 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7ff3a80b6720 con 0x7ff3b8074790 2024-04-25T10:33:49.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.553+0000 7ff3b5d9b640 1 --2- 172.21.15.106:0/211393658 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7ff398074490 0x7ff3980769a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:49.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.554+0000 7ff3b5d9b640 1 --2- 172.21.15.106:0/211393658 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7ff398074490 0x7ff3980769a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac006fd0 tx=0x7ff3ac006d80 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:49.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.558+0000 7ff3a6ffd640 1 -- 172.21.15.106:0/211393658 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff3a804d070 con 0x7ff3b8074790 2024-04-25T10:33:49.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.934+0000 7ff3bc8e7640 1 -- 172.21.15.106:0/211393658 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7ff3b8156930 con 0x7ff3b8074790 2024-04-25T10:33:49.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.935+0000 7ff3a6ffd640 1 -- 172.21.15.106:0/211393658 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1569 (secure 0 0 0) 0x7ff3a8080ea0 con 0x7ff3b8074790 2024-04-25T10:33:49.937 INFO:teuthology.orchestra.run.smithi106.stdout:{"election_epoch":22,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":16,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"3ee77985-e8e4-41cf-81e9-7c774cf7891f","modified":"2024-04-25T10:29:33.446228Z","created":"2024-04-25T10:29:33.446228Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:3300","nonce":0},{"type":"v1","addr":"172.21.15.106:6789","nonce":0}]},"addr":"172.21.15.106:6789/0","public_addr":"172.21.15.106:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T10:33:49.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.937+0000 7ff3bc8e7640 1 -- 172.21.15.106:0/211393658 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7ff398074490 msgr2=0x7ff3980769a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:49.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.937+0000 7ff3bc8e7640 1 --2- 172.21.15.106:0/211393658 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7ff398074490 0x7ff3980769a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac006fd0 tx=0x7ff3ac006d80 comp rx=0 tx=0).stop 2024-04-25T10:33:49.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.937+0000 7ff3bc8e7640 1 -- 172.21.15.106:0/211393658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3b8074790 msgr2=0x7ff3b81561c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:49.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.937+0000 7ff3bc8e7640 1 --2- 172.21.15.106:0/211393658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3b8074790 0x7ff3b81561c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff3a8000c00 tx=0x7ff3a8004730 comp rx=0 tx=0).stop 2024-04-25T10:33:49.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.938+0000 7ff3bc8e7640 1 -- 172.21.15.106:0/211393658 shutdown_connections 2024-04-25T10:33:49.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.938+0000 7ff3bc8e7640 1 --2- 172.21.15.106:0/211393658 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7ff398074490 0x7ff3980769a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:49.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.938+0000 7ff3bc8e7640 1 --2- 172.21.15.106:0/211393658 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3b8153a80 0x7ff3b8154ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:49.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.938+0000 7ff3bc8e7640 1 --2- 172.21.15.106:0/211393658 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff3b80c8fe0 0x7ff3b8154760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:49.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.938+0000 7ff3bc8e7640 1 --2- 172.21.15.106:0/211393658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3b8074790 0x7ff3b81561c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:49.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.938+0000 7ff3bc8e7640 1 -- 172.21.15.106:0/211393658 >> 172.21.15.106:0/211393658 conn(0x7ff3b806c5c0 msgr2=0x7ff3b8073860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:49.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.938+0000 7ff3bc8e7640 1 -- 172.21.15.106:0/211393658 shutdown_connections 2024-04-25T10:33:49.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:49.938+0000 7ff3bc8e7640 1 -- 172.21.15.106:0/211393658 wait complete. 2024-04-25T10:33:51.467 INFO:tasks.workunit.client.0.smithi106.stdout: 2024-04-25T10:33:51.468 INFO:tasks.workunit.client.0.smithi106.stdout: Children see throughput for 1 initial writers = 709385.31 kB/sec 2024-04-25T10:33:51.468 INFO:tasks.workunit.client.0.smithi106.stdout: Parent sees throughput for 1 initial writers = 709375.48 kB/sec 2024-04-25T10:33:51.468 INFO:tasks.workunit.client.0.smithi106.stdout: Min throughput per process = 709385.31 kB/sec 2024-04-25T10:33:51.468 INFO:tasks.workunit.client.0.smithi106.stdout: Max throughput per process = 709385.31 kB/sec 2024-04-25T10:33:51.468 INFO:tasks.workunit.client.0.smithi106.stdout: Avg throughput per process = 709385.31 kB/sec 2024-04-25T10:33:51.468 INFO:tasks.workunit.client.0.smithi106.stdout: Min xfer = 10485760.00 kB 2024-04-25T10:33:52.957 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T10:33:53.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:53.074+0000 7fb9bb292640 1 Processor -- start 2024-04-25T10:33:53.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:53.074+0000 7fb9bb292640 1 -- start start 2024-04-25T10:33:53.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:53.077+0000 7fb9bb292640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb9b40744c0 0x7fb9b406f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:53.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:53.077+0000 7fb9bb292640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9b4068ad0 con 0x7fb9b4073020 2024-04-25T10:33:53.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:53.077+0000 7fb9bb292640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9b40c9010 con 0x7fb9b4068600 2024-04-25T10:33:53.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:53.077+0000 7fb9bb292640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb9b4071190 con 0x7fb9b40744c0 2024-04-25T10:33:53.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:53.077+0000 7fb9b882d640 1 -- >> v1:172.21.15.106:6789/0 conn(0x7fb9b4073020 legacy=0x7fb9b4073db0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.106:6789/0 2024-04-25T10:33:53.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:53.077+0000 7fb9b982f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb9b40744c0 0x7fb9b406f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:53.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:53.077+0000 7fb9b902e640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fb9b4068600 0x7fb9b4070a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.106:57326/0 (socket says 172.21.15.106:57326) 2024-04-25T10:33:53.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:53.077+0000 7fb9b902e640 1 -- 172.21.15.106:0/4254259836 learned_addr learned my addr 172.21.15.106:0/4254259836 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:33:53.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:53.077+0000 7fb9b982f640 1 -- 172.21.15.106:0/4254259836 >> v1:172.21.15.134:6789/0 conn(0x7fb9b4068600 legacy=0x7fb9b4070a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:53.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:53.077+0000 7fb9b982f640 1 -- 172.21.15.106:0/4254259836 >> v1:172.21.15.106:6789/0 conn(0x7fb9b4073020 legacy=0x7fb9b4073db0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:33:53.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:53.077+0000 7fb9b982f640 1 -- 172.21.15.106:0/4254259836 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9b40c92f0 con 0x7fb9b40744c0 2024-04-25T10:33:53.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:53.077+0000 7fb9b982f640 1 --2- 172.21.15.106:0/4254259836 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb9b40744c0 0x7fb9b406f780 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac0097d0 tx=0x7fb9ac02a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70fce13f905e56ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:53.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:53.077+0000 7fb9abfff640 1 -- 172.21.15.106:0/4254259836 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac037070 con 0x7fb9b40744c0 2024-04-25T10:33:53.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:53.077+0000 7fb9abfff640 1 -- 172.21.15.106:0/4254259836 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9ac032440 con 0x7fb9b40744c0 2024-04-25T10:33:53.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:53.077+0000 7fb9abfff640 1 -- 172.21.15.106:0/4254259836 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac032810 con 0x7fb9b40744c0 2024-04-25T10:33:53.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:53.078+0000 7fb9bb292640 1 -- 172.21.15.106:0/4254259836 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb9b40744c0 msgr2=0x7fb9b406f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:53.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:53.078+0000 7fb9bb292640 1 --2- 172.21.15.106:0/4254259836 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb9b40744c0 0x7fb9b406f780 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac0097d0 tx=0x7fb9ac02a1c0 comp rx=0 tx=0).stop 2024-04-25T10:33:53.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:53.078+0000 7fb9bb292640 1 -- 172.21.15.106:0/4254259836 shutdown_connections 2024-04-25T10:33:53.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:53.078+0000 7fb9bb292640 1 --2- 172.21.15.106:0/4254259836 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb9b40744c0 0x7fb9b406f780 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:53.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:53.078+0000 7fb9bb292640 1 -- 172.21.15.106:0/4254259836 >> 172.21.15.106:0/4254259836 conn(0x7fb9b40ce570 msgr2=0x7fb9b40ce980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:53.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:53.078+0000 7fb9bb292640 1 -- 172.21.15.106:0/4254259836 shutdown_connections 2024-04-25T10:33:53.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:53.078+0000 7fb9bb292640 1 -- 172.21.15.106:0/4254259836 wait complete. 2024-04-25T10:33:53.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:53.080+0000 7fb9bb292640 1 Processor -- start 2024-04-25T10:33:53.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:53.080+0000 7fb9bb292640 1 -- start start 2024-04-25T10:33:53.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:53.081+0000 7fb9bb292640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb9b4068600 0x7fb9b40c5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:53.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:53.081+0000 7fb9bb292640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb9b4073020 0x7fb9b414e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:53.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:53.081+0000 7fb9bb292640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9b40744c0 0x7fb9b4159c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:53.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:53.081+0000 7fb9bb292640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b40c9ce0 con 0x7fb9b4073020 2024-04-25T10:33:53.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:53.081+0000 7fb9bb292640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b406fcc0 con 0x7fb9b40744c0 2024-04-25T10:33:53.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:53.081+0000 7fb9bb292640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb9b4150ed0 con 0x7fb9b4068600 2024-04-25T10:33:53.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:53.081+0000 7fb9b882d640 1 -- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb9b4073020 msgr2=0x7fb9b414e340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.106:3300/0 2024-04-25T10:33:53.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:53.081+0000 7fb9b882d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb9b4073020 0x7fb9b414e340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:33:53.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:53.082+0000 7fb9b902e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb9b4068600 0x7fb9b40c5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:53.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:53.082+0000 7fb9b902e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb9b4068600 0x7fb9b40c5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.106:34556/0 (socket says 172.21.15.106:34556) 2024-04-25T10:33:53.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:53.082+0000 7fb9b902e640 1 -- 172.21.15.106:0/1654659072 learned_addr learned my addr 172.21.15.106:0/1654659072 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:33:53.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:53.082+0000 7fb9b982f640 1 --2- 172.21.15.106:0/1654659072 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9b40744c0 0x7fb9b4159c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:53.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:53.082+0000 7fb9b902e640 1 -- 172.21.15.106:0/1654659072 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9b40744c0 msgr2=0x7fb9b4159c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:53.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:53.082+0000 7fb9b902e640 1 --2- 172.21.15.106:0/1654659072 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9b40744c0 0x7fb9b4159c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:53.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:53.082+0000 7fb9b902e640 1 -- 172.21.15.106:0/1654659072 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb9b4073020 msgr2=0x7fb9b414e340 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:33:53.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:53.082+0000 7fb9b902e640 1 --2- 172.21.15.106:0/1654659072 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb9b4073020 0x7fb9b414e340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:53.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:53.082+0000 7fb9b902e640 1 -- 172.21.15.106:0/1654659072 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9b414e880 con 0x7fb9b4068600 2024-04-25T10:33:53.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:53.082+0000 7fb9b902e640 1 --2- 172.21.15.106:0/1654659072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb9b4068600 0x7fb9b40c5120 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb9b000caa0 tx=0x7fb9b000cf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:53.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:53.082+0000 7fb9a9ffb640 1 -- 172.21.15.106:0/1654659072 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b0017070 con 0x7fb9b4068600 2024-04-25T10:33:53.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:53.082+0000 7fb9a9ffb640 1 -- 172.21.15.106:0/1654659072 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9b000ac00 con 0x7fb9b4068600 2024-04-25T10:33:53.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:53.082+0000 7fb9bb292640 1 -- 172.21.15.106:0/1654659072 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9b40c3f40 con 0x7fb9b4068600 2024-04-25T10:33:53.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:53.082+0000 7fb9a9ffb640 1 -- 172.21.15.106:0/1654659072 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b0012400 con 0x7fb9b4068600 2024-04-25T10:33:53.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:53.083+0000 7fb9bb292640 1 -- 172.21.15.106:0/1654659072 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9b40c4500 con 0x7fb9b4068600 2024-04-25T10:33:53.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:53.083+0000 7fb9bb292640 1 -- 172.21.15.106:0/1654659072 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9b406a3d0 con 0x7fb9b4068600 2024-04-25T10:33:53.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:53.084+0000 7fb9a9ffb640 1 -- 172.21.15.106:0/1654659072 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb9b001a070 con 0x7fb9b4068600 2024-04-25T10:33:53.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:53.085+0000 7fb9a9ffb640 1 --2- 172.21.15.106:0/1654659072 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fb998074460 0x7fb998076970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:53.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:53.085+0000 7fb9a9ffb640 1 -- 172.21.15.106:0/1654659072 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fb9b0094f60 con 0x7fb9b4068600 2024-04-25T10:33:53.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:53.085+0000 7fb9b882d640 1 --2- 172.21.15.106:0/1654659072 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fb998074460 0x7fb998076970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:53.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:53.085+0000 7fb9b882d640 1 --2- 172.21.15.106:0/1654659072 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fb998074460 0x7fb998076970 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb9a4006fd0 tx=0x7fb9a4006d80 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:53.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:53.091+0000 7fb9a9ffb640 1 -- 172.21.15.106:0/1654659072 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb9b005f120 con 0x7fb9b4068600 2024-04-25T10:33:53.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:53.332+0000 7fb9bb292640 1 -- 172.21.15.106:0/1654659072 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fb9b4158600 con 0x7fb9b4068600 2024-04-25T10:33:53.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:53.333+0000 7fb9a9ffb640 1 -- 172.21.15.106:0/1654659072 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1569 (secure 0 0 0) 0x7fb9b0063420 con 0x7fb9b4068600 2024-04-25T10:33:53.334 INFO:teuthology.orchestra.run.smithi106.stdout:{"election_epoch":22,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":19,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"3ee77985-e8e4-41cf-81e9-7c774cf7891f","modified":"2024-04-25T10:29:33.446228Z","created":"2024-04-25T10:29:33.446228Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:3300","nonce":0},{"type":"v1","addr":"172.21.15.106:6789","nonce":0}]},"addr":"172.21.15.106:6789/0","public_addr":"172.21.15.106:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T10:33:53.337 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:53.335+0000 7fb9bb292640 1 -- 172.21.15.106:0/1654659072 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fb998074460 msgr2=0x7fb998076970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:53.337 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:53.335+0000 7fb9bb292640 1 --2- 172.21.15.106:0/1654659072 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fb998074460 0x7fb998076970 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb9a4006fd0 tx=0x7fb9a4006d80 comp rx=0 tx=0).stop 2024-04-25T10:33:53.337 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:53.335+0000 7fb9bb292640 1 -- 172.21.15.106:0/1654659072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb9b4068600 msgr2=0x7fb9b40c5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:53.337 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:53.335+0000 7fb9bb292640 1 --2- 172.21.15.106:0/1654659072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb9b4068600 0x7fb9b40c5120 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb9b000caa0 tx=0x7fb9b000cf60 comp rx=0 tx=0).stop 2024-04-25T10:33:53.337 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:53.336+0000 7fb9bb292640 1 -- 172.21.15.106:0/1654659072 shutdown_connections 2024-04-25T10:33:53.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:53.336+0000 7fb9bb292640 1 --2- 172.21.15.106:0/1654659072 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fb998074460 0x7fb998076970 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:53.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:53.336+0000 7fb9bb292640 1 --2- 172.21.15.106:0/1654659072 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9b40744c0 0x7fb9b4159c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:53.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:53.336+0000 7fb9bb292640 1 --2- 172.21.15.106:0/1654659072 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb9b4073020 0x7fb9b414e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:53.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:53.336+0000 7fb9bb292640 1 --2- 172.21.15.106:0/1654659072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb9b4068600 0x7fb9b40c5120 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:53.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:53.336+0000 7fb9bb292640 1 -- 172.21.15.106:0/1654659072 >> 172.21.15.106:0/1654659072 conn(0x7fb9b40ce570 msgr2=0x7fb9b40ce980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:53.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:53.336+0000 7fb9bb292640 1 -- 172.21.15.106:0/1654659072 shutdown_connections 2024-04-25T10:33:53.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:53.336+0000 7fb9bb292640 1 -- 172.21.15.106:0/1654659072 wait complete. 2024-04-25T10:33:56.349 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T10:33:56.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:56.520+0000 7f1636415640 1 Processor -- start 2024-04-25T10:33:56.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:56.521+0000 7f1636415640 1 -- start start 2024-04-25T10:33:56.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:56.523+0000 7f1636415640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f16300c7e30 0x7f16300c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:56.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:56.523+0000 7f1636415640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f1630156ef0 0x7f16300b4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:56.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:56.523+0000 7f1636415640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16300ce630 con 0x7f163006d000 2024-04-25T10:33:56.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:56.523+0000 7f1636415640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f16300c7c60 con 0x7f16300c7e30 2024-04-25T10:33:56.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:56.523+0000 7f1636415640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f16300c7980 con 0x7f1630156ef0 2024-04-25T10:33:56.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:56.523+0000 7f162ffff640 1 -- >> v2:172.21.15.106:3300/0 conn(0x7f16300c7e30 msgr2=0x7f16300c8d90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.106:3300/0 2024-04-25T10:33:56.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:56.523+0000 7f162ffff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f16300c7e30 0x7f16300c8d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:33:56.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:56.523+0000 7f162f7fe640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f1630156ef0 0x7f16300b4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:56.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:56.523+0000 7f162f7fe640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f1630156ef0 0x7f16300b4ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.106:51212/0 (socket says 172.21.15.106:51212) 2024-04-25T10:33:56.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:56.523+0000 7f162f7fe640 1 -- 172.21.15.106:0/233609859 learned_addr learned my addr 172.21.15.106:0/233609859 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:33:56.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:56.523+0000 7f162f7fe640 1 -- 172.21.15.106:0/233609859 >> v2:172.21.15.134:3300/0 conn(0x7f1630156ef0 msgr2=0x7f16300b4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T10:33:56.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:56.523+0000 7f162f7fe640 1 -- 172.21.15.106:0/233609859 >> v2:172.21.15.134:3300/0 conn(0x7f1630156ef0 msgr2=0x7f16300b4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:33:56.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:56.523+0000 7f162f7fe640 1 --2- 172.21.15.106:0/233609859 >> v2:172.21.15.134:3300/0 conn(0x7f1630156ef0 0x7f16300b4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:33:56.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:56.523+0000 7f162f7fe640 1 --2- 172.21.15.106:0/233609859 >> v2:172.21.15.134:3300/0 conn(0x7f1630156ef0 0x7f16300b4ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:33:56.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:56.724+0000 7f162ffff640 1 -- 172.21.15.106:0/233609859 >> v2:172.21.15.106:3300/0 conn(0x7f16300c7e30 msgr2=0x7f16300c8d90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.106:3300/0 2024-04-25T10:33:56.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:56.724+0000 7f162ffff640 1 --2- 172.21.15.106:0/233609859 >> v2:172.21.15.106:3300/0 conn(0x7f16300c7e30 0x7f16300c8d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T10:33:56.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:56.724+0000 7f162f7fe640 1 --2- 172.21.15.106:0/233609859 >> v2:172.21.15.134:3300/0 conn(0x7f1630156ef0 0x7f16300b4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:56.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:56.724+0000 7f162f7fe640 1 -- 172.21.15.106:0/233609859 >> v2:172.21.15.134:3300/0 conn(0x7f1630156ef0 msgr2=0x7f16300b4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T10:33:56.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:56.724+0000 7f162f7fe640 1 -- 172.21.15.106:0/233609859 >> v2:172.21.15.134:3300/0 conn(0x7f1630156ef0 msgr2=0x7f16300b4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:33:56.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:56.724+0000 7f162f7fe640 1 --2- 172.21.15.106:0/233609859 >> v2:172.21.15.134:3300/0 conn(0x7f1630156ef0 0x7f16300b4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:33:56.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:56.724+0000 7f162f7fe640 1 --2- 172.21.15.106:0/233609859 >> v2:172.21.15.134:3300/0 conn(0x7f1630156ef0 0x7f16300b4ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T10:33:57.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:57.125+0000 7f162ffff640 1 -- 172.21.15.106:0/233609859 >> v2:172.21.15.106:3300/0 conn(0x7f16300c7e30 msgr2=0x7f16300c8d90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.106:3300/0 2024-04-25T10:33:57.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:57.125+0000 7f162ffff640 1 --2- 172.21.15.106:0/233609859 >> v2:172.21.15.106:3300/0 conn(0x7f16300c7e30 0x7f16300c8d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T10:33:57.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:57.125+0000 7f162f7fe640 1 --2- 172.21.15.106:0/233609859 >> v2:172.21.15.134:3300/0 conn(0x7f1630156ef0 0x7f16300b4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:57.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:57.125+0000 7f162f7fe640 1 -- 172.21.15.106:0/233609859 >> v2:172.21.15.134:3300/0 conn(0x7f1630156ef0 msgr2=0x7f16300b4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T10:33:57.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:57.125+0000 7f162f7fe640 1 -- 172.21.15.106:0/233609859 >> v2:172.21.15.134:3300/0 conn(0x7f1630156ef0 msgr2=0x7f16300b4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:33:57.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:57.125+0000 7f162f7fe640 1 --2- 172.21.15.106:0/233609859 >> v2:172.21.15.134:3300/0 conn(0x7f1630156ef0 0x7f16300b4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:33:57.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:57.125+0000 7f162f7fe640 1 --2- 172.21.15.106:0/233609859 >> v2:172.21.15.134:3300/0 conn(0x7f1630156ef0 0x7f16300b4ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T10:33:57.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:57.926+0000 7f162ffff640 1 -- 172.21.15.106:0/233609859 >> v2:172.21.15.106:3300/0 conn(0x7f16300c7e30 msgr2=0x7f16300c8d90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.106:3300/0 2024-04-25T10:33:57.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:57.926+0000 7f162ffff640 1 --2- 172.21.15.106:0/233609859 >> v2:172.21.15.106:3300/0 conn(0x7f16300c7e30 0x7f16300c8d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T10:33:57.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:57.926+0000 7f162f7fe640 1 --2- 172.21.15.106:0/233609859 >> v2:172.21.15.134:3300/0 conn(0x7f1630156ef0 0x7f16300b4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:57.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:57.926+0000 7f162f7fe640 1 -- 172.21.15.106:0/233609859 >> v2:172.21.15.134:3300/0 conn(0x7f1630156ef0 msgr2=0x7f16300b4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T10:33:57.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:57.926+0000 7f162f7fe640 1 -- 172.21.15.106:0/233609859 >> v2:172.21.15.134:3300/0 conn(0x7f1630156ef0 msgr2=0x7f16300b4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:33:57.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:57.926+0000 7f162f7fe640 1 --2- 172.21.15.106:0/233609859 >> v2:172.21.15.134:3300/0 conn(0x7f1630156ef0 0x7f16300b4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:33:57.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:57.926+0000 7f162f7fe640 1 --2- 172.21.15.106:0/233609859 >> v2:172.21.15.134:3300/0 conn(0x7f1630156ef0 0x7f16300b4ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T10:33:59.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:59.523+0000 7f162dffb640 1 -- 172.21.15.106:0/233609859 >> v2:172.21.15.134:3300/0 conn(0x7f1630156ef0 msgr2=0x7f16300b4ce0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:33:59.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:59.523+0000 7f162dffb640 1 --2- 172.21.15.106:0/233609859 >> v2:172.21.15.134:3300/0 conn(0x7f1630156ef0 0x7f16300b4ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:59.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:59.523+0000 7f162dffb640 1 -- 172.21.15.106:0/233609859 >> v2:172.21.15.106:3300/0 conn(0x7f16300c7e30 msgr2=0x7f16300c8d90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:33:59.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:59.523+0000 7f162dffb640 1 --2- 172.21.15.106:0/233609859 >> v2:172.21.15.106:3300/0 conn(0x7f16300c7e30 0x7f16300c8d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:59.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:59.523+0000 7f162dffb640 1 -- 172.21.15.106:0/233609859 >> v1:172.21.15.134:6789/0 conn(0x7f163006d000 legacy=0x7f163015ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:59.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:59.523+0000 7f162dffb640 1 --2- 172.21.15.106:0/233609859 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f161c003690 0x7f161c005b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:59.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:59.523+0000 7f162dffb640 1 -- 172.21.15.106:0/233609859 --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16300c7010 con 0x7f161c0061c0 2024-04-25T10:33:59.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:59.523+0000 7f162dffb640 1 -- 172.21.15.106:0/233609859 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f16300c7c60 con 0x7f161c003690 2024-04-25T10:33:59.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:59.523+0000 7f162f7fe640 1 -- 172.21.15.106:0/233609859 >> v1:172.21.15.106:6789/0 conn(0x7f161c0061c0 legacy=0x7f161c0086b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.106:6789/0 2024-04-25T10:33:59.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:59.523+0000 7f162ffff640 1 --2- 172.21.15.106:0/233609859 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f161c003690 0x7f161c005b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:59.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:59.524+0000 7f162ffff640 1 -- 172.21.15.106:0/233609859 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f161c003690 msgr2=0x7f161c005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T10:33:59.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:59.524+0000 7f162ffff640 1 -- 172.21.15.106:0/233609859 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f161c003690 msgr2=0x7f161c005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:33:59.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:59.524+0000 7f162ffff640 1 --2- 172.21.15.106:0/233609859 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f161c003690 0x7f161c005b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:33:59.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:59.524+0000 7f162ffff640 1 --2- 172.21.15.106:0/233609859 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f161c003690 0x7f161c005b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:33:59.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:59.724+0000 7f162f7fe640 1 -- 172.21.15.106:0/233609859 >> v1:172.21.15.106:6789/0 conn(0x7f161c0061c0 legacy=0x7f161c0086b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.106:6789/0 2024-04-25T10:33:59.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:59.725+0000 7f162ffff640 1 --2- 172.21.15.106:0/233609859 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f161c003690 0x7f161c005b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:59.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:59.725+0000 7f162ffff640 1 -- 172.21.15.106:0/233609859 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f161c003690 msgr2=0x7f161c005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T10:33:59.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:59.725+0000 7f162ffff640 1 -- 172.21.15.106:0/233609859 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f161c003690 msgr2=0x7f161c005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:33:59.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:59.725+0000 7f162ffff640 1 --2- 172.21.15.106:0/233609859 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f161c003690 0x7f161c005b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:33:59.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:33:59.725+0000 7f162ffff640 1 --2- 172.21.15.106:0/233609859 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f161c003690 0x7f161c005b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T10:34:00.126 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:00.125+0000 7f162f7fe640 1 -- 172.21.15.106:0/233609859 >> v1:172.21.15.106:6789/0 conn(0x7f161c0061c0 legacy=0x7f161c0086b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.106:6789/0 2024-04-25T10:34:00.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:00.126+0000 7f162ffff640 1 --2- 172.21.15.106:0/233609859 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f161c003690 0x7f161c005b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:00.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:00.126+0000 7f162ffff640 1 -- 172.21.15.106:0/233609859 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f161c003690 msgr2=0x7f161c005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T10:34:00.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:00.126+0000 7f162ffff640 1 -- 172.21.15.106:0/233609859 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f161c003690 msgr2=0x7f161c005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:34:00.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:00.126+0000 7f162ffff640 1 --2- 172.21.15.106:0/233609859 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f161c003690 0x7f161c005b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:34:00.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:00.126+0000 7f162ffff640 1 --2- 172.21.15.106:0/233609859 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f161c003690 0x7f161c005b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T10:34:00.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:00.926+0000 7f162f7fe640 1 -- 172.21.15.106:0/233609859 >> v1:172.21.15.106:6789/0 conn(0x7f161c0061c0 legacy=0x7f161c0086b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.106:6789/0 2024-04-25T10:34:00.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:00.927+0000 7f162ffff640 1 --2- 172.21.15.106:0/233609859 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f161c003690 0x7f161c005b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:00.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:00.929+0000 7f162ffff640 1 -- 172.21.15.106:0/233609859 >> v1:172.21.15.106:6789/0 conn(0x7f161c0061c0 legacy=0x7f161c0086b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:34:00.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:00.929+0000 7f162ffff640 1 -- 172.21.15.106:0/233609859 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f161c00b0a0 con 0x7f161c003690 2024-04-25T10:34:00.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:00.931+0000 7f162ffff640 1 --2- 172.21.15.106:0/233609859 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f161c003690 0x7f161c005b30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1620009d40 tx=0x7f162002b180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47b87cbca329d427 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:00.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:00.931+0000 7f162effd640 1 -- 172.21.15.106:0/233609859 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1620031070 con 0x7f161c003690 2024-04-25T10:34:00.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:00.932+0000 7f162effd640 1 -- 172.21.15.106:0/233609859 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1620035410 con 0x7f161c003690 2024-04-25T10:34:00.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:00.932+0000 7f162effd640 1 -- 172.21.15.106:0/233609859 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1620035800 con 0x7f161c003690 2024-04-25T10:34:00.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:00.932+0000 7f1636415640 1 -- 172.21.15.106:0/233609859 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f161c003690 msgr2=0x7f161c005b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:00.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:00.932+0000 7f1636415640 1 --2- 172.21.15.106:0/233609859 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f161c003690 0x7f161c005b30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1620009d40 tx=0x7f162002b180 comp rx=0 tx=0).stop 2024-04-25T10:34:00.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:00.932+0000 7f16349b2640 1 -- 172.21.15.106:0/233609859 reap_dead start 2024-04-25T10:34:00.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:00.933+0000 7f1636415640 1 -- 172.21.15.106:0/233609859 shutdown_connections 2024-04-25T10:34:00.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:00.933+0000 7f1636415640 1 -- 172.21.15.106:0/233609859 >> 172.21.15.106:0/233609859 conn(0x7f1630068b10 msgr2=0x7f16300d1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:00.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:00.933+0000 7f1636415640 1 -- 172.21.15.106:0/233609859 shutdown_connections 2024-04-25T10:34:00.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:00.933+0000 7f1636415640 1 -- 172.21.15.106:0/233609859 wait complete. 2024-04-25T10:34:00.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:00.935+0000 7f1636415640 1 Processor -- start 2024-04-25T10:34:00.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:00.935+0000 7f1636415640 1 -- start start 2024-04-25T10:34:00.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:00.937+0000 7f1636415640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f161c003690 0x7f16301f6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:00.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:00.937+0000 7f1636415640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1630151e80 0x7f16300d0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:00.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:00.937+0000 7f1636415640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1630152290 0x7f16300d0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:00.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:00.937+0000 7f162f7fe640 1 -- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1630151e80 msgr2=0x7f16300d0a40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.106:3300/0 2024-04-25T10:34:00.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:00.937+0000 7f1636415640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f163015d6c0 con 0x7f1630151e80 2024-04-25T10:34:00.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:00.937+0000 7f1636415640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f163015d540 con 0x7f161c003690 2024-04-25T10:34:00.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:00.937+0000 7f1636415640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f16301f77d0 con 0x7f1630152290 2024-04-25T10:34:00.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:00.937+0000 7f162f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1630151e80 0x7f16300d0a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:34:00.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:00.937+0000 7f16349b2640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1630152290 0x7f16300d0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:00.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:00.937+0000 7f162ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f161c003690 0x7f16301f6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:00.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:00.937+0000 7f16349b2640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1630152290 0x7f16300d0f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.106:54992/0 (socket says 172.21.15.106:54992) 2024-04-25T10:34:00.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:00.937+0000 7f16349b2640 1 -- 172.21.15.106:0/2050335986 learned_addr learned my addr 172.21.15.106:0/2050335986 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:34:00.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:00.938+0000 7f16349b2640 1 -- 172.21.15.106:0/2050335986 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f161c003690 msgr2=0x7f16301f6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:00.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:00.938+0000 7f16349b2640 1 --2- 172.21.15.106:0/2050335986 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f161c003690 0x7f16301f6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:00.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:00.938+0000 7f16349b2640 1 -- 172.21.15.106:0/2050335986 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1630151e80 msgr2=0x7f16300d0a40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:34:00.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:00.938+0000 7f16349b2640 1 --2- 172.21.15.106:0/2050335986 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1630151e80 0x7f16300d0a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:00.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:00.938+0000 7f16349b2640 1 -- 172.21.15.106:0/2050335986 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16301f7300 con 0x7f1630152290 2024-04-25T10:34:00.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:00.938+0000 7f162ffff640 1 --2- 172.21.15.106:0/2050335986 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f161c003690 0x7f16301f6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:34:00.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:00.939+0000 7f16349b2640 1 --2- 172.21.15.106:0/2050335986 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1630152290 0x7f16300d0f80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f16300b5f90 tx=0x7f162800cb90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:00.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:00.939+0000 7f162cff9640 1 -- 172.21.15.106:0/2050335986 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1628017070 con 0x7f1630152290 2024-04-25T10:34:00.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:00.939+0000 7f162cff9640 1 -- 172.21.15.106:0/2050335986 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1628005d40 con 0x7f1630152290 2024-04-25T10:34:00.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:00.939+0000 7f162cff9640 1 -- 172.21.15.106:0/2050335986 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f162800a850 con 0x7f1630152290 2024-04-25T10:34:00.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:00.939+0000 7f1636415640 1 -- 172.21.15.106:0/2050335986 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16300d0320 con 0x7f1630152290 2024-04-25T10:34:00.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:00.939+0000 7f1636415640 1 -- 172.21.15.106:0/2050335986 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f163015bac0 con 0x7f1630152290 2024-04-25T10:34:00.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:00.941+0000 7f1636415640 1 -- 172.21.15.106:0/2050335986 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f163006a3d0 con 0x7f1630152290 2024-04-25T10:34:00.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:00.942+0000 7f162cff9640 1 -- 172.21.15.106:0/2050335986 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f162801a070 con 0x7f1630152290 2024-04-25T10:34:00.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:00.942+0000 7f162cff9640 1 --2- 172.21.15.106:0/2050335986 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f161c075290 0x7f161c0777a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:00.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:00.943+0000 7f162cff9640 1 -- 172.21.15.106:0/2050335986 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f1628094510 con 0x7f1630152290 2024-04-25T10:34:00.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:00.943+0000 7f162ffff640 1 --2- 172.21.15.106:0/2050335986 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f161c075290 0x7f161c0777a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:00.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:00.943+0000 7f162ffff640 1 --2- 172.21.15.106:0/2050335986 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f161c075290 0x7f161c0777a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1620035000 tx=0x7f1620031000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:00.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:00.951+0000 7f162cff9640 1 -- 172.21.15.106:0/2050335986 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f162805e6f0 con 0x7f1630152290 2024-04-25T10:34:01.337 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.336+0000 7f1636415640 1 -- 172.21.15.106:0/2050335986 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f163015a5a0 con 0x7f1630152290 2024-04-25T10:34:01.338 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.337+0000 7f162cff9640 1 -- 172.21.15.106:0/2050335986 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1562 (secure 0 0 0) 0x7f16280629f0 con 0x7f1630152290 2024-04-25T10:34:01.338 INFO:teuthology.orchestra.run.smithi106.stdout:{"election_epoch":24,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"3ee77985-e8e4-41cf-81e9-7c774cf7891f","modified":"2024-04-25T10:29:33.446228Z","created":"2024-04-25T10:29:33.446228Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:3300","nonce":0},{"type":"v1","addr":"172.21.15.106:6789","nonce":0}]},"addr":"172.21.15.106:6789/0","public_addr":"172.21.15.106:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T10:34:01.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.340+0000 7f1636415640 1 -- 172.21.15.106:0/2050335986 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f161c075290 msgr2=0x7f161c0777a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:01.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.340+0000 7f1636415640 1 --2- 172.21.15.106:0/2050335986 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f161c075290 0x7f161c0777a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1620035000 tx=0x7f1620031000 comp rx=0 tx=0).stop 2024-04-25T10:34:01.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.340+0000 7f1636415640 1 -- 172.21.15.106:0/2050335986 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1630152290 msgr2=0x7f16300d0f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:01.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.340+0000 7f1636415640 1 --2- 172.21.15.106:0/2050335986 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1630152290 0x7f16300d0f80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f16300b5f90 tx=0x7f162800cb90 comp rx=0 tx=0).stop 2024-04-25T10:34:01.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.340+0000 7f1636415640 1 -- 172.21.15.106:0/2050335986 shutdown_connections 2024-04-25T10:34:01.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.340+0000 7f1636415640 1 --2- 172.21.15.106:0/2050335986 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f161c075290 0x7f161c0777a0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:01.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.340+0000 7f1636415640 1 --2- 172.21.15.106:0/2050335986 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1630152290 0x7f16300d0f80 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:01.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.340+0000 7f1636415640 1 --2- 172.21.15.106:0/2050335986 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1630151e80 0x7f16300d0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:01.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.340+0000 7f1636415640 1 --2- 172.21.15.106:0/2050335986 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f161c003690 0x7f16301f6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:01.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.340+0000 7f1636415640 1 -- 172.21.15.106:0/2050335986 >> 172.21.15.106:0/2050335986 conn(0x7f1630068b10 msgr2=0x7f16300d1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:01.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.340+0000 7f1636415640 1 -- 172.21.15.106:0/2050335986 shutdown_connections 2024-04-25T10:34:01.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.340+0000 7f1636415640 1 -- 172.21.15.106:0/2050335986 wait complete. 2024-04-25T10:34:01.358 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-04-25T10:34:01.358 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-04-25T10:34:01.358 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-25T10:34:01.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.528+0000 7ff923ec6640 1 Processor -- start 2024-04-25T10:34:01.530 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.528+0000 7ff923ec6640 1 -- start start 2024-04-25T10:34:01.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.530+0000 7ff923ec6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff91c073870 0x7ff91c0c7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:01.532 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.530+0000 7ff923ec6640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff91c15dd00 con 0x7ff91c0c7e40 2024-04-25T10:34:01.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.531+0000 7ff923ec6640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff91c0ca390 con 0x7ff91c0c9820 2024-04-25T10:34:01.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.531+0000 7ff923ec6640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff91c15d870 con 0x7ff91c073870 2024-04-25T10:34:01.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.531+0000 7ff922463640 1 -- >> v1:172.21.15.106:6789/0 conn(0x7ff91c0c7e40 legacy=0x7ff91c0cf030 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.106:6789/0 2024-04-25T10:34:01.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.531+0000 7ff921461640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff91c073870 0x7ff91c0c7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:01.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.531+0000 7ff921c62640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7ff91c0c9820 0x7ff91c156f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.106:47582/0 (socket says 172.21.15.106:47582) 2024-04-25T10:34:01.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.531+0000 7ff921c62640 1 -- 172.21.15.106:0/687216055 learned_addr learned my addr 172.21.15.106:0/687216055 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:34:01.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.531+0000 7ff920c60640 1 -- 172.21.15.106:0/687216055 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 122885858 0 0) 0x7ff91c0ca390 con 0x7ff91c0c9820 2024-04-25T10:34:01.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.531+0000 7ff920c60640 1 -- 172.21.15.106:0/687216055 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff908003520 con 0x7ff91c0c9820 2024-04-25T10:34:01.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.531+0000 7ff921461640 1 -- 172.21.15.106:0/687216055 >> v1:172.21.15.134:6789/0 conn(0x7ff91c0c9820 legacy=0x7ff91c156f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:01.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.531+0000 7ff921461640 1 -- 172.21.15.106:0/687216055 >> v1:172.21.15.106:6789/0 conn(0x7ff91c0c7e40 legacy=0x7ff91c0cf030 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:34:01.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.531+0000 7ff921461640 1 -- 172.21.15.106:0/687216055 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff91c15cf40 con 0x7ff91c073870 2024-04-25T10:34:01.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.532+0000 7ff921461640 1 --2- 172.21.15.106:0/687216055 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff91c073870 0x7ff91c0c7900 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff9180097d0 tx=0x7ff91802a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6de6993cc513f9f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:01.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.533+0000 7ff920c60640 1 -- 172.21.15.106:0/687216055 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff918037070 con 0x7ff91c073870 2024-04-25T10:34:01.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.533+0000 7ff920c60640 1 -- 172.21.15.106:0/687216055 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff918032440 con 0x7ff91c073870 2024-04-25T10:34:01.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.533+0000 7ff920c60640 1 -- 172.21.15.106:0/687216055 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff918032810 con 0x7ff91c073870 2024-04-25T10:34:01.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.534+0000 7ff923ec6640 1 -- 172.21.15.106:0/687216055 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff91c073870 msgr2=0x7ff91c0c7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:01.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.534+0000 7ff923ec6640 1 --2- 172.21.15.106:0/687216055 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff91c073870 0x7ff91c0c7900 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff9180097d0 tx=0x7ff91802a250 comp rx=0 tx=0).stop 2024-04-25T10:34:01.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.534+0000 7ff923ec6640 1 -- 172.21.15.106:0/687216055 shutdown_connections 2024-04-25T10:34:01.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.534+0000 7ff923ec6640 1 --2- 172.21.15.106:0/687216055 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff91c073870 0x7ff91c0c7900 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:01.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.534+0000 7ff923ec6640 1 -- 172.21.15.106:0/687216055 >> 172.21.15.106:0/687216055 conn(0x7ff91c06d8b0 msgr2=0x7ff91c06dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:01.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.534+0000 7ff923ec6640 1 -- 172.21.15.106:0/687216055 shutdown_connections 2024-04-25T10:34:01.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.534+0000 7ff923ec6640 1 -- 172.21.15.106:0/687216055 wait complete. 2024-04-25T10:34:01.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.537+0000 7ff923ec6640 1 Processor -- start 2024-04-25T10:34:01.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.538+0000 7ff923ec6640 1 -- start start 2024-04-25T10:34:01.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.540+0000 7ff923ec6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff91c073870 0x7ff91c14f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:01.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.540+0000 7ff923ec6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff91c0c7e40 0x7ff91c0cd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:01.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.540+0000 7ff923ec6640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff91c0c9820 0x7ff91c15a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:01.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.540+0000 7ff921c62640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff91c073870 0x7ff91c14f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:01.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.540+0000 7ff922463640 1 -- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff91c0c9820 msgr2=0x7ff91c15a020 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.106:3300/0 2024-04-25T10:34:01.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.540+0000 7ff921c62640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff91c073870 0x7ff91c14f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.106:51248/0 (socket says 172.21.15.106:51248) 2024-04-25T10:34:01.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.540+0000 7ff922463640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff91c0c9820 0x7ff91c15a020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:34:01.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.540+0000 7ff921461640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff91c0c7e40 0x7ff91c0cd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:01.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.540+0000 7ff921461640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff91c0c7e40 0x7ff91c0cd9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.106:55002/0 (socket says 172.21.15.106:55002) 2024-04-25T10:34:01.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.540+0000 7ff923ec6640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff91c0c7190 con 0x7ff91c0c9820 2024-04-25T10:34:01.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.540+0000 7ff923ec6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff91c15d500 con 0x7ff91c073870 2024-04-25T10:34:01.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.540+0000 7ff923ec6640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff91c0747b0 con 0x7ff91c0c7e40 2024-04-25T10:34:01.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.541+0000 7ff921c62640 1 -- 172.21.15.106:0/2862024924 learned_addr learned my addr 172.21.15.106:0/2862024924 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:34:01.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.541+0000 7ff921461640 1 -- 172.21.15.106:0/2862024924 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff91c073870 msgr2=0x7ff91c14f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:01.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.541+0000 7ff921461640 1 --2- 172.21.15.106:0/2862024924 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff91c073870 0x7ff91c14f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:01.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.541+0000 7ff921461640 1 -- 172.21.15.106:0/2862024924 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff91c0c9820 msgr2=0x7ff91c15a020 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:34:01.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.541+0000 7ff921461640 1 --2- 172.21.15.106:0/2862024924 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff91c0c9820 0x7ff91c15a020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:01.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.541+0000 7ff921461640 1 -- 172.21.15.106:0/2862024924 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff91c14fb10 con 0x7ff91c0c7e40 2024-04-25T10:34:01.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.541+0000 7ff921c62640 1 --2- 172.21.15.106:0/2862024924 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff91c073870 0x7ff91c14f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:34:01.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.541+0000 7ff921461640 1 --2- 172.21.15.106:0/2862024924 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff91c0c7e40 0x7ff91c0cd9d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff918031000 tx=0x7ff9180046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:01.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.542+0000 7ff9127fc640 1 -- 172.21.15.106:0/2862024924 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff918041070 con 0x7ff91c0c7e40 2024-04-25T10:34:01.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.542+0000 7ff9127fc640 1 -- 172.21.15.106:0/2862024924 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff91803c8b0 con 0x7ff91c0c7e40 2024-04-25T10:34:01.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.542+0000 7ff9127fc640 1 -- 172.21.15.106:0/2862024924 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff918037040 con 0x7ff91c0c7e40 2024-04-25T10:34:01.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.542+0000 7ff923ec6640 1 -- 172.21.15.106:0/2862024924 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff91c1f2dd0 con 0x7ff91c0c7e40 2024-04-25T10:34:01.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.542+0000 7ff923ec6640 1 -- 172.21.15.106:0/2862024924 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff91c1f3330 con 0x7ff91c0c7e40 2024-04-25T10:34:01.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.544+0000 7ff923ec6640 1 --2- 172.21.15.106:0/2862024924 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff91c0cc4f0 0x7ff91c1f3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:01.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.544+0000 7ff923ec6640 1 -- 172.21.15.106:0/2862024924 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff91c14ffe0 con 0x7ff91c0cc4f0 2024-04-25T10:34:01.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.544+0000 7ff921c62640 1 --2- 172.21.15.106:0/2862024924 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff91c0cc4f0 0x7ff91c1f3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:01.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.544+0000 7ff9127fc640 1 -- 172.21.15.106:0/2862024924 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff918044070 con 0x7ff91c0c7e40 2024-04-25T10:34:01.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.545+0000 7ff921c62640 1 -- 172.21.15.106:0/2862024924 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff91c06a3d0 con 0x7ff91c0cc4f0 2024-04-25T10:34:01.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.545+0000 7ff9127fc640 1 --2- 172.21.15.106:0/2862024924 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7ff900074460 0x7ff900076970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:01.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.545+0000 7ff9127fc640 1 -- 172.21.15.106:0/2862024924 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7ff9180b5f10 con 0x7ff91c0c7e40 2024-04-25T10:34:01.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.545+0000 7ff922463640 1 --2- 172.21.15.106:0/2862024924 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7ff900074460 0x7ff900076970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:01.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.546+0000 7ff921c62640 1 --2- 172.21.15.106:0/2862024924 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff91c0cc4f0 0x7ff91c1f3610 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff90c00fae0 tx=0x7ff90c001d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:01.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.546+0000 7ff9127fc640 1 -- 172.21.15.106:0/2862024924 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff90c005ac0 con 0x7ff91c0cc4f0 2024-04-25T10:34:01.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.546+0000 7ff922463640 1 --2- 172.21.15.106:0/2862024924 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7ff900074460 0x7ff900076970 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff91c155340 tx=0x7ff914009270 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:01.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.546+0000 7ff9127fc640 1 -- 172.21.15.106:0/2862024924 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7ff90c018070 con 0x7ff91c0cc4f0 2024-04-25T10:34:01.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.546+0000 7ff9127fc640 1 -- 172.21.15.106:0/2862024924 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff91c0cc4f0 msgr2=0x7ff91c1f3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:01.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.546+0000 7ff9127fc640 1 --2- 172.21.15.106:0/2862024924 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff91c0cc4f0 0x7ff91c1f3610 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff90c00fae0 tx=0x7ff90c001d60 comp rx=0 tx=0).stop 2024-04-25T10:34:01.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.546+0000 7ff9127fc640 1 -- 172.21.15.106:0/2862024924 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff91c0cc4f0 msgr2=0x7ff91c1f3610 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T10:34:01.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.546+0000 7ff9127fc640 1 --2- 172.21.15.106:0/2862024924 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff91c0cc4f0 0x7ff91c1f3610 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:01.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.554+0000 7ff923ec6640 1 --2- 172.21.15.106:0/2862024924 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff91c15a560 0x7ff91c14c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:01.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.554+0000 7ff923ec6640 1 -- 172.21.15.106:0/2862024924 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff91c06de40 con 0x7ff91c15a560 2024-04-25T10:34:01.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.554+0000 7ff921c62640 1 --2- 172.21.15.106:0/2862024924 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff91c15a560 0x7ff91c14c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:01.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.555+0000 7ff921c62640 1 -- 172.21.15.106:0/2862024924 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff91c0cc000 con 0x7ff91c15a560 2024-04-25T10:34:01.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.555+0000 7ff921c62640 1 --2- 172.21.15.106:0/2862024924 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff91c15a560 0x7ff91c14c730 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff90c017000 tx=0x7ff90c000ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:01.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.555+0000 7ff9127fc640 1 -- 172.21.15.106:0/2862024924 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff90c01da50 con 0x7ff91c15a560 2024-04-25T10:34:01.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.556+0000 7ff9127fc640 1 -- 172.21.15.106:0/2862024924 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5059 (secure 0 0 0) 0x7ff90c00d7c0 con 0x7ff91c15a560 2024-04-25T10:34:01.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.556+0000 7ff9127fc640 1 -- 172.21.15.106:0/2862024924 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff91c15a560 msgr2=0x7ff91c14c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:01.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.556+0000 7ff9127fc640 1 --2- 172.21.15.106:0/2862024924 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff91c15a560 0x7ff91c14c730 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff90c017000 tx=0x7ff90c000ed0 comp rx=0 tx=0).stop 2024-04-25T10:34:01.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.556+0000 7ff9127fc640 1 -- 172.21.15.106:0/2862024924 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff91c15a560 msgr2=0x7ff91c14c730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T10:34:01.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.556+0000 7ff9127fc640 1 --2- 172.21.15.106:0/2862024924 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff91c15a560 0x7ff91c14c730 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:01.558 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-25T10:34:01.558 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "b", 2024-04-25T10:34:01.558 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 1, 2024-04-25T10:34:01.558 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "leader", 2024-04-25T10:34:01.558 INFO:teuthology.orchestra.run.smithi106.stdout: "election_epoch": 24, 2024-04-25T10:34:01.559 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 28080, 2024-04-25T10:34:01.559 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum": [ 2024-04-25T10:34:01.559 INFO:teuthology.orchestra.run.smithi106.stdout: 1, 2024-04-25T10:34:01.559 INFO:teuthology.orchestra.run.smithi106.stdout: 2 2024-04-25T10:34:01.559 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:34:01.559 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_age": 1, 2024-04-25T10:34:01.559 INFO:teuthology.orchestra.run.smithi106.stdout: "features": { 2024-04-25T10:34:01.559 INFO:teuthology.orchestra.run.smithi106.stdout: "required_con": "2450521708007866372", 2024-04-25T10:34:01.559 INFO:teuthology.orchestra.run.smithi106.stdout: "required_mon": [ 2024-04-25T10:34:01.559 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:34:01.559 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:34:01.559 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:34:01.559 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:34:01.559 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:34:01.559 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:34:01.560 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:34:01.560 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:34:01.560 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:34:01.560 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:34:01.560 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:34:01.560 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:34:01.560 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_con": "4540701547738038271", 2024-04-25T10:34:01.560 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_mon": [ 2024-04-25T10:34:01.560 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:34:01.560 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:34:01.560 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:34:01.560 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:34:01.560 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:34:01.561 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:34:01.561 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:34:01.561 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:34:01.561 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:34:01.561 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:34:01.561 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:34:01.561 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:34:01.561 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:34:01.561 INFO:teuthology.orchestra.run.smithi106.stdout: "outside_quorum": [], 2024-04-25T10:34:01.561 INFO:teuthology.orchestra.run.smithi106.stdout: "extra_probe_peers": [], 2024-04-25T10:34:01.561 INFO:teuthology.orchestra.run.smithi106.stdout: "sync_provider": [], 2024-04-25T10:34:01.561 INFO:teuthology.orchestra.run.smithi106.stdout: "monmap": { 2024-04-25T10:34:01.561 INFO:teuthology.orchestra.run.smithi106.stdout: "epoch": 1, 2024-04-25T10:34:01.561 INFO:teuthology.orchestra.run.smithi106.stdout: "fsid": "3ee77985-e8e4-41cf-81e9-7c774cf7891f", 2024-04-25T10:34:01.562 INFO:teuthology.orchestra.run.smithi106.stdout: "modified": "2024-04-25T10:29:33.446228Z", 2024-04-25T10:34:01.562 INFO:teuthology.orchestra.run.smithi106.stdout: "created": "2024-04-25T10:29:33.446228Z", 2024-04-25T10:34:01.562 INFO:teuthology.orchestra.run.smithi106.stdout: "min_mon_release": 19, 2024-04-25T10:34:01.562 INFO:teuthology.orchestra.run.smithi106.stdout: "min_mon_release_name": "squid", 2024-04-25T10:34:01.562 INFO:teuthology.orchestra.run.smithi106.stdout: "election_strategy": 1, 2024-04-25T10:34:01.562 INFO:teuthology.orchestra.run.smithi106.stdout: "disallowed_leaders: ": "", 2024-04-25T10:34:01.562 INFO:teuthology.orchestra.run.smithi106.stdout: "stretch_mode": false, 2024-04-25T10:34:01.562 INFO:teuthology.orchestra.run.smithi106.stdout: "tiebreaker_mon": "", 2024-04-25T10:34:01.562 INFO:teuthology.orchestra.run.smithi106.stdout: "removed_ranks: ": "", 2024-04-25T10:34:01.562 INFO:teuthology.orchestra.run.smithi106.stdout: "features": { 2024-04-25T10:34:01.562 INFO:teuthology.orchestra.run.smithi106.stdout: "persistent": [ 2024-04-25T10:34:01.562 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:34:01.562 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:34:01.563 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:34:01.563 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:34:01.563 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:34:01.563 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:34:01.563 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:34:01.563 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:34:01.563 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:34:01.563 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:34:01.563 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:34:01.563 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:34:01.563 INFO:teuthology.orchestra.run.smithi106.stdout: "optional": [] 2024-04-25T10:34:01.563 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:34:01.563 INFO:teuthology.orchestra.run.smithi106.stdout: "mons": [ 2024-04-25T10:34:01.563 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:34:01.564 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 0, 2024-04-25T10:34:01.564 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "a", 2024-04-25T10:34:01.564 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:34:01.564 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:34:01.564 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:34:01.564 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:34:01.564 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:3300", 2024-04-25T10:34:01.564 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:34:01.564 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:34:01.564 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:34:01.564 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:34:01.564 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:6789", 2024-04-25T10:34:01.564 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:34:01.564 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:34:01.564 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:34:01.565 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:34:01.565 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:6789/0", 2024-04-25T10:34:01.565 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.106:6789/0", 2024-04-25T10:34:01.565 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:34:01.565 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:34:01.565 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:34:01.565 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:34:01.565 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:34:01.565 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 1, 2024-04-25T10:34:01.565 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "b", 2024-04-25T10:34:01.565 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:34:01.565 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:34:01.565 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:34:01.565 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:34:01.566 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:3300", 2024-04-25T10:34:01.566 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:34:01.566 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:34:01.566 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:34:01.566 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:34:01.566 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6789", 2024-04-25T10:34:01.566 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:34:01.566 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:34:01.566 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:34:01.566 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:34:01.566 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6789/0", 2024-04-25T10:34:01.566 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.134:6789/0", 2024-04-25T10:34:01.566 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:34:01.567 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:34:01.567 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:34:01.567 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:34:01.567 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:34:01.567 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 2, 2024-04-25T10:34:01.567 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "c", 2024-04-25T10:34:01.567 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:34:01.567 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:34:01.567 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:34:01.567 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:34:01.567 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:3301", 2024-04-25T10:34:01.567 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:34:01.567 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:34:01.567 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:34:01.567 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:34:01.568 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6790", 2024-04-25T10:34:01.568 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:34:01.568 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:34:01.568 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:34:01.568 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:34:01.568 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6790/0", 2024-04-25T10:34:01.568 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.134:6790/0", 2024-04-25T10:34:01.568 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:34:01.568 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:34:01.568 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:34:01.568 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:34:01.568 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:34:01.568 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:34:01.568 INFO:teuthology.orchestra.run.smithi106.stdout: "feature_map": { 2024-04-25T10:34:01.569 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": [ 2024-04-25T10:34:01.569 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:34:01.569 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:34:01.569 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:34:01.569 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 1 2024-04-25T10:34:01.569 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:34:01.569 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:34:01.569 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": [ 2024-04-25T10:34:01.569 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:34:01.569 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:34:01.569 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:34:01.569 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 2 2024-04-25T10:34:01.569 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:34:01.569 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:34:01.570 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": [ 2024-04-25T10:34:01.570 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:34:01.570 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:34:01.570 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:34:01.570 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 3 2024-04-25T10:34:01.570 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:34:01.570 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:34:01.570 INFO:teuthology.orchestra.run.smithi106.stdout: "client": [ 2024-04-25T10:34:01.570 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:34:01.570 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:34:01.570 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:34:01.570 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 3 2024-04-25T10:34:01.570 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:34:01.571 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:34:01.571 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": [ 2024-04-25T10:34:01.571 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:34:01.571 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:34:01.571 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:34:01.571 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 1 2024-04-25T10:34:01.571 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:34:01.571 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:34:01.571 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:34:01.571 INFO:teuthology.orchestra.run.smithi106.stdout: "stretch_mode": false 2024-04-25T10:34:01.571 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-25T10:34:01.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.557+0000 7ff923ec6640 1 -- 172.21.15.106:0/2862024924 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7ff900074460 msgr2=0x7ff900076970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:01.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.557+0000 7ff923ec6640 1 --2- 172.21.15.106:0/2862024924 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7ff900074460 0x7ff900076970 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff91c155340 tx=0x7ff914009270 comp rx=0 tx=0).stop 2024-04-25T10:34:01.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.557+0000 7ff923ec6640 1 -- 172.21.15.106:0/2862024924 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff91c0c7e40 msgr2=0x7ff91c0cd9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:01.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.557+0000 7ff923ec6640 1 --2- 172.21.15.106:0/2862024924 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff91c0c7e40 0x7ff91c0cd9d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff918031000 tx=0x7ff9180046c0 comp rx=0 tx=0).stop 2024-04-25T10:34:01.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.557+0000 7ff922463640 1 -- 172.21.15.106:0/2862024924 reap_dead start 2024-04-25T10:34:01.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.557+0000 7ff922463640 1 -- 172.21.15.106:0/2862024924 reap_dead start 2024-04-25T10:34:01.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.557+0000 7ff923ec6640 1 -- 172.21.15.106:0/2862024924 shutdown_connections 2024-04-25T10:34:01.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.557+0000 7ff923ec6640 1 -- 172.21.15.106:0/2862024924 >> 172.21.15.106:0/2862024924 conn(0x7ff91c06d8b0 msgr2=0x7ff91c0caf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:01.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.557+0000 7ff923ec6640 1 -- 172.21.15.106:0/2862024924 shutdown_connections 2024-04-25T10:34:01.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.557+0000 7ff923ec6640 1 -- 172.21.15.106:0/2862024924 wait complete. 2024-04-25T10:34:01.578 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-04-25T10:34:01.578 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-04-25T10:34:01.578 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-25T10:34:01.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.752+0000 7f121e7b2640 1 Processor -- start 2024-04-25T10:34:01.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.752+0000 7f121e7b2640 1 -- start start 2024-04-25T10:34:01.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.754+0000 7f121e7b2640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12180c8d90 0x7f1218156ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:01.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.754+0000 7f121e7b2640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f12180b4ce0 0x7f121815ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:01.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.754+0000 7f121e7b2640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12180ce630 con 0x7f12180c7e30 2024-04-25T10:34:01.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.754+0000 7f121e7b2640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f12180b50f0 con 0x7f12180b4ce0 2024-04-25T10:34:01.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.754+0000 7f121e7b2640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f12180c7c60 con 0x7f12180c8d90 2024-04-25T10:34:01.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.755+0000 7f1217fff640 1 -- >> v1:172.21.15.106:6789/0 conn(0x7f12180c7e30 legacy=0x7f121806d000 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.106:6789/0 2024-04-25T10:34:01.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.755+0000 7f120ffff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12180c8d90 0x7f1218156ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:01.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.755+0000 7f120ffff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12180c8d90 0x7f1218156ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.106:55012/0 (socket says 172.21.15.106:55012) 2024-04-25T10:34:01.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.755+0000 7f120ffff640 1 -- 172.21.15.106:0/3370304030 learned_addr learned my addr 172.21.15.106:0/3370304030 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:34:01.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.755+0000 7f121cd4f640 1 --2- 172.21.15.106:0/3370304030 >> v2:172.21.15.134:3300/0 conn(0x7f12180b4ce0 0x7f121815ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:01.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.755+0000 7f120ffff640 1 -- 172.21.15.106:0/3370304030 >> v2:172.21.15.134:3300/0 conn(0x7f12180b4ce0 msgr2=0x7f121815ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:01.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.755+0000 7f120ffff640 1 --2- 172.21.15.106:0/3370304030 >> v2:172.21.15.134:3300/0 conn(0x7f12180b4ce0 0x7f121815ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:01.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.755+0000 7f120ffff640 1 -- 172.21.15.106:0/3370304030 >> v1:172.21.15.106:6789/0 conn(0x7f12180c7e30 legacy=0x7f121806d000 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:34:01.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.755+0000 7f120ffff640 1 -- 172.21.15.106:0/3370304030 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12180c7980 con 0x7f12180c8d90 2024-04-25T10:34:01.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.757+0000 7f120ffff640 1 --2- 172.21.15.106:0/3370304030 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12180c8d90 0x7f1218156ef0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f12180c7de0 tx=0x7f120002a140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fa839bedc2e2265 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:01.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.757+0000 7f12177fe640 1 -- 172.21.15.106:0/3370304030 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120002ae50 con 0x7f12180c8d90 2024-04-25T10:34:01.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.757+0000 7f12177fe640 1 -- 172.21.15.106:0/3370304030 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1200004030 con 0x7f12180c8d90 2024-04-25T10:34:01.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.757+0000 7f12177fe640 1 -- 172.21.15.106:0/3370304030 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1200004460 con 0x7f12180c8d90 2024-04-25T10:34:01.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.758+0000 7f121e7b2640 1 -- 172.21.15.106:0/3370304030 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12180c8d90 msgr2=0x7f1218156ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:01.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.758+0000 7f121e7b2640 1 --2- 172.21.15.106:0/3370304030 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12180c8d90 0x7f1218156ef0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f12180c7de0 tx=0x7f120002a140 comp rx=0 tx=0).stop 2024-04-25T10:34:01.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.759+0000 7f121e7b2640 1 -- 172.21.15.106:0/3370304030 shutdown_connections 2024-04-25T10:34:01.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.759+0000 7f121e7b2640 1 --2- 172.21.15.106:0/3370304030 >> v2:172.21.15.134:3300/0 conn(0x7f12180b4ce0 0x7f121815ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:01.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.759+0000 7f121e7b2640 1 --2- 172.21.15.106:0/3370304030 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12180c8d90 0x7f1218156ef0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:01.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.759+0000 7f121e7b2640 1 -- 172.21.15.106:0/3370304030 >> 172.21.15.106:0/3370304030 conn(0x7f1218068b10 msgr2=0x7f12180d1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:01.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.759+0000 7f121e7b2640 1 -- 172.21.15.106:0/3370304030 shutdown_connections 2024-04-25T10:34:01.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.759+0000 7f121e7b2640 1 -- 172.21.15.106:0/3370304030 wait complete. 2024-04-25T10:34:01.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.761+0000 7f121e7b2640 1 Processor -- start 2024-04-25T10:34:01.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.761+0000 7f121e7b2640 1 -- start start 2024-04-25T10:34:01.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.762+0000 7f121e7b2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12180b4ce0 0x7f12181f37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:01.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.762+0000 7f121e7b2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f12180c7e30 0x7f12180d1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:01.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.762+0000 7f121e7b2640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12180c8d90 0x7f121814b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:01.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.762+0000 7f121e7b2640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f121806d710 con 0x7f12180c7e30 2024-04-25T10:34:01.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.762+0000 7f121e7b2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f12180c72f0 con 0x7f12180b4ce0 2024-04-25T10:34:01.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.762+0000 7f121e7b2640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f121806d890 con 0x7f12180c8d90 2024-04-25T10:34:01.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.762+0000 7f120ffff640 1 -- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f12180c7e30 msgr2=0x7f12180d1120 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.106:3300/0 2024-04-25T10:34:01.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.762+0000 7f120ffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f12180c7e30 0x7f12180d1120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:34:01.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.762+0000 7f1217fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12180b4ce0 0x7f12181f37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:01.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.762+0000 7f1217fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12180b4ce0 0x7f12181f37d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.106:51284/0 (socket says 172.21.15.106:51284) 2024-04-25T10:34:01.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.762+0000 7f1217fff640 1 -- 172.21.15.106:0/2516228899 learned_addr learned my addr 172.21.15.106:0/2516228899 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:34:01.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.762+0000 7f121cd4f640 1 --2- 172.21.15.106:0/2516228899 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12180c8d90 0x7f121814b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:01.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.763+0000 7f121cd4f640 1 -- 172.21.15.106:0/2516228899 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12180b4ce0 msgr2=0x7f12181f37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:01.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.763+0000 7f121cd4f640 1 --2- 172.21.15.106:0/2516228899 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12180b4ce0 0x7f12181f37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:01.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.763+0000 7f121cd4f640 1 -- 172.21.15.106:0/2516228899 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f12180c7e30 msgr2=0x7f12180d1120 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:34:01.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.763+0000 7f121cd4f640 1 --2- 172.21.15.106:0/2516228899 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f12180c7e30 0x7f12180d1120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:01.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.763+0000 7f121cd4f640 1 -- 172.21.15.106:0/2516228899 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12181f6d40 con 0x7f12180c8d90 2024-04-25T10:34:01.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.763+0000 7f1217fff640 1 --2- 172.21.15.106:0/2516228899 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12180b4ce0 0x7f12181f37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:34:01.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.763+0000 7f121cd4f640 1 --2- 172.21.15.106:0/2516228899 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12180c8d90 0x7f121814b910 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1218090000 tx=0x7f121000c0c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:01.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.764+0000 7f12157fa640 1 -- 172.21.15.106:0/2516228899 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1210018020 con 0x7f12180c8d90 2024-04-25T10:34:01.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.764+0000 7f12157fa640 1 -- 172.21.15.106:0/2516228899 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f121000cc80 con 0x7f12180c8d90 2024-04-25T10:34:01.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.764+0000 7f12157fa640 1 -- 172.21.15.106:0/2516228899 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12100146c0 con 0x7f12180c8d90 2024-04-25T10:34:01.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.764+0000 7f121e7b2640 1 -- 172.21.15.106:0/2516228899 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12181f7020 con 0x7f12180c8d90 2024-04-25T10:34:01.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.764+0000 7f121e7b2640 1 -- 172.21.15.106:0/2516228899 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f12181f75e0 con 0x7f12180c8d90 2024-04-25T10:34:01.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.765+0000 7f121e7b2640 1 --2- 172.21.15.106:0/2516228899 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f121814fe20 0x7f121814be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:01.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.765+0000 7f121e7b2640 1 -- 172.21.15.106:0/2516228899 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f12181f77b0 con 0x7f121814fe20 2024-04-25T10:34:01.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.765+0000 7f1217fff640 1 --2- 172.21.15.106:0/2516228899 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f121814fe20 0x7f121814be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:01.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.766+0000 7f1217fff640 1 -- 172.21.15.106:0/2516228899 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f121806a3d0 con 0x7f121814fe20 2024-04-25T10:34:01.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.766+0000 7f12157fa640 1 -- 172.21.15.106:0/2516228899 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f121001b020 con 0x7f12180c8d90 2024-04-25T10:34:01.768 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.766+0000 7f1217fff640 1 --2- 172.21.15.106:0/2516228899 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f121814fe20 0x7f121814be50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f12080053c0 tx=0x7f1208005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:01.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.767+0000 7f12157fa640 1 --2- 172.21.15.106:0/2516228899 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f11f8074460 0x7f11f8076970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:01.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.768+0000 7f120ffff640 1 --2- 172.21.15.106:0/2516228899 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f11f8074460 0x7f11f8076970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:01.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.768+0000 7f12157fa640 1 -- 172.21.15.106:0/2516228899 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f1210095e40 con 0x7f12180c8d90 2024-04-25T10:34:01.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.768+0000 7f12157fa640 1 -- 172.21.15.106:0/2516228899 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1208016070 con 0x7f121814fe20 2024-04-25T10:34:01.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.768+0000 7f12157fa640 1 -- 172.21.15.106:0/2516228899 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f120800eaf0 con 0x7f121814fe20 2024-04-25T10:34:01.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.768+0000 7f12157fa640 1 -- 172.21.15.106:0/2516228899 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f121814fe20 msgr2=0x7f121814be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:01.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.768+0000 7f12157fa640 1 --2- 172.21.15.106:0/2516228899 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f121814fe20 0x7f121814be50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f12080053c0 tx=0x7f1208005880 comp rx=0 tx=0).stop 2024-04-25T10:34:01.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.768+0000 7f12157fa640 1 -- 172.21.15.106:0/2516228899 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f121814fe20 msgr2=0x7f121814be50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T10:34:01.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.768+0000 7f12157fa640 1 --2- 172.21.15.106:0/2516228899 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f121814fe20 0x7f121814be50 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:01.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.768+0000 7f120ffff640 1 --2- 172.21.15.106:0/2516228899 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f11f8074460 0x7f11f8076970 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1200033000 tx=0x7f120002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:01.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.778+0000 7f121e7b2640 1 --2- 172.21.15.106:0/2516228899 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1218157470 0x7f12181557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:01.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.779+0000 7f121e7b2640 1 -- 172.21.15.106:0/2516228899 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f1218158d70 con 0x7f1218157470 2024-04-25T10:34:01.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.779+0000 7f1217fff640 1 --2- 172.21.15.106:0/2516228899 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1218157470 0x7f12181557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:01.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.780+0000 7f1217fff640 1 -- 172.21.15.106:0/2516228899 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f1218076370 con 0x7f1218157470 2024-04-25T10:34:01.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.780+0000 7f1217fff640 1 --2- 172.21.15.106:0/2516228899 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1218157470 0x7f12181557f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f12080053c0 tx=0x7f12080059d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:01.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.781+0000 7f12157fa640 1 -- 172.21.15.106:0/2516228899 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1208016040 con 0x7f1218157470 2024-04-25T10:34:01.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.781+0000 7f12157fa640 1 -- 172.21.15.106:0/2516228899 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5057 (secure 0 0 0) 0x7f120801b450 con 0x7f1218157470 2024-04-25T10:34:01.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.781+0000 7f12157fa640 1 -- 172.21.15.106:0/2516228899 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1218157470 msgr2=0x7f12181557f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:01.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.781+0000 7f12157fa640 1 --2- 172.21.15.106:0/2516228899 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1218157470 0x7f12181557f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f12080053c0 tx=0x7f12080059d0 comp rx=0 tx=0).stop 2024-04-25T10:34:01.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.781+0000 7f12157fa640 1 -- 172.21.15.106:0/2516228899 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1218157470 msgr2=0x7f12181557f0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T10:34:01.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.781+0000 7f12157fa640 1 --2- 172.21.15.106:0/2516228899 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1218157470 0x7f12181557f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:01.783 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-25T10:34:01.783 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "c", 2024-04-25T10:34:01.783 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 2, 2024-04-25T10:34:01.783 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "peon", 2024-04-25T10:34:01.783 INFO:teuthology.orchestra.run.smithi106.stdout: "election_epoch": 24, 2024-04-25T10:34:01.783 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 80704, 2024-04-25T10:34:01.783 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum": [ 2024-04-25T10:34:01.783 INFO:teuthology.orchestra.run.smithi106.stdout: 1, 2024-04-25T10:34:01.784 INFO:teuthology.orchestra.run.smithi106.stdout: 2 2024-04-25T10:34:01.784 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:34:01.784 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_age": 1, 2024-04-25T10:34:01.784 INFO:teuthology.orchestra.run.smithi106.stdout: "features": { 2024-04-25T10:34:01.784 INFO:teuthology.orchestra.run.smithi106.stdout: "required_con": "2450521708007866372", 2024-04-25T10:34:01.784 INFO:teuthology.orchestra.run.smithi106.stdout: "required_mon": [ 2024-04-25T10:34:01.784 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:34:01.784 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:34:01.784 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:34:01.784 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:34:01.784 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:34:01.784 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:34:01.784 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:34:01.784 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:34:01.785 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:34:01.785 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:34:01.785 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:34:01.785 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:34:01.785 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_con": "4540701547738038271", 2024-04-25T10:34:01.785 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_mon": [ 2024-04-25T10:34:01.785 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:34:01.785 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:34:01.785 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:34:01.785 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:34:01.785 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:34:01.785 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:34:01.785 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:34:01.785 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:34:01.786 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:34:01.786 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:34:01.786 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:34:01.786 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:34:01.786 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:34:01.786 INFO:teuthology.orchestra.run.smithi106.stdout: "outside_quorum": [], 2024-04-25T10:34:01.786 INFO:teuthology.orchestra.run.smithi106.stdout: "extra_probe_peers": [], 2024-04-25T10:34:01.786 INFO:teuthology.orchestra.run.smithi106.stdout: "sync_provider": [], 2024-04-25T10:34:01.786 INFO:teuthology.orchestra.run.smithi106.stdout: "monmap": { 2024-04-25T10:34:01.786 INFO:teuthology.orchestra.run.smithi106.stdout: "epoch": 1, 2024-04-25T10:34:01.786 INFO:teuthology.orchestra.run.smithi106.stdout: "fsid": "3ee77985-e8e4-41cf-81e9-7c774cf7891f", 2024-04-25T10:34:01.786 INFO:teuthology.orchestra.run.smithi106.stdout: "modified": "2024-04-25T10:29:33.446228Z", 2024-04-25T10:34:01.786 INFO:teuthology.orchestra.run.smithi106.stdout: "created": "2024-04-25T10:29:33.446228Z", 2024-04-25T10:34:01.786 INFO:teuthology.orchestra.run.smithi106.stdout: "min_mon_release": 19, 2024-04-25T10:34:01.787 INFO:teuthology.orchestra.run.smithi106.stdout: "min_mon_release_name": "squid", 2024-04-25T10:34:01.787 INFO:teuthology.orchestra.run.smithi106.stdout: "election_strategy": 1, 2024-04-25T10:34:01.787 INFO:teuthology.orchestra.run.smithi106.stdout: "disallowed_leaders: ": "", 2024-04-25T10:34:01.787 INFO:teuthology.orchestra.run.smithi106.stdout: "stretch_mode": false, 2024-04-25T10:34:01.787 INFO:teuthology.orchestra.run.smithi106.stdout: "tiebreaker_mon": "", 2024-04-25T10:34:01.787 INFO:teuthology.orchestra.run.smithi106.stdout: "removed_ranks: ": "", 2024-04-25T10:34:01.787 INFO:teuthology.orchestra.run.smithi106.stdout: "features": { 2024-04-25T10:34:01.787 INFO:teuthology.orchestra.run.smithi106.stdout: "persistent": [ 2024-04-25T10:34:01.787 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:34:01.787 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:34:01.787 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:34:01.787 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:34:01.787 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:34:01.787 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:34:01.787 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:34:01.788 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:34:01.788 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:34:01.788 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:34:01.788 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:34:01.788 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:34:01.788 INFO:teuthology.orchestra.run.smithi106.stdout: "optional": [] 2024-04-25T10:34:01.788 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:34:01.788 INFO:teuthology.orchestra.run.smithi106.stdout: "mons": [ 2024-04-25T10:34:01.788 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:34:01.788 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 0, 2024-04-25T10:34:01.788 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "a", 2024-04-25T10:34:01.788 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:34:01.788 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:34:01.788 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:34:01.788 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:34:01.789 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:3300", 2024-04-25T10:34:01.789 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:34:01.789 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:34:01.789 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:34:01.789 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:34:01.789 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:6789", 2024-04-25T10:34:01.789 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:34:01.790 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:34:01.790 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:34:01.790 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:34:01.790 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:6789/0", 2024-04-25T10:34:01.790 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.106:6789/0", 2024-04-25T10:34:01.790 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:34:01.790 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:34:01.790 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:34:01.790 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:34:01.790 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:34:01.790 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 1, 2024-04-25T10:34:01.790 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "b", 2024-04-25T10:34:01.790 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:34:01.790 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:34:01.791 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:34:01.791 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:34:01.791 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:3300", 2024-04-25T10:34:01.791 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:34:01.791 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:34:01.791 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:34:01.791 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:34:01.791 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6789", 2024-04-25T10:34:01.791 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:34:01.791 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:34:01.791 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:34:01.791 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:34:01.791 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6789/0", 2024-04-25T10:34:01.791 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.134:6789/0", 2024-04-25T10:34:01.791 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:34:01.791 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:34:01.792 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:34:01.792 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:34:01.792 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:34:01.792 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 2, 2024-04-25T10:34:01.792 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "c", 2024-04-25T10:34:01.792 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:34:01.792 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:34:01.792 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:34:01.792 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:34:01.792 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:3301", 2024-04-25T10:34:01.792 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:34:01.792 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:34:01.792 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:34:01.793 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:34:01.793 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6790", 2024-04-25T10:34:01.793 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:34:01.793 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:34:01.793 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:34:01.793 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:34:01.793 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6790/0", 2024-04-25T10:34:01.794 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.134:6790/0", 2024-04-25T10:34:01.794 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:34:01.794 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:34:01.794 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:34:01.794 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:34:01.795 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:34:01.795 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:34:01.795 INFO:teuthology.orchestra.run.smithi106.stdout: "feature_map": { 2024-04-25T10:34:01.795 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": [ 2024-04-25T10:34:01.795 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:34:01.795 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:34:01.796 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:34:01.796 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 1 2024-04-25T10:34:01.796 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:34:01.796 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:34:01.796 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": [ 2024-04-25T10:34:01.796 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:34:01.797 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:34:01.797 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:34:01.797 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 4 2024-04-25T10:34:01.797 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:34:01.797 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:34:01.797 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": [ 2024-04-25T10:34:01.798 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:34:01.798 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:34:01.798 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:34:01.798 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 5 2024-04-25T10:34:01.798 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:34:01.798 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:34:01.799 INFO:teuthology.orchestra.run.smithi106.stdout: "client": [ 2024-04-25T10:34:01.799 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:34:01.799 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:34:01.799 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:34:01.799 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 5 2024-04-25T10:34:01.800 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:34:01.800 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:34:01.800 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": [ 2024-04-25T10:34:01.800 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:34:01.800 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:34:01.800 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:34:01.801 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 1 2024-04-25T10:34:01.801 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:34:01.801 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:34:01.801 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:34:01.801 INFO:teuthology.orchestra.run.smithi106.stdout: "stretch_mode": false 2024-04-25T10:34:01.801 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-25T10:34:01.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.782+0000 7f121e7b2640 1 -- 172.21.15.106:0/2516228899 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f11f8074460 msgr2=0x7f11f8076970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:01.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.782+0000 7f121e7b2640 1 --2- 172.21.15.106:0/2516228899 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f11f8074460 0x7f11f8076970 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1200033000 tx=0x7f120002e000 comp rx=0 tx=0).stop 2024-04-25T10:34:01.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.782+0000 7f121e7b2640 1 -- 172.21.15.106:0/2516228899 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12180c8d90 msgr2=0x7f121814b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:01.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.782+0000 7f121cd4f640 1 -- 172.21.15.106:0/2516228899 reap_dead start 2024-04-25T10:34:01.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.782+0000 7f121e7b2640 1 --2- 172.21.15.106:0/2516228899 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12180c8d90 0x7f121814b910 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1218090000 tx=0x7f121000c0c0 comp rx=0 tx=0).stop 2024-04-25T10:34:01.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.782+0000 7f121e7b2640 1 -- 172.21.15.106:0/2516228899 shutdown_connections 2024-04-25T10:34:01.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.782+0000 7f121e7b2640 1 --2- 172.21.15.106:0/2516228899 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12180c8d90 0x7f121814b910 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:01.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.782+0000 7f121e7b2640 1 -- 172.21.15.106:0/2516228899 >> 172.21.15.106:0/2516228899 conn(0x7f1218068b10 msgr2=0x7f121815ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:01.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.782+0000 7f121e7b2640 1 -- 172.21.15.106:0/2516228899 shutdown_connections 2024-04-25T10:34:01.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:01.782+0000 7f121e7b2640 1 -- 172.21.15.106:0/2516228899 wait complete. 2024-04-25T10:34:01.806 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-04-25T10:34:01.806 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-04-25T10:34:02.831 DEBUG:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:34:02.834 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:34:08.413 INFO:tasks.workunit.client.0.smithi106.stdout: 2024-04-25T10:34:08.413 INFO:tasks.workunit.client.0.smithi106.stdout: Children see throughput for 1 rewriters = 832916.81 kB/sec 2024-04-25T10:34:08.413 INFO:tasks.workunit.client.0.smithi106.stdout: Parent sees throughput for 1 rewriters = 832901.95 kB/sec 2024-04-25T10:34:08.414 INFO:tasks.workunit.client.0.smithi106.stdout: Min throughput per process = 832916.81 kB/sec 2024-04-25T10:34:08.414 INFO:tasks.workunit.client.0.smithi106.stdout: Max throughput per process = 832916.81 kB/sec 2024-04-25T10:34:08.414 INFO:tasks.workunit.client.0.smithi106.stdout: Avg throughput per process = 832916.81 kB/sec 2024-04-25T10:34:08.414 INFO:tasks.workunit.client.0.smithi106.stdout: Min xfer = 10485760.00 kB 2024-04-25T10:34:13.462 INFO:tasks.workunit.client.0.smithi106.stdout: 2024-04-25T10:34:13.462 INFO:tasks.workunit.client.0.smithi106.stdout: Children see throughput for 1 readers = 7966921.50 kB/sec 2024-04-25T10:34:13.462 INFO:tasks.workunit.client.0.smithi106.stdout: Parent sees throughput for 1 readers = 7965638.84 kB/sec 2024-04-25T10:34:13.462 INFO:tasks.workunit.client.0.smithi106.stdout: Min throughput per process = 7966921.50 kB/sec 2024-04-25T10:34:13.462 INFO:tasks.workunit.client.0.smithi106.stdout: Max throughput per process = 7966921.50 kB/sec 2024-04-25T10:34:13.462 INFO:tasks.workunit.client.0.smithi106.stdout: Avg throughput per process = 7966921.50 kB/sec 2024-04-25T10:34:13.462 INFO:tasks.workunit.client.0.smithi106.stdout: Min xfer = 10485760.00 kB 2024-04-25T10:34:16.925 INFO:tasks.workunit.client.0.smithi106.stdout: 2024-04-25T10:34:16.925 INFO:tasks.workunit.client.0.smithi106.stdout: Children see throughput for 1 re-readers = 7941808.50 kB/sec 2024-04-25T10:34:16.925 INFO:tasks.workunit.client.0.smithi106.stdout: Parent sees throughput for 1 re-readers = 7940473.92 kB/sec 2024-04-25T10:34:16.925 INFO:tasks.workunit.client.0.smithi106.stdout: Min throughput per process = 7941808.50 kB/sec 2024-04-25T10:34:16.925 INFO:tasks.workunit.client.0.smithi106.stdout: Max throughput per process = 7941808.50 kB/sec 2024-04-25T10:34:16.925 INFO:tasks.workunit.client.0.smithi106.stdout: Avg throughput per process = 7941808.50 kB/sec 2024-04-25T10:34:16.925 INFO:tasks.workunit.client.0.smithi106.stdout: Min xfer = 10485760.00 kB 2024-04-25T10:34:21.806 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-04-25T10:34:21.807 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2024-04-25T10:34:21.807 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-25T10:34:21.807 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-25T10:34:21.810 INFO:tasks.ceph.mon.a:Started 2024-04-25T10:34:21.810 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-04-25T10:34:21.810 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T10:34:21.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:21.918+0000 7fd8e679d640 1 Processor -- start 2024-04-25T10:34:21.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:21.918+0000 7fd8e679d640 1 -- start start 2024-04-25T10:34:21.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:21.919+0000 7fd8e679d640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fd8e01608b0 0x7fd8e0156c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:21.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:21.919+0000 7fd8e679d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8e00c9aa0 0x7fd8e00c9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:21.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:21.919+0000 7fd8e679d640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8e0160d80 con 0x7fd8e00737b0 2024-04-25T10:34:21.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:21.919+0000 7fd8e679d640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fd8e015d5d0 con 0x7fd8e01608b0 2024-04-25T10:34:21.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:21.919+0000 7fd8e679d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fd8e015c9c0 con 0x7fd8e00c9aa0 2024-04-25T10:34:21.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:21.920+0000 7fd8e579b640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fd8e01608b0 0x7fd8e0156c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:21.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:21.920+0000 7fd8e5f9c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8e00c9aa0 0x7fd8e00c9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:21.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:21.920+0000 7fd8e579b640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fd8e01608b0 0x7fd8e0156c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.106:42882/0 (socket says 172.21.15.106:42882) 2024-04-25T10:34:21.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:21.920+0000 7fd8e579b640 1 -- 172.21.15.106:0/777985916 learned_addr learned my addr 172.21.15.106:0/777985916 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:34:21.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:21.920+0000 7fd8e5f9c640 1 -- 172.21.15.106:0/777985916 >> v2:172.21.15.134:3300/0 conn(0x7fd8e01608b0 msgr2=0x7fd8e0156c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:21.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:21.920+0000 7fd8e5f9c640 1 --2- 172.21.15.106:0/777985916 >> v2:172.21.15.134:3300/0 conn(0x7fd8e01608b0 0x7fd8e0156c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:21.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:21.920+0000 7fd8e5f9c640 1 -- 172.21.15.106:0/777985916 >> v1:172.21.15.134:6789/0 conn(0x7fd8e00737b0 legacy=0x7fd8e006c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:21.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:21.920+0000 7fd8e5f9c640 1 -- 172.21.15.106:0/777985916 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8e015cca0 con 0x7fd8e00c9aa0 2024-04-25T10:34:21.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:21.920+0000 7fd8e579b640 1 --2- 172.21.15.106:0/777985916 >> v2:172.21.15.134:3300/0 conn(0x7fd8e01608b0 0x7fd8e0156c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:34:21.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:21.921+0000 7fd8e5f9c640 1 --2- 172.21.15.106:0/777985916 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8e00c9aa0 0x7fd8e00c9eb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc0097a0 tx=0x7fd8dc02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2a54b67b960c880 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:21.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:21.921+0000 7fd8d7fff640 1 -- 172.21.15.106:0/777985916 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc037070 con 0x7fd8e00c9aa0 2024-04-25T10:34:21.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:21.921+0000 7fd8d7fff640 1 -- 172.21.15.106:0/777985916 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8dc032440 con 0x7fd8e00c9aa0 2024-04-25T10:34:21.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:21.921+0000 7fd8d7fff640 1 -- 172.21.15.106:0/777985916 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc032810 con 0x7fd8e00c9aa0 2024-04-25T10:34:21.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:21.922+0000 7fd8e679d640 1 -- 172.21.15.106:0/777985916 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8e00c9aa0 msgr2=0x7fd8e00c9eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:21.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:21.922+0000 7fd8e679d640 1 --2- 172.21.15.106:0/777985916 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8e00c9aa0 0x7fd8e00c9eb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc0097a0 tx=0x7fd8dc02a210 comp rx=0 tx=0).stop 2024-04-25T10:34:21.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:21.922+0000 7fd8e679d640 1 -- 172.21.15.106:0/777985916 shutdown_connections 2024-04-25T10:34:21.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:21.922+0000 7fd8e679d640 1 --2- 172.21.15.106:0/777985916 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8e00c9aa0 0x7fd8e00c9eb0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:21.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:21.922+0000 7fd8e679d640 1 --2- 172.21.15.106:0/777985916 >> v2:172.21.15.134:3300/0 conn(0x7fd8e01608b0 0x7fd8e0156c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:21.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:21.922+0000 7fd8e679d640 1 -- 172.21.15.106:0/777985916 >> 172.21.15.106:0/777985916 conn(0x7fd8e006dd00 msgr2=0x7fd8e00cee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:21.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:21.922+0000 7fd8e679d640 1 -- 172.21.15.106:0/777985916 shutdown_connections 2024-04-25T10:34:21.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:21.922+0000 7fd8e679d640 1 -- 172.21.15.106:0/777985916 wait complete. 2024-04-25T10:34:21.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:21.923+0000 7fd8e679d640 1 Processor -- start 2024-04-25T10:34:21.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:21.923+0000 7fd8e679d640 1 -- start start 2024-04-25T10:34:21.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:21.925+0000 7fd8e679d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd8e00737b0 0x7fd8e01f6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:21.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:21.925+0000 7fd8e679d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8e00c9aa0 0x7fd8e0153b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:21.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:21.925+0000 7fd8e679d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8e01608b0 0x7fd8e00cf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:21.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:21.925+0000 7fd8e679d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e00ca3f0 con 0x7fd8e00737b0 2024-04-25T10:34:21.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:21.925+0000 7fd8e679d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e015d260 con 0x7fd8e01608b0 2024-04-25T10:34:21.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:21.925+0000 7fd8e679d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fd8e0073d70 con 0x7fd8e00c9aa0 2024-04-25T10:34:21.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:21.925+0000 7fd8e579b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd8e00737b0 0x7fd8e01f6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:21.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:21.925+0000 7fd8e579b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd8e00737b0 0x7fd8e01f6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:32860/0 (socket says 172.21.15.106:32860) 2024-04-25T10:34:21.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:21.925+0000 7fd8e579b640 1 -- 172.21.15.106:0/3767018365 learned_addr learned my addr 172.21.15.106:0/3767018365 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:34:21.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:21.925+0000 7fd8e579b640 1 -- 172.21.15.106:0/3767018365 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd8e00737b0 msgr2=0x7fd8e01f6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-04-25T10:34:21.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:21.925+0000 7fd8e579b640 1 -- 172.21.15.106:0/3767018365 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd8e00737b0 msgr2=0x7fd8e01f6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T10:34:21.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:21.925+0000 7fd8e579b640 1 --2- 172.21.15.106:0/3767018365 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd8e00737b0 0x7fd8e01f6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:34:21.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:21.925+0000 7fd8e579b640 1 --2- 172.21.15.106:0/3767018365 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd8e00737b0 0x7fd8e01f6d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:34:21.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:21.925+0000 7fd8e5f9c640 1 --2- 172.21.15.106:0/3767018365 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8e01608b0 0x7fd8e00cf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:21.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:21.925+0000 7fd8e4f9a640 1 --2- 172.21.15.106:0/3767018365 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8e00c9aa0 0x7fd8e0153b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:21.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:21.925+0000 7fd8e4f9a640 1 -- 172.21.15.106:0/3767018365 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8e01608b0 msgr2=0x7fd8e00cf2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:21.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:21.925+0000 7fd8e4f9a640 1 --2- 172.21.15.106:0/3767018365 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8e01608b0 0x7fd8e00cf2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:21.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:21.925+0000 7fd8e4f9a640 1 -- 172.21.15.106:0/3767018365 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd8e00737b0 msgr2=0x7fd8e01f6d20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:34:21.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:21.925+0000 7fd8e4f9a640 1 --2- 172.21.15.106:0/3767018365 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd8e00737b0 0x7fd8e01f6d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:21.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:21.925+0000 7fd8e4f9a640 1 -- 172.21.15.106:0/3767018365 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8e0158d90 con 0x7fd8e00c9aa0 2024-04-25T10:34:21.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:21.925+0000 7fd8e5f9c640 1 --2- 172.21.15.106:0/3767018365 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8e01608b0 0x7fd8e00cf2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:34:21.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:21.925+0000 7fd8e4f9a640 1 --2- 172.21.15.106:0/3767018365 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8e00c9aa0 0x7fd8e0153b40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd8d000c3c0 tx=0x7fd8d000c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:21.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:21.926+0000 7fd8d5ffb640 1 -- 172.21.15.106:0/3767018365 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8d00052d0 con 0x7fd8e00c9aa0 2024-04-25T10:34:21.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:21.926+0000 7fd8d5ffb640 1 -- 172.21.15.106:0/3767018365 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8d0005490 con 0x7fd8e00c9aa0 2024-04-25T10:34:21.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:21.926+0000 7fd8e679d640 1 -- 172.21.15.106:0/3767018365 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8e00ceeb0 con 0x7fd8e00c9aa0 2024-04-25T10:34:21.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:21.926+0000 7fd8d5ffb640 1 -- 172.21.15.106:0/3767018365 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8d0007260 con 0x7fd8e00c9aa0 2024-04-25T10:34:21.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:21.926+0000 7fd8e679d640 1 -- 172.21.15.106:0/3767018365 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8e00ce350 con 0x7fd8e00c9aa0 2024-04-25T10:34:21.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:21.927+0000 7fd8e679d640 1 -- 172.21.15.106:0/3767018365 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8e0050740 con 0x7fd8e00c9aa0 2024-04-25T10:34:21.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:21.927+0000 7fd8d5ffb640 1 -- 172.21.15.106:0/3767018365 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd8d0007400 con 0x7fd8e00c9aa0 2024-04-25T10:34:21.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:21.928+0000 7fd8d5ffb640 1 --2- 172.21.15.106:0/3767018365 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fd8c40744b0 0x7fd8c40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:21.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:21.928+0000 7fd8e579b640 1 --2- 172.21.15.106:0/3767018365 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fd8c40744b0 0x7fd8c40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:21.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:21.928+0000 7fd8d5ffb640 1 -- 172.21.15.106:0/3767018365 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fd8d00944f0 con 0x7fd8e00c9aa0 2024-04-25T10:34:21.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:21.928+0000 7fd8e579b640 1 --2- 172.21.15.106:0/3767018365 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fd8c40744b0 0x7fd8c40769c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd8d80072e0 tx=0x7fd8d800b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:21.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:21.933+0000 7fd8d5ffb640 1 -- 172.21.15.106:0/3767018365 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd8d005e640 con 0x7fd8e00c9aa0 2024-04-25T10:34:22.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:22.174+0000 7fd8e679d640 1 -- 172.21.15.106:0/3767018365 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fd8e0076520 con 0x7fd8e00c9aa0 2024-04-25T10:34:22.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:22.175+0000 7fd8d5ffb640 1 -- 172.21.15.106:0/3767018365 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1563 (secure 0 0 0) 0x7fd8d0062940 con 0x7fd8e00c9aa0 2024-04-25T10:34:22.178 INFO:teuthology.orchestra.run.smithi106.stdout:{"election_epoch":24,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":21,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"3ee77985-e8e4-41cf-81e9-7c774cf7891f","modified":"2024-04-25T10:29:33.446228Z","created":"2024-04-25T10:29:33.446228Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:3300","nonce":0},{"type":"v1","addr":"172.21.15.106:6789","nonce":0}]},"addr":"172.21.15.106:6789/0","public_addr":"172.21.15.106:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T10:34:22.179 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:22.177+0000 7fd8e679d640 1 -- 172.21.15.106:0/3767018365 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fd8c40744b0 msgr2=0x7fd8c40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:22.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:22.177+0000 7fd8e679d640 1 --2- 172.21.15.106:0/3767018365 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fd8c40744b0 0x7fd8c40769c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd8d80072e0 tx=0x7fd8d800b040 comp rx=0 tx=0).stop 2024-04-25T10:34:22.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:22.177+0000 7fd8e679d640 1 -- 172.21.15.106:0/3767018365 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8e00c9aa0 msgr2=0x7fd8e0153b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:22.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:22.177+0000 7fd8e679d640 1 --2- 172.21.15.106:0/3767018365 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8e00c9aa0 0x7fd8e0153b40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd8d000c3c0 tx=0x7fd8d000c880 comp rx=0 tx=0).stop 2024-04-25T10:34:22.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:22.177+0000 7fd8e679d640 1 -- 172.21.15.106:0/3767018365 shutdown_connections 2024-04-25T10:34:22.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:22.177+0000 7fd8e679d640 1 --2- 172.21.15.106:0/3767018365 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fd8c40744b0 0x7fd8c40769c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:22.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:22.177+0000 7fd8e679d640 1 --2- 172.21.15.106:0/3767018365 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8e01608b0 0x7fd8e00cf2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:22.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:22.177+0000 7fd8e679d640 1 --2- 172.21.15.106:0/3767018365 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8e00c9aa0 0x7fd8e0153b40 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:22.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:22.177+0000 7fd8e679d640 1 --2- 172.21.15.106:0/3767018365 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd8e00737b0 0x7fd8e01f6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:22.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:22.177+0000 7fd8e679d640 1 -- 172.21.15.106:0/3767018365 >> 172.21.15.106:0/3767018365 conn(0x7fd8e006dd00 msgr2=0x7fd8e015beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:22.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:22.177+0000 7fd8e679d640 1 -- 172.21.15.106:0/3767018365 shutdown_connections 2024-04-25T10:34:22.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:22.177+0000 7fd8e679d640 1 -- 172.21.15.106:0/3767018365 wait complete. 2024-04-25T10:34:23.815 INFO:tasks.workunit.client.0.smithi106.stdout: 2024-04-25T10:34:23.815 INFO:tasks.workunit.client.0.smithi106.stdout: 2024-04-25T10:34:23.815 INFO:tasks.workunit.client.0.smithi106.stdout: 2024-04-25T10:34:23.815 INFO:tasks.workunit.client.0.smithi106.stdout:iozone test complete. 2024-04-25T10:34:23.816 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-25T10:34:23.816 DEBUG:teuthology.orchestra.run.smithi106:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-25T10:34:23.852 INFO:tasks.workunit:Stopping ['suites/iozone.sh'] on client.0... 2024-04-25T10:34:23.852 DEBUG:teuthology.orchestra.run.smithi106:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-25T10:34:24.157 DEBUG:teuthology.parallel:result is None 2024-04-25T10:34:24.157 DEBUG:teuthology.orchestra.run.smithi106:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-25T10:34:24.188 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-25T10:34:24.188 DEBUG:teuthology.run_tasks:Unwinding manager mon_thrash 2024-04-25T10:34:24.198 INFO:tasks.mon_thrash:joining mon_thrasher 2024-04-25T10:34:25.191 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T10:34:25.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.301+0000 7f8afebb7640 1 Processor -- start 2024-04-25T10:34:25.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.301+0000 7f8afebb7640 1 -- start start 2024-04-25T10:34:25.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.302+0000 7f8afebb7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8af80ce540 0x7f8af806d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:25.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.302+0000 7f8afebb7640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f8af815b300 0x7f8af806d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:25.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.302+0000 7f8afebb7640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8af80c7780 con 0x7f8af8161390 2024-04-25T10:34:25.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.302+0000 7f8afebb7640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f8af80c7530 con 0x7f8af815b300 2024-04-25T10:34:25.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.302+0000 7f8afebb7640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f8af80c7a60 con 0x7f8af80ce540 2024-04-25T10:34:25.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.302+0000 7f8af7fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f8af815b300 0x7f8af806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:25.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.302+0000 7f8af7fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f8af815b300 0x7f8af806d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60278/0 (socket says 172.21.15.106:60278) 2024-04-25T10:34:25.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.302+0000 7f8afd154640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f8af8161390 0x7f8af81617a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:46738/0 (socket says 172.21.15.106:46738) 2024-04-25T10:34:25.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.302+0000 7f8af7fff640 1 -- 172.21.15.106:0/1335705007 learned_addr learned my addr 172.21.15.106:0/1335705007 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:34:25.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.302+0000 7f8af7fff640 1 -- 172.21.15.106:0/1335705007 >> v2:172.21.15.106:3300/0 conn(0x7f8af815b300 msgr2=0x7f8af806d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T10:34:25.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.302+0000 7f8af7fff640 1 -- 172.21.15.106:0/1335705007 >> v2:172.21.15.106:3300/0 conn(0x7f8af815b300 msgr2=0x7f8af806d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:34:25.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.302+0000 7f8af7fff640 1 --2- 172.21.15.106:0/1335705007 >> v2:172.21.15.106:3300/0 conn(0x7f8af815b300 0x7f8af806d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:34:25.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.302+0000 7f8af7fff640 1 --2- 172.21.15.106:0/1335705007 >> v2:172.21.15.106:3300/0 conn(0x7f8af815b300 0x7f8af806d540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:34:25.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.302+0000 7f8afc953640 1 --2- 172.21.15.106:0/1335705007 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8af80ce540 0x7f8af806d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:25.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.303+0000 7f8afc953640 1 -- 172.21.15.106:0/1335705007 >> v2:172.21.15.106:3300/0 conn(0x7f8af815b300 msgr2=0x7f8af806d540 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:34:25.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.303+0000 7f8afc953640 1 --2- 172.21.15.106:0/1335705007 >> v2:172.21.15.106:3300/0 conn(0x7f8af815b300 0x7f8af806d540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:25.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.303+0000 7f8afc953640 1 -- 172.21.15.106:0/1335705007 >> v1:172.21.15.106:6789/0 conn(0x7f8af8161390 legacy=0x7f8af81617a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:25.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.303+0000 7f8afc953640 1 -- 172.21.15.106:0/1335705007 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8af80c7d40 con 0x7f8af80ce540 2024-04-25T10:34:25.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.304+0000 7f8afc953640 1 --2- 172.21.15.106:0/1335705007 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8af80ce540 0x7f8af806d000 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8ae80097a0 tx=0x7f8ae802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60a7b909442cc2fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:25.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.304+0000 7f8af77fe640 1 -- 172.21.15.106:0/1335705007 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae8037070 con 0x7f8af80ce540 2024-04-25T10:34:25.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.304+0000 7f8af77fe640 1 -- 172.21.15.106:0/1335705007 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ae8032440 con 0x7f8af80ce540 2024-04-25T10:34:25.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.304+0000 7f8af77fe640 1 -- 172.21.15.106:0/1335705007 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae8032810 con 0x7f8af80ce540 2024-04-25T10:34:25.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.304+0000 7f8afebb7640 1 -- 172.21.15.106:0/1335705007 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8af80ce540 msgr2=0x7f8af806d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:25.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.304+0000 7f8afebb7640 1 --2- 172.21.15.106:0/1335705007 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8af80ce540 0x7f8af806d000 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8ae80097a0 tx=0x7f8ae802a180 comp rx=0 tx=0).stop 2024-04-25T10:34:25.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.304+0000 7f8afebb7640 1 -- 172.21.15.106:0/1335705007 shutdown_connections 2024-04-25T10:34:25.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.304+0000 7f8afebb7640 1 --2- 172.21.15.106:0/1335705007 >> v2:172.21.15.106:3300/0 conn(0x7f8af815b300 0x7f8af806d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:25.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.305+0000 7f8afebb7640 1 --2- 172.21.15.106:0/1335705007 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8af80ce540 0x7f8af806d000 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:25.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.305+0000 7f8afebb7640 1 -- 172.21.15.106:0/1335705007 >> 172.21.15.106:0/1335705007 conn(0x7f8af8068b10 msgr2=0x7f8af80d1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:25.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.305+0000 7f8afebb7640 1 -- 172.21.15.106:0/1335705007 shutdown_connections 2024-04-25T10:34:25.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.305+0000 7f8afebb7640 1 -- 172.21.15.106:0/1335705007 wait complete. 2024-04-25T10:34:25.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.306+0000 7f8afebb7640 1 Processor -- start 2024-04-25T10:34:25.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.306+0000 7f8afebb7640 1 -- start start 2024-04-25T10:34:25.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.308+0000 7f8afebb7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8af80ce540 0x7f8af8156f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:25.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.308+0000 7f8afebb7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8af815b300 0x7f8af80cbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:25.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.308+0000 7f8afebb7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8af8161390 0x7f8af8155f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:25.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.308+0000 7f8afebb7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8af80c6ee0 con 0x7f8af80ce540 2024-04-25T10:34:25.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.308+0000 7f8afebb7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8af80b6190 con 0x7f8af815b300 2024-04-25T10:34:25.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.308+0000 7f8afebb7640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f8af80b4d10 con 0x7f8af8161390 2024-04-25T10:34:25.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.308+0000 7f8afc953640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8af80ce540 0x7f8af8156f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:25.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.308+0000 7f8afc953640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8af80ce540 0x7f8af8156f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60284/0 (socket says 172.21.15.106:60284) 2024-04-25T10:34:25.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.308+0000 7f8afc953640 1 -- 172.21.15.106:0/1322207599 learned_addr learned my addr 172.21.15.106:0/1322207599 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:34:25.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.308+0000 7f8afc953640 1 -- 172.21.15.106:0/1322207599 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8af80ce540 msgr2=0x7f8af8156f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-04-25T10:34:25.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.308+0000 7f8afc953640 1 -- 172.21.15.106:0/1322207599 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8af80ce540 msgr2=0x7f8af8156f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T10:34:25.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.308+0000 7f8afc953640 1 --2- 172.21.15.106:0/1322207599 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8af80ce540 0x7f8af8156f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:34:25.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.308+0000 7f8afc953640 1 --2- 172.21.15.106:0/1322207599 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8af80ce540 0x7f8af8156f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:34:25.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.308+0000 7f8afd154640 1 --2- 172.21.15.106:0/1322207599 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8af8161390 0x7f8af8155f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:25.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.308+0000 7f8af7fff640 1 --2- 172.21.15.106:0/1322207599 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8af815b300 0x7f8af80cbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:25.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.308+0000 7f8afd154640 1 -- 172.21.15.106:0/1322207599 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8af815b300 msgr2=0x7f8af80cbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:25.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.308+0000 7f8afd154640 1 --2- 172.21.15.106:0/1322207599 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8af815b300 0x7f8af80cbbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:25.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.308+0000 7f8afd154640 1 -- 172.21.15.106:0/1322207599 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8af80ce540 msgr2=0x7f8af8156f30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:34:25.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.308+0000 7f8afd154640 1 --2- 172.21.15.106:0/1322207599 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8af80ce540 0x7f8af8156f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:25.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.308+0000 7f8afd154640 1 -- 172.21.15.106:0/1322207599 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8af81f6d90 con 0x7f8af8161390 2024-04-25T10:34:25.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.309+0000 7f8afd154640 1 --2- 172.21.15.106:0/1322207599 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8af8161390 0x7f8af8155f20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8af000c700 tx=0x7f8af000cbc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:25.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.309+0000 7f8af57fa640 1 -- 172.21.15.106:0/1322207599 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af0018070 con 0x7f8af8161390 2024-04-25T10:34:25.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.309+0000 7f8af57fa640 1 -- 172.21.15.106:0/1322207599 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8af0004d60 con 0x7f8af8161390 2024-04-25T10:34:25.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.309+0000 7f8afebb7640 1 -- 172.21.15.106:0/1322207599 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8af81f7050 con 0x7f8af8161390 2024-04-25T10:34:25.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.309+0000 7f8af57fa640 1 -- 172.21.15.106:0/1322207599 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af000a860 con 0x7f8af8161390 2024-04-25T10:34:25.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.309+0000 7f8afebb7640 1 -- 172.21.15.106:0/1322207599 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8af81f7610 con 0x7f8af8161390 2024-04-25T10:34:25.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.310+0000 7f8afebb7640 1 -- 172.21.15.106:0/1322207599 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8af81626b0 con 0x7f8af8161390 2024-04-25T10:34:25.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.311+0000 7f8af57fa640 1 -- 172.21.15.106:0/1322207599 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8af0036b70 con 0x7f8af8161390 2024-04-25T10:34:25.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.311+0000 7f8af57fa640 1 --2- 172.21.15.106:0/1322207599 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f8ad80744b0 0x7f8ad80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:25.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.311+0000 7f8afc953640 1 --2- 172.21.15.106:0/1322207599 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f8ad80744b0 0x7f8ad80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:25.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.311+0000 7f8af57fa640 1 -- 172.21.15.106:0/1322207599 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f8af0094540 con 0x7f8af8161390 2024-04-25T10:34:25.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.311+0000 7f8afc953640 1 --2- 172.21.15.106:0/1322207599 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f8ad80744b0 0x7f8ad80769c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8035040 tx=0x7f8ae802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:25.317 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.315+0000 7f8af57fa640 1 -- 172.21.15.106:0/1322207599 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8af005e690 con 0x7f8af8161390 2024-04-25T10:34:25.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.556+0000 7f8afebb7640 1 -- 172.21.15.106:0/1322207599 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f8af806a3d0 con 0x7f8af8161390 2024-04-25T10:34:25.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.557+0000 7f8af57fa640 1 -- 172.21.15.106:0/1322207599 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1563 (secure 0 0 0) 0x7f8af0062990 con 0x7f8af8161390 2024-04-25T10:34:25.560 INFO:teuthology.orchestra.run.smithi106.stdout:{"election_epoch":24,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":25,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"3ee77985-e8e4-41cf-81e9-7c774cf7891f","modified":"2024-04-25T10:29:33.446228Z","created":"2024-04-25T10:29:33.446228Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:3300","nonce":0},{"type":"v1","addr":"172.21.15.106:6789","nonce":0}]},"addr":"172.21.15.106:6789/0","public_addr":"172.21.15.106:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T10:34:25.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.559+0000 7f8afebb7640 1 -- 172.21.15.106:0/1322207599 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f8ad80744b0 msgr2=0x7f8ad80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:25.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.559+0000 7f8afebb7640 1 --2- 172.21.15.106:0/1322207599 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f8ad80744b0 0x7f8ad80769c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8035040 tx=0x7f8ae802e000 comp rx=0 tx=0).stop 2024-04-25T10:34:25.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.559+0000 7f8afebb7640 1 -- 172.21.15.106:0/1322207599 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8af8161390 msgr2=0x7f8af8155f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:25.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.559+0000 7f8afebb7640 1 --2- 172.21.15.106:0/1322207599 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8af8161390 0x7f8af8155f20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8af000c700 tx=0x7f8af000cbc0 comp rx=0 tx=0).stop 2024-04-25T10:34:25.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.559+0000 7f8afebb7640 1 -- 172.21.15.106:0/1322207599 shutdown_connections 2024-04-25T10:34:25.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.559+0000 7f8afebb7640 1 --2- 172.21.15.106:0/1322207599 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f8ad80744b0 0x7f8ad80769c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:25.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.559+0000 7f8afebb7640 1 --2- 172.21.15.106:0/1322207599 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8af8161390 0x7f8af8155f20 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:25.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.559+0000 7f8afebb7640 1 --2- 172.21.15.106:0/1322207599 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8af815b300 0x7f8af80cbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:25.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.559+0000 7f8afebb7640 1 --2- 172.21.15.106:0/1322207599 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8af80ce540 0x7f8af8156f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:25.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.559+0000 7f8afebb7640 1 -- 172.21.15.106:0/1322207599 >> 172.21.15.106:0/1322207599 conn(0x7f8af8068b10 msgr2=0x7f8af806d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:25.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.559+0000 7f8afebb7640 1 -- 172.21.15.106:0/1322207599 shutdown_connections 2024-04-25T10:34:25.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:25.559+0000 7f8afebb7640 1 -- 172.21.15.106:0/1322207599 wait complete. 2024-04-25T10:34:28.575 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T10:34:28.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.697+0000 7fb9709be640 1 Processor -- start 2024-04-25T10:34:28.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.697+0000 7fb9709be640 1 -- start start 2024-04-25T10:34:28.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.698+0000 7fb9709be640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb96c0c7e30 0x7fb96c0c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:28.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.698+0000 7fb9709be640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb96c0b4ce0 0x7fb96c15ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:28.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.698+0000 7fb9709be640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb96c0ce630 con 0x7fb96c156ef0 2024-04-25T10:34:28.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.698+0000 7fb9709be640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fb96c0b50f0 con 0x7fb96c0c7e30 2024-04-25T10:34:28.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.698+0000 7fb9709be640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb96c0c7c60 con 0x7fb96c0b4ce0 2024-04-25T10:34:28.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.698+0000 7fb96a59c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb96c0c7e30 0x7fb96c0c8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:28.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.698+0000 7fb96a59c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb96c0c7e30 0x7fb96c0c8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60300/0 (socket says 172.21.15.106:60300) 2024-04-25T10:34:28.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.698+0000 7fb96a59c640 1 -- 172.21.15.106:0/719364335 learned_addr learned my addr 172.21.15.106:0/719364335 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:34:28.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.698+0000 7fb96ad9d640 1 --2- 172.21.15.106:0/719364335 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb96c0b4ce0 0x7fb96c15ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:28.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.699+0000 7fb96a59c640 1 -- 172.21.15.106:0/719364335 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb96c0b4ce0 msgr2=0x7fb96c15ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:28.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.699+0000 7fb96a59c640 1 --2- 172.21.15.106:0/719364335 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb96c0b4ce0 0x7fb96c15ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:28.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.699+0000 7fb96a59c640 1 -- 172.21.15.106:0/719364335 >> v1:172.21.15.134:6789/0 conn(0x7fb96c156ef0 legacy=0x7fb96c06d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:28.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.699+0000 7fb96a59c640 1 -- 172.21.15.106:0/719364335 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb96c0c7980 con 0x7fb96c0c7e30 2024-04-25T10:34:28.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.699+0000 7fb96ad9d640 1 --2- 172.21.15.106:0/719364335 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb96c0b4ce0 0x7fb96c15ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:34:28.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.699+0000 7fb96a59c640 1 --2- 172.21.15.106:0/719364335 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb96c0c7e30 0x7fb96c0c8d90 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb96c0c7de0 tx=0x7fb95c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=190171f477bb697a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:28.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.699+0000 7fb969d9b640 1 -- 172.21.15.106:0/719364335 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb95c037070 con 0x7fb96c0c7e30 2024-04-25T10:34:28.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.699+0000 7fb969d9b640 1 -- 172.21.15.106:0/719364335 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb95c032440 con 0x7fb96c0c7e30 2024-04-25T10:34:28.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.699+0000 7fb969d9b640 1 -- 172.21.15.106:0/719364335 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb95c032810 con 0x7fb96c0c7e30 2024-04-25T10:34:28.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.700+0000 7fb9709be640 1 -- 172.21.15.106:0/719364335 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb96c0c7e30 msgr2=0x7fb96c0c8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:28.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.700+0000 7fb9709be640 1 --2- 172.21.15.106:0/719364335 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb96c0c7e30 0x7fb96c0c8d90 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb96c0c7de0 tx=0x7fb95c02a1c0 comp rx=0 tx=0).stop 2024-04-25T10:34:28.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.700+0000 7fb9709be640 1 -- 172.21.15.106:0/719364335 shutdown_connections 2024-04-25T10:34:28.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.700+0000 7fb9709be640 1 --2- 172.21.15.106:0/719364335 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb96c0b4ce0 0x7fb96c15ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:28.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.700+0000 7fb9709be640 1 --2- 172.21.15.106:0/719364335 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb96c0c7e30 0x7fb96c0c8d90 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:28.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.700+0000 7fb9709be640 1 -- 172.21.15.106:0/719364335 >> 172.21.15.106:0/719364335 conn(0x7fb96c068b10 msgr2=0x7fb96c0d1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:28.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.700+0000 7fb9709be640 1 -- 172.21.15.106:0/719364335 shutdown_connections 2024-04-25T10:34:28.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.700+0000 7fb9709be640 1 -- 172.21.15.106:0/719364335 wait complete. 2024-04-25T10:34:28.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.702+0000 7fb9709be640 1 Processor -- start 2024-04-25T10:34:28.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.702+0000 7fb9709be640 1 -- start start 2024-04-25T10:34:28.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.703+0000 7fb9709be640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb96c0b4ce0 0x7fb96c157be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:28.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.703+0000 7fb9709be640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb96c0c7e30 0x7fb96c1f3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:28.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.703+0000 7fb9709be640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb96c156ef0 0x7fb96c0d1340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:28.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.703+0000 7fb9709be640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb96c06d710 con 0x7fb96c0b4ce0 2024-04-25T10:34:28.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.703+0000 7fb9709be640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb96c0c72f0 con 0x7fb96c0c7e30 2024-04-25T10:34:28.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.703+0000 7fb9709be640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb96c06d890 con 0x7fb96c156ef0 2024-04-25T10:34:28.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.704+0000 7fb96a59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb96c0b4ce0 0x7fb96c157be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:28.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.704+0000 7fb96a59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb96c0b4ce0 0x7fb96c157be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60306/0 (socket says 172.21.15.106:60306) 2024-04-25T10:34:28.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.704+0000 7fb96a59c640 1 -- 172.21.15.106:0/2888932892 learned_addr learned my addr 172.21.15.106:0/2888932892 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:34:28.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.704+0000 7fb96ad9d640 1 --2- 172.21.15.106:0/2888932892 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb96c156ef0 0x7fb96c0d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:28.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.704+0000 7fb963fff640 1 --2- 172.21.15.106:0/2888932892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb96c0c7e30 0x7fb96c1f3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:28.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.704+0000 7fb96a59c640 1 -- 172.21.15.106:0/2888932892 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb96c156ef0 msgr2=0x7fb96c0d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:28.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.704+0000 7fb96a59c640 1 --2- 172.21.15.106:0/2888932892 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb96c156ef0 0x7fb96c0d1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:28.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.704+0000 7fb96a59c640 1 -- 172.21.15.106:0/2888932892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb96c0c7e30 msgr2=0x7fb96c1f3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:28.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.704+0000 7fb96a59c640 1 --2- 172.21.15.106:0/2888932892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb96c0c7e30 0x7fb96c1f3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:28.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.704+0000 7fb96a59c640 1 -- 172.21.15.106:0/2888932892 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb96c0d1880 con 0x7fb96c0b4ce0 2024-04-25T10:34:28.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.704+0000 7fb963fff640 1 --2- 172.21.15.106:0/2888932892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb96c0c7e30 0x7fb96c1f3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:34:28.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.704+0000 7fb96a59c640 1 --2- 172.21.15.106:0/2888932892 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb96c0b4ce0 0x7fb96c157be0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb95c02a200 tx=0x7fb95c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:28.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.704+0000 7fb962ffd640 1 -- 172.21.15.106:0/2888932892 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb95c041020 con 0x7fb96c0b4ce0 2024-04-25T10:34:28.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.704+0000 7fb962ffd640 1 -- 172.21.15.106:0/2888932892 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb95c0306c0 con 0x7fb96c0b4ce0 2024-04-25T10:34:28.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.704+0000 7fb9709be640 1 -- 172.21.15.106:0/2888932892 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb96c0d0830 con 0x7fb96c0b4ce0 2024-04-25T10:34:28.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.704+0000 7fb962ffd640 1 -- 172.21.15.106:0/2888932892 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb95c037040 con 0x7fb96c0b4ce0 2024-04-25T10:34:28.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.704+0000 7fb9709be640 1 -- 172.21.15.106:0/2888932892 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb96c1f3d40 con 0x7fb96c0b4ce0 2024-04-25T10:34:28.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.705+0000 7fb9709be640 1 -- 172.21.15.106:0/2888932892 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb96c06a3d0 con 0x7fb96c0b4ce0 2024-04-25T10:34:28.710 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.706+0000 7fb962ffd640 1 -- 172.21.15.106:0/2888932892 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb95c044020 con 0x7fb96c0b4ce0 2024-04-25T10:34:28.710 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.706+0000 7fb962ffd640 1 --2- 172.21.15.106:0/2888932892 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fb94c0744e0 0x7fb94c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:28.710 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.706+0000 7fb962ffd640 1 -- 172.21.15.106:0/2888932892 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fb95c0b5aa0 con 0x7fb96c0b4ce0 2024-04-25T10:34:28.710 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.706+0000 7fb963fff640 1 --2- 172.21.15.106:0/2888932892 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fb94c0744e0 0x7fb94c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:28.710 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.707+0000 7fb963fff640 1 --2- 172.21.15.106:0/2888932892 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fb94c0744e0 0x7fb94c0769f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb96c0d07c0 tx=0x7fb954009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:28.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.711+0000 7fb962ffd640 1 -- 172.21.15.106:0/2888932892 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb95c04b030 con 0x7fb96c0b4ce0 2024-04-25T10:34:28.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.957+0000 7fb9709be640 1 -- 172.21.15.106:0/2888932892 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fb96c15b150 con 0x7fb96c0b4ce0 2024-04-25T10:34:28.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.958+0000 7fb962ffd640 1 -- 172.21.15.106:0/2888932892 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1568 (secure 0 0 0) 0x7fb95c0801b0 con 0x7fb96c0b4ce0 2024-04-25T10:34:28.960 INFO:teuthology.orchestra.run.smithi106.stdout:{"election_epoch":28,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"3ee77985-e8e4-41cf-81e9-7c774cf7891f","modified":"2024-04-25T10:29:33.446228Z","created":"2024-04-25T10:29:33.446228Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:3300","nonce":0},{"type":"v1","addr":"172.21.15.106:6789","nonce":0}]},"addr":"172.21.15.106:6789/0","public_addr":"172.21.15.106:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T10:34:28.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.960+0000 7fb9709be640 1 -- 172.21.15.106:0/2888932892 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fb94c0744e0 msgr2=0x7fb94c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:28.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.960+0000 7fb9709be640 1 --2- 172.21.15.106:0/2888932892 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fb94c0744e0 0x7fb94c0769f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb96c0d07c0 tx=0x7fb954009040 comp rx=0 tx=0).stop 2024-04-25T10:34:28.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.960+0000 7fb9709be640 1 -- 172.21.15.106:0/2888932892 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb96c0b4ce0 msgr2=0x7fb96c157be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:28.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.960+0000 7fb9709be640 1 --2- 172.21.15.106:0/2888932892 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb96c0b4ce0 0x7fb96c157be0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb95c02a200 tx=0x7fb95c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:34:28.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.960+0000 7fb9709be640 1 -- 172.21.15.106:0/2888932892 shutdown_connections 2024-04-25T10:34:28.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.960+0000 7fb9709be640 1 --2- 172.21.15.106:0/2888932892 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fb94c0744e0 0x7fb94c0769f0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:28.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.960+0000 7fb9709be640 1 --2- 172.21.15.106:0/2888932892 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb96c156ef0 0x7fb96c0d1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:28.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.960+0000 7fb9709be640 1 --2- 172.21.15.106:0/2888932892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb96c0c7e30 0x7fb96c1f3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:28.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.960+0000 7fb9709be640 1 --2- 172.21.15.106:0/2888932892 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb96c0b4ce0 0x7fb96c157be0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:28.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.960+0000 7fb9709be640 1 -- 172.21.15.106:0/2888932892 >> 172.21.15.106:0/2888932892 conn(0x7fb96c068b10 msgr2=0x7fb96c15cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:28.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.960+0000 7fb9709be640 1 -- 172.21.15.106:0/2888932892 shutdown_connections 2024-04-25T10:34:28.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:28.960+0000 7fb9709be640 1 -- 172.21.15.106:0/2888932892 wait complete. 2024-04-25T10:34:28.974 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-04-25T10:34:28.974 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-04-25T10:34:28.974 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-25T10:34:29.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.081+0000 7f0cf5e6e640 1 Processor -- start 2024-04-25T10:34:29.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.081+0000 7f0cf5e6e640 1 -- start start 2024-04-25T10:34:29.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.082+0000 7f0cf5e6e640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f0cf00c7e30 0x7f0cf00c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:29.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.082+0000 7f0cf5e6e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0cf00b4ce0 0x7f0cf015ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:29.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.082+0000 7f0cf5e6e640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0cf00ce630 con 0x7f0cf0156ef0 2024-04-25T10:34:29.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.082+0000 7f0cf5e6e640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f0cf00b50f0 con 0x7f0cf00c7e30 2024-04-25T10:34:29.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.082+0000 7f0cf5e6e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f0cf00c7c60 con 0x7f0cf00b4ce0 2024-04-25T10:34:29.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.082+0000 7f0ceffff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0cf00b4ce0 0x7f0cf015ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:29.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.082+0000 7f0cef7fe640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f0cf00c7e30 0x7f0cf00c8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:29.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.082+0000 7f0ceffff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0cf00b4ce0 0x7f0cf015ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.106:47668/0 (socket says 172.21.15.106:47668) 2024-04-25T10:34:29.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.082+0000 7f0ceffff640 1 -- 172.21.15.106:0/3243034371 learned_addr learned my addr 172.21.15.106:0/3243034371 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:34:29.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.083+0000 7f0ceffff640 1 -- 172.21.15.106:0/3243034371 >> v2:172.21.15.134:3300/0 conn(0x7f0cf00c7e30 msgr2=0x7f0cf00c8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:29.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.083+0000 7f0ceffff640 1 --2- 172.21.15.106:0/3243034371 >> v2:172.21.15.134:3300/0 conn(0x7f0cf00c7e30 0x7f0cf00c8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:29.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.083+0000 7f0ceffff640 1 -- 172.21.15.106:0/3243034371 >> v1:172.21.15.134:6789/0 conn(0x7f0cf0156ef0 legacy=0x7f0cf006d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:29.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.083+0000 7f0ceeffd640 1 -- 172.21.15.106:0/3243034371 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1242549138 0 0) 0x7f0cf00ce630 con 0x7f0cf0156ef0 2024-04-25T10:34:29.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.083+0000 7f0ceffff640 1 -- 172.21.15.106:0/3243034371 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cf00c7980 con 0x7f0cf00b4ce0 2024-04-25T10:34:29.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.083+0000 7f0cef7fe640 1 --2- 172.21.15.106:0/3243034371 >> v2:172.21.15.134:3300/0 conn(0x7f0cf00c7e30 0x7f0cf00c8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:34:29.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.084+0000 7f0ceffff640 1 --2- 172.21.15.106:0/3243034371 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0cf00b4ce0 0x7f0cf015ce30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0cf00c7de0 tx=0x7f0ce802a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cffcde8504f9c747 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:29.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.084+0000 7f0ceeffd640 1 -- 172.21.15.106:0/3243034371 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce8036070 con 0x7f0cf00b4ce0 2024-04-25T10:34:29.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.084+0000 7f0ceeffd640 1 -- 172.21.15.106:0/3243034371 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ce8031600 con 0x7f0cf00b4ce0 2024-04-25T10:34:29.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.084+0000 7f0ceeffd640 1 -- 172.21.15.106:0/3243034371 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce80319d0 con 0x7f0cf00b4ce0 2024-04-25T10:34:29.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.084+0000 7f0cf5e6e640 1 -- 172.21.15.106:0/3243034371 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0cf00b4ce0 msgr2=0x7f0cf015ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:29.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.084+0000 7f0cf5e6e640 1 --2- 172.21.15.106:0/3243034371 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0cf00b4ce0 0x7f0cf015ce30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0cf00c7de0 tx=0x7f0ce802a680 comp rx=0 tx=0).stop 2024-04-25T10:34:29.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.085+0000 7f0cf5e6e640 1 -- 172.21.15.106:0/3243034371 shutdown_connections 2024-04-25T10:34:29.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.085+0000 7f0cf5e6e640 1 --2- 172.21.15.106:0/3243034371 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0cf00b4ce0 0x7f0cf015ce30 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:29.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.085+0000 7f0cf5e6e640 1 --2- 172.21.15.106:0/3243034371 >> v2:172.21.15.134:3300/0 conn(0x7f0cf00c7e30 0x7f0cf00c8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:29.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.085+0000 7f0cf5e6e640 1 -- 172.21.15.106:0/3243034371 >> 172.21.15.106:0/3243034371 conn(0x7f0cf0068b10 msgr2=0x7f0cf00d1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:29.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.085+0000 7f0cf5e6e640 1 -- 172.21.15.106:0/3243034371 shutdown_connections 2024-04-25T10:34:29.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.085+0000 7f0cf5e6e640 1 -- 172.21.15.106:0/3243034371 wait complete. 2024-04-25T10:34:29.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.086+0000 7f0cf5e6e640 1 Processor -- start 2024-04-25T10:34:29.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.086+0000 7f0cf5e6e640 1 -- start start 2024-04-25T10:34:29.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.087+0000 7f0cf5e6e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0cf00b4ce0 0x7f0cf0157f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:29.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.087+0000 7f0cf5e6e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0cf00c7e30 0x7f0cf00c9750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:29.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.087+0000 7f0cf5e6e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0cf0156ef0 0x7f0cf01557a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:29.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.087+0000 7f0cf5e6e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf006d710 con 0x7f0cf00c7e30 2024-04-25T10:34:29.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.087+0000 7f0cf5e6e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf00c72f0 con 0x7f0cf00b4ce0 2024-04-25T10:34:29.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.087+0000 7f0cf5e6e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f0cf006d890 con 0x7f0cf0156ef0 2024-04-25T10:34:29.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.088+0000 7f0ce7fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0cf00c7e30 0x7f0cf00c9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:29.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.088+0000 7f0ce7fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0cf00c7e30 0x7f0cf00c9750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60312/0 (socket says 172.21.15.106:60312) 2024-04-25T10:34:29.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.088+0000 7f0ce7fff640 1 -- 172.21.15.106:0/3268912411 learned_addr learned my addr 172.21.15.106:0/3268912411 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:34:29.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.088+0000 7f0ceffff640 1 --2- 172.21.15.106:0/3268912411 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0cf0156ef0 0x7f0cf01557a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:29.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.088+0000 7f0cef7fe640 1 --2- 172.21.15.106:0/3268912411 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0cf00b4ce0 0x7f0cf0157f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:29.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.088+0000 7f0ce7fff640 1 -- 172.21.15.106:0/3268912411 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0cf0156ef0 msgr2=0x7f0cf01557a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:29.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.088+0000 7f0ce7fff640 1 --2- 172.21.15.106:0/3268912411 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0cf0156ef0 0x7f0cf01557a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:29.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.088+0000 7f0ce7fff640 1 -- 172.21.15.106:0/3268912411 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0cf00b4ce0 msgr2=0x7f0cf0157f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:29.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.088+0000 7f0ce7fff640 1 --2- 172.21.15.106:0/3268912411 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0cf00b4ce0 0x7f0cf0157f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:29.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.088+0000 7f0ce7fff640 1 -- 172.21.15.106:0/3268912411 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cf00d1780 con 0x7f0cf00c7e30 2024-04-25T10:34:29.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.088+0000 7f0ce7fff640 1 --2- 172.21.15.106:0/3268912411 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0cf00c7e30 0x7f0cf00c9750 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0cd800c440 tx=0x7f0cd800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:29.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.088+0000 7f0cecff9640 1 -- 172.21.15.106:0/3268912411 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd8017070 con 0x7f0cf00c7e30 2024-04-25T10:34:29.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.088+0000 7f0cecff9640 1 -- 172.21.15.106:0/3268912411 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cd8003640 con 0x7f0cf00c7e30 2024-04-25T10:34:29.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.088+0000 7f0cf5e6e640 1 -- 172.21.15.106:0/3268912411 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cf00c9c90 con 0x7f0cf00c7e30 2024-04-25T10:34:29.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.088+0000 7f0cecff9640 1 -- 172.21.15.106:0/3268912411 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd80054e0 con 0x7f0cf00c7e30 2024-04-25T10:34:29.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.088+0000 7f0cf5e6e640 1 -- 172.21.15.106:0/3268912411 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cf01f2c70 con 0x7f0cf00c7e30 2024-04-25T10:34:29.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.089+0000 7f0cf5e6e640 1 --2- 172.21.15.106:0/3268912411 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0cf01564f0 0x7f0cf00c8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:29.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.089+0000 7f0cf5e6e640 1 -- 172.21.15.106:0/3268912411 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf00ca3d0 con 0x7f0cf01564f0 2024-04-25T10:34:29.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.089+0000 7f0cef7fe640 1 --2- 172.21.15.106:0/3268912411 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0cf01564f0 0x7f0cf00c8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:29.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.089+0000 7f0cef7fe640 1 -- 172.21.15.106:0/3268912411 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0cf006a3d0 con 0x7f0cf01564f0 2024-04-25T10:34:29.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.090+0000 7f0cef7fe640 1 --2- 172.21.15.106:0/3268912411 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0cf01564f0 0x7f0cf00c8bd0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0011530 tx=0x7f0ce00119f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:29.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.090+0000 7f0cecff9640 1 -- 172.21.15.106:0/3268912411 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce000f8c0 con 0x7f0cf01564f0 2024-04-25T10:34:29.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.090+0000 7f0cecff9640 1 -- 172.21.15.106:0/3268912411 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0cd801a070 con 0x7f0cf00c7e30 2024-04-25T10:34:29.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.090+0000 7f0cecff9640 1 --2- 172.21.15.106:0/3268912411 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f0cd00743b0 0x7f0cd00768c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:29.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.091+0000 7f0ceffff640 1 --2- 172.21.15.106:0/3268912411 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f0cd00743b0 0x7f0cd00768c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:29.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.091+0000 7f0cecff9640 1 -- 172.21.15.106:0/3268912411 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f0cd8094680 con 0x7f0cf00c7e30 2024-04-25T10:34:29.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.091+0000 7f0cecff9640 1 -- 172.21.15.106:0/3268912411 <== mon.0 v2:172.21.15.106:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f0ce000fa60 con 0x7f0cf01564f0 2024-04-25T10:34:29.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.091+0000 7f0cecff9640 1 -- 172.21.15.106:0/3268912411 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0cf01564f0 msgr2=0x7f0cf00c8bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:29.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.091+0000 7f0cecff9640 1 --2- 172.21.15.106:0/3268912411 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0cf01564f0 0x7f0cf00c8bd0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0011530 tx=0x7f0ce00119f0 comp rx=0 tx=0).stop 2024-04-25T10:34:29.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.091+0000 7f0cecff9640 1 -- 172.21.15.106:0/3268912411 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0cf01564f0 msgr2=0x7f0cf00c8bd0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T10:34:29.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.091+0000 7f0cecff9640 1 --2- 172.21.15.106:0/3268912411 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0cf01564f0 0x7f0cf00c8bd0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:29.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.091+0000 7f0ceffff640 1 --2- 172.21.15.106:0/3268912411 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f0cd00743b0 0x7f0cd00768c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8033000 tx=0x7f0ce8009000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:29.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.097+0000 7f0cf5e6e640 1 --2- 172.21.15.106:0/3268912411 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0cf0155ce0 0x7f0cf01f6040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:29.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.097+0000 7f0cf5e6e640 1 -- 172.21.15.106:0/3268912411 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf00cd8b0 con 0x7f0cf0155ce0 2024-04-25T10:34:29.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.097+0000 7f0cef7fe640 1 --2- 172.21.15.106:0/3268912411 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0cf0155ce0 0x7f0cf01f6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:29.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.098+0000 7f0cef7fe640 1 -- 172.21.15.106:0/3268912411 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f0cf00ce2e0 con 0x7f0cf0155ce0 2024-04-25T10:34:29.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.098+0000 7f0cef7fe640 1 --2- 172.21.15.106:0/3268912411 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0cf0155ce0 0x7f0cf01f6040 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0011530 tx=0x7f0ce00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:29.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.098+0000 7f0cecff9640 1 -- 172.21.15.106:0/3268912411 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce00067e0 con 0x7f0cf0155ce0 2024-04-25T10:34:29.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.098+0000 7f0cecff9640 1 -- 172.21.15.106:0/3268912411 <== mon.0 v2:172.21.15.106:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4559 (secure 0 0 0) 0x7f0ce00067c0 con 0x7f0cf0155ce0 2024-04-25T10:34:29.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.098+0000 7f0cecff9640 1 -- 172.21.15.106:0/3268912411 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0cf0155ce0 msgr2=0x7f0cf01f6040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:29.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.098+0000 7f0cecff9640 1 --2- 172.21.15.106:0/3268912411 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0cf0155ce0 0x7f0cf01f6040 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0011530 tx=0x7f0ce00046c0 comp rx=0 tx=0).stop 2024-04-25T10:34:29.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.098+0000 7f0cecff9640 1 -- 172.21.15.106:0/3268912411 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0cf0155ce0 msgr2=0x7f0cf01f6040 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T10:34:29.101 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.098+0000 7f0cecff9640 1 --2- 172.21.15.106:0/3268912411 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0cf0155ce0 0x7f0cf01f6040 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:29.101 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-25T10:34:29.101 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "a", 2024-04-25T10:34:29.101 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 0, 2024-04-25T10:34:29.101 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "leader", 2024-04-25T10:34:29.102 INFO:teuthology.orchestra.run.smithi106.stdout: "election_epoch": 28, 2024-04-25T10:34:29.102 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 7189, 2024-04-25T10:34:29.102 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum": [ 2024-04-25T10:34:29.102 INFO:teuthology.orchestra.run.smithi106.stdout: 0, 2024-04-25T10:34:29.102 INFO:teuthology.orchestra.run.smithi106.stdout: 1, 2024-04-25T10:34:29.102 INFO:teuthology.orchestra.run.smithi106.stdout: 2 2024-04-25T10:34:29.102 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:34:29.102 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_age": 0, 2024-04-25T10:34:29.102 INFO:teuthology.orchestra.run.smithi106.stdout: "features": { 2024-04-25T10:34:29.102 INFO:teuthology.orchestra.run.smithi106.stdout: "required_con": "2450521708007866372", 2024-04-25T10:34:29.102 INFO:teuthology.orchestra.run.smithi106.stdout: "required_mon": [ 2024-04-25T10:34:29.102 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:34:29.102 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:34:29.102 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:34:29.103 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:34:29.103 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:34:29.103 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:34:29.103 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:34:29.103 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:34:29.103 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:34:29.103 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:34:29.103 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:34:29.103 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:34:29.103 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_con": "4540701547738038271", 2024-04-25T10:34:29.103 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_mon": [ 2024-04-25T10:34:29.103 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:34:29.103 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:34:29.103 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:34:29.104 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:34:29.104 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:34:29.104 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:34:29.104 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:34:29.104 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:34:29.104 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:34:29.104 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:34:29.104 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:34:29.104 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:34:29.104 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:34:29.104 INFO:teuthology.orchestra.run.smithi106.stdout: "outside_quorum": [], 2024-04-25T10:34:29.104 INFO:teuthology.orchestra.run.smithi106.stdout: "extra_probe_peers": [], 2024-04-25T10:34:29.104 INFO:teuthology.orchestra.run.smithi106.stdout: "sync_provider": [], 2024-04-25T10:34:29.104 INFO:teuthology.orchestra.run.smithi106.stdout: "monmap": { 2024-04-25T10:34:29.104 INFO:teuthology.orchestra.run.smithi106.stdout: "epoch": 1, 2024-04-25T10:34:29.105 INFO:teuthology.orchestra.run.smithi106.stdout: "fsid": "3ee77985-e8e4-41cf-81e9-7c774cf7891f", 2024-04-25T10:34:29.105 INFO:teuthology.orchestra.run.smithi106.stdout: "modified": "2024-04-25T10:29:33.446228Z", 2024-04-25T10:34:29.105 INFO:teuthology.orchestra.run.smithi106.stdout: "created": "2024-04-25T10:29:33.446228Z", 2024-04-25T10:34:29.105 INFO:teuthology.orchestra.run.smithi106.stdout: "min_mon_release": 19, 2024-04-25T10:34:29.105 INFO:teuthology.orchestra.run.smithi106.stdout: "min_mon_release_name": "squid", 2024-04-25T10:34:29.105 INFO:teuthology.orchestra.run.smithi106.stdout: "election_strategy": 1, 2024-04-25T10:34:29.105 INFO:teuthology.orchestra.run.smithi106.stdout: "disallowed_leaders: ": "", 2024-04-25T10:34:29.105 INFO:teuthology.orchestra.run.smithi106.stdout: "stretch_mode": false, 2024-04-25T10:34:29.105 INFO:teuthology.orchestra.run.smithi106.stdout: "tiebreaker_mon": "", 2024-04-25T10:34:29.105 INFO:teuthology.orchestra.run.smithi106.stdout: "removed_ranks: ": "", 2024-04-25T10:34:29.105 INFO:teuthology.orchestra.run.smithi106.stdout: "features": { 2024-04-25T10:34:29.105 INFO:teuthology.orchestra.run.smithi106.stdout: "persistent": [ 2024-04-25T10:34:29.105 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:34:29.105 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:34:29.105 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:34:29.106 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:34:29.106 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:34:29.106 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:34:29.106 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:34:29.106 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:34:29.106 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:34:29.106 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:34:29.106 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:34:29.106 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:34:29.106 INFO:teuthology.orchestra.run.smithi106.stdout: "optional": [] 2024-04-25T10:34:29.106 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:34:29.106 INFO:teuthology.orchestra.run.smithi106.stdout: "mons": [ 2024-04-25T10:34:29.106 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:34:29.107 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 0, 2024-04-25T10:34:29.107 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "a", 2024-04-25T10:34:29.107 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:34:29.107 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:34:29.107 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:34:29.107 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:34:29.107 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:3300", 2024-04-25T10:34:29.107 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:34:29.107 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:34:29.107 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:34:29.107 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:34:29.107 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:6789", 2024-04-25T10:34:29.107 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:34:29.107 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:34:29.107 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:34:29.108 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:34:29.108 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:6789/0", 2024-04-25T10:34:29.108 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.106:6789/0", 2024-04-25T10:34:29.108 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:34:29.108 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:34:29.108 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:34:29.108 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:34:29.108 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:34:29.108 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 1, 2024-04-25T10:34:29.108 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "b", 2024-04-25T10:34:29.108 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:34:29.108 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:34:29.108 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:34:29.108 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:34:29.108 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:3300", 2024-04-25T10:34:29.109 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:34:29.109 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:34:29.109 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:34:29.109 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:34:29.109 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6789", 2024-04-25T10:34:29.109 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:34:29.109 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:34:29.109 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:34:29.109 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:34:29.109 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6789/0", 2024-04-25T10:34:29.109 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.134:6789/0", 2024-04-25T10:34:29.109 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:34:29.109 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:34:29.109 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:34:29.109 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:34:29.109 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:34:29.110 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 2, 2024-04-25T10:34:29.110 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "c", 2024-04-25T10:34:29.110 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:34:29.110 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:34:29.110 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:34:29.110 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:34:29.110 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:3301", 2024-04-25T10:34:29.110 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:34:29.110 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:34:29.110 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:34:29.110 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:34:29.110 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6790", 2024-04-25T10:34:29.110 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:34:29.110 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:34:29.110 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:34:29.111 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:34:29.111 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6790/0", 2024-04-25T10:34:29.111 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.134:6790/0", 2024-04-25T10:34:29.111 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:34:29.111 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:34:29.111 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:34:29.111 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:34:29.111 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:34:29.111 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:34:29.111 INFO:teuthology.orchestra.run.smithi106.stdout: "feature_map": { 2024-04-25T10:34:29.111 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": [ 2024-04-25T10:34:29.111 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:34:29.111 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:34:29.111 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:34:29.111 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 1 2024-04-25T10:34:29.112 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:34:29.112 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:34:29.112 INFO:teuthology.orchestra.run.smithi106.stdout: "client": [ 2024-04-25T10:34:29.112 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:34:29.112 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:34:29.112 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:34:29.112 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 2 2024-04-25T10:34:29.112 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:34:29.112 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:34:29.112 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:34:29.112 INFO:teuthology.orchestra.run.smithi106.stdout: "stretch_mode": false 2024-04-25T10:34:29.112 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-25T10:34:29.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.098+0000 7f0cf5e6e640 1 -- 172.21.15.106:0/3268912411 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f0cd00743b0 msgr2=0x7f0cd00768c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:29.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.098+0000 7f0cf5e6e640 1 --2- 172.21.15.106:0/3268912411 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f0cd00743b0 0x7f0cd00768c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8033000 tx=0x7f0ce8009000 comp rx=0 tx=0).stop 2024-04-25T10:34:29.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.098+0000 7f0cf5e6e640 1 -- 172.21.15.106:0/3268912411 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0cf00c7e30 msgr2=0x7f0cf00c9750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:29.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.098+0000 7f0cf5e6e640 1 --2- 172.21.15.106:0/3268912411 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0cf00c7e30 0x7f0cf00c9750 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0cd800c440 tx=0x7f0cd800c900 comp rx=0 tx=0).stop 2024-04-25T10:34:29.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.098+0000 7f0ceffff640 1 -- 172.21.15.106:0/3268912411 reap_dead start 2024-04-25T10:34:29.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.098+0000 7f0ceffff640 1 -- 172.21.15.106:0/3268912411 reap_dead start 2024-04-25T10:34:29.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.099+0000 7f0cf5e6e640 1 -- 172.21.15.106:0/3268912411 shutdown_connections 2024-04-25T10:34:29.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.099+0000 7f0cf5e6e640 1 -- 172.21.15.106:0/3268912411 >> 172.21.15.106:0/3268912411 conn(0x7f0cf0068b10 msgr2=0x7f0cf00b6600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:29.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.099+0000 7f0cf5e6e640 1 -- 172.21.15.106:0/3268912411 shutdown_connections 2024-04-25T10:34:29.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.099+0000 7f0cf5e6e640 1 -- 172.21.15.106:0/3268912411 wait complete. 2024-04-25T10:34:29.114 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T10:34:29.115 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-04-25T10:34:29.115 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-25T10:34:29.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.222+0000 7ff7c8157640 1 Processor -- start 2024-04-25T10:34:29.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.222+0000 7ff7c8157640 1 -- start start 2024-04-25T10:34:29.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.224+0000 7ff7c8157640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff7c00cb2b0 0x7ff7c00cce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:29.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.224+0000 7ff7c8157640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff7c0154ac0 0x7ff7c00cd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:29.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.224+0000 7ff7c8157640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7c015b8c0 con 0x7ff7c00cc600 2024-04-25T10:34:29.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.224+0000 7ff7c8157640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7ff7c015eea0 con 0x7ff7c00cb2b0 2024-04-25T10:34:29.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.224+0000 7ff7c8157640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7ff7c015b5e0 con 0x7ff7c0154ac0 2024-04-25T10:34:29.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.224+0000 7ff7c56f2640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff7c00cb2b0 0x7ff7c00cce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:29.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.224+0000 7ff7c5ef3640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7ff7c00cc600 0x7ff7c00c5390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:46748/0 (socket says 172.21.15.106:46748) 2024-04-25T10:34:29.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.224+0000 7ff7c5ef3640 1 -- 172.21.15.106:0/1750302365 learned_addr learned my addr 172.21.15.106:0/1750302365 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:34:29.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.224+0000 7ff7c66f4640 1 --2- 172.21.15.106:0/1750302365 >> v2:172.21.15.134:3300/0 conn(0x7ff7c0154ac0 0x7ff7c00cd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:29.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.224+0000 7ff7c56f2640 1 -- 172.21.15.106:0/1750302365 >> v2:172.21.15.134:3300/0 conn(0x7ff7c0154ac0 msgr2=0x7ff7c00cd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:29.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.224+0000 7ff7c56f2640 1 --2- 172.21.15.106:0/1750302365 >> v2:172.21.15.134:3300/0 conn(0x7ff7c0154ac0 0x7ff7c00cd3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:29.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.224+0000 7ff7c56f2640 1 -- 172.21.15.106:0/1750302365 >> v1:172.21.15.106:6789/0 conn(0x7ff7c00cc600 legacy=0x7ff7c00c5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:29.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.224+0000 7ff7c56f2640 1 -- 172.21.15.106:0/1750302365 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7c015e0e0 con 0x7ff7c00cb2b0 2024-04-25T10:34:29.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.225+0000 7ff7c56f2640 1 --2- 172.21.15.106:0/1750302365 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff7c00cb2b0 0x7ff7c00cce90 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff7c00b60e0 tx=0x7ff7b002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63b4eef1e89454d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:29.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.225+0000 7ff7c4ef1640 1 -- 172.21.15.106:0/1750302365 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b0037070 con 0x7ff7c00cb2b0 2024-04-25T10:34:29.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.225+0000 7ff7c4ef1640 1 -- 172.21.15.106:0/1750302365 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7b0032440 con 0x7ff7c00cb2b0 2024-04-25T10:34:29.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.225+0000 7ff7c4ef1640 1 -- 172.21.15.106:0/1750302365 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b0032810 con 0x7ff7c00cb2b0 2024-04-25T10:34:29.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.226+0000 7ff7c8157640 1 -- 172.21.15.106:0/1750302365 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff7c00cb2b0 msgr2=0x7ff7c00cce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:29.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.226+0000 7ff7c8157640 1 --2- 172.21.15.106:0/1750302365 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff7c00cb2b0 0x7ff7c00cce90 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff7c00b60e0 tx=0x7ff7b002a1a0 comp rx=0 tx=0).stop 2024-04-25T10:34:29.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.226+0000 7ff7c8157640 1 -- 172.21.15.106:0/1750302365 shutdown_connections 2024-04-25T10:34:29.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.226+0000 7ff7c8157640 1 --2- 172.21.15.106:0/1750302365 >> v2:172.21.15.134:3300/0 conn(0x7ff7c0154ac0 0x7ff7c00cd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:29.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.226+0000 7ff7c8157640 1 --2- 172.21.15.106:0/1750302365 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff7c00cb2b0 0x7ff7c00cce90 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:29.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.226+0000 7ff7c8157640 1 -- 172.21.15.106:0/1750302365 >> 172.21.15.106:0/1750302365 conn(0x7ff7c00727d0 msgr2=0x7ff7c0072c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:29.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.226+0000 7ff7c8157640 1 -- 172.21.15.106:0/1750302365 shutdown_connections 2024-04-25T10:34:29.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.226+0000 7ff7c8157640 1 -- 172.21.15.106:0/1750302365 wait complete. 2024-04-25T10:34:29.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.228+0000 7ff7c8157640 1 Processor -- start 2024-04-25T10:34:29.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.228+0000 7ff7c8157640 1 -- start start 2024-04-25T10:34:29.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.229+0000 7ff7c8157640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff7c00cb2b0 0x7ff7c006dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:29.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.229+0000 7ff7c8157640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff7c00cc600 0x7ff7c0151360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:29.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.229+0000 7ff7c8157640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7c0154ac0 0x7ff7c0158930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:29.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.229+0000 7ff7c8157640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c015e980 con 0x7ff7c00cc600 2024-04-25T10:34:29.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.229+0000 7ff7c8157640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c015e6a0 con 0x7ff7c0154ac0 2024-04-25T10:34:29.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.229+0000 7ff7c8157640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff7c00cbff0 con 0x7ff7c00cb2b0 2024-04-25T10:34:29.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.229+0000 7ff7c56f2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff7c00cc600 0x7ff7c0151360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:29.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.229+0000 7ff7c56f2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff7c00cc600 0x7ff7c0151360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60346/0 (socket says 172.21.15.106:60346) 2024-04-25T10:34:29.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.229+0000 7ff7c56f2640 1 -- 172.21.15.106:0/3093305975 learned_addr learned my addr 172.21.15.106:0/3093305975 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:34:29.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.229+0000 7ff7c66f4640 1 --2- 172.21.15.106:0/3093305975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7c0154ac0 0x7ff7c0158930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:29.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.229+0000 7ff7c56f2640 1 -- 172.21.15.106:0/3093305975 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff7c00cb2b0 msgr2=0x7ff7c006dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:29.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.229+0000 7ff7c5ef3640 1 --2- 172.21.15.106:0/3093305975 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff7c00cb2b0 0x7ff7c006dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:29.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.229+0000 7ff7c56f2640 1 --2- 172.21.15.106:0/3093305975 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff7c00cb2b0 0x7ff7c006dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:29.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.229+0000 7ff7c56f2640 1 -- 172.21.15.106:0/3093305975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7c0154ac0 msgr2=0x7ff7c0158930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:29.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.229+0000 7ff7c56f2640 1 --2- 172.21.15.106:0/3093305975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7c0154ac0 0x7ff7c0158930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:29.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.229+0000 7ff7c56f2640 1 -- 172.21.15.106:0/3093305975 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7c01f3a10 con 0x7ff7c00cc600 2024-04-25T10:34:29.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.229+0000 7ff7c66f4640 1 --2- 172.21.15.106:0/3093305975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7c0154ac0 0x7ff7c0158930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:34:29.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.229+0000 7ff7c5ef3640 1 --2- 172.21.15.106:0/3093305975 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff7c00cb2b0 0x7ff7c006dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:34:29.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.230+0000 7ff7c56f2640 1 --2- 172.21.15.106:0/3093305975 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff7c00cc600 0x7ff7c0151360 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff7b0009890 tx=0x7ff7b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:29.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.230+0000 7ff7b67fc640 1 -- 172.21.15.106:0/3093305975 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b0037040 con 0x7ff7c00cc600 2024-04-25T10:34:29.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.230+0000 7ff7b67fc640 1 -- 172.21.15.106:0/3093305975 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7b0042070 con 0x7ff7c00cc600 2024-04-25T10:34:29.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.230+0000 7ff7b67fc640 1 -- 172.21.15.106:0/3093305975 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b0030bd0 con 0x7ff7c00cc600 2024-04-25T10:34:29.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.230+0000 7ff7c8157640 1 -- 172.21.15.106:0/3093305975 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7c006e2d0 con 0x7ff7c00cc600 2024-04-25T10:34:29.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.230+0000 7ff7c8157640 1 -- 172.21.15.106:0/3093305975 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7c015dbc0 con 0x7ff7c00cc600 2024-04-25T10:34:29.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.230+0000 7ff7c8157640 1 --2- 172.21.15.106:0/3093305975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7c006d6d0 0x7ff7c0158e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:29.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.230+0000 7ff7c8157640 1 -- 172.21.15.106:0/3093305975 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c006e7a0 con 0x7ff7c006d6d0 2024-04-25T10:34:29.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.231+0000 7ff7c5ef3640 1 --2- 172.21.15.106:0/3093305975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7c006d6d0 0x7ff7c0158e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:29.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.231+0000 7ff7c5ef3640 1 -- 172.21.15.106:0/3093305975 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff7c01f3760 con 0x7ff7c006d6d0 2024-04-25T10:34:29.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.231+0000 7ff7b67fc640 1 -- 172.21.15.106:0/3093305975 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff7b0047460 con 0x7ff7c00cc600 2024-04-25T10:34:29.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.231+0000 7ff7c5ef3640 1 --2- 172.21.15.106:0/3093305975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7c006d6d0 0x7ff7c0158e70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff7c00749e0 tx=0x7ff7bc00d880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:29.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.232+0000 7ff7b67fc640 1 --2- 172.21.15.106:0/3093305975 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7ff7a40744e0 0x7ff7a40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:29.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.232+0000 7ff7c66f4640 1 --2- 172.21.15.106:0/3093305975 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7ff7a40744e0 0x7ff7a40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:29.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.232+0000 7ff7b67fc640 1 -- 172.21.15.106:0/3093305975 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7ff7b00b5e30 con 0x7ff7c00cc600 2024-04-25T10:34:29.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.232+0000 7ff7b67fc640 1 -- 172.21.15.106:0/3093305975 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7bc005ce0 con 0x7ff7c006d6d0 2024-04-25T10:34:29.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.232+0000 7ff7b67fc640 1 -- 172.21.15.106:0/3093305975 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7ff7bc005020 con 0x7ff7c006d6d0 2024-04-25T10:34:29.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.232+0000 7ff7b67fc640 1 -- 172.21.15.106:0/3093305975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7c006d6d0 msgr2=0x7ff7c0158e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:29.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.232+0000 7ff7b67fc640 1 --2- 172.21.15.106:0/3093305975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7c006d6d0 0x7ff7c0158e70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff7c00749e0 tx=0x7ff7bc00d880 comp rx=0 tx=0).stop 2024-04-25T10:34:29.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.232+0000 7ff7b67fc640 1 -- 172.21.15.106:0/3093305975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7c006d6d0 msgr2=0x7ff7c0158e70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T10:34:29.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.232+0000 7ff7b67fc640 1 --2- 172.21.15.106:0/3093305975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7c006d6d0 0x7ff7c0158e70 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:29.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.232+0000 7ff7c66f4640 1 --2- 172.21.15.106:0/3093305975 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7ff7a40744e0 0x7ff7a40769f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff7c00b6250 tx=0x7ff7b800b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:29.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.238+0000 7ff7c8157640 1 --2- 172.21.15.106:0/3093305975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7c01518a0 0x7ff7c01f5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:29.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.238+0000 7ff7c8157640 1 -- 172.21.15.106:0/3093305975 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c01f3500 con 0x7ff7c01518a0 2024-04-25T10:34:29.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.239+0000 7ff7c5ef3640 1 --2- 172.21.15.106:0/3093305975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7c01518a0 0x7ff7c01f5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:29.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.239+0000 7ff7c5ef3640 1 -- 172.21.15.106:0/3093305975 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff7c006a3d0 con 0x7ff7c01518a0 2024-04-25T10:34:29.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.239+0000 7ff7c5ef3640 1 --2- 172.21.15.106:0/3093305975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7c01518a0 0x7ff7c01f5ff0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff7c00749e0 tx=0x7ff7bc000ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:29.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.240+0000 7ff7b67fc640 1 -- 172.21.15.106:0/3093305975 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7bc01a2c0 con 0x7ff7c01518a0 2024-04-25T10:34:29.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.240+0000 7ff7b67fc640 1 -- 172.21.15.106:0/3093305975 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5068 (secure 0 0 0) 0x7ff7bc01a2c0 con 0x7ff7c01518a0 2024-04-25T10:34:29.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.240+0000 7ff7b67fc640 1 -- 172.21.15.106:0/3093305975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7c01518a0 msgr2=0x7ff7c01f5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:29.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.240+0000 7ff7b67fc640 1 --2- 172.21.15.106:0/3093305975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7c01518a0 0x7ff7c01f5ff0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff7c00749e0 tx=0x7ff7bc000ed0 comp rx=0 tx=0).stop 2024-04-25T10:34:29.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.240+0000 7ff7b67fc640 1 -- 172.21.15.106:0/3093305975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7c01518a0 msgr2=0x7ff7c01f5ff0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T10:34:29.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.240+0000 7ff7b67fc640 1 --2- 172.21.15.106:0/3093305975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7c01518a0 0x7ff7c01f5ff0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:29.243 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-25T10:34:29.243 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "b", 2024-04-25T10:34:29.243 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 1, 2024-04-25T10:34:29.243 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "peon", 2024-04-25T10:34:29.243 INFO:teuthology.orchestra.run.smithi106.stdout: "election_epoch": 28, 2024-04-25T10:34:29.243 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 55764, 2024-04-25T10:34:29.243 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum": [ 2024-04-25T10:34:29.243 INFO:teuthology.orchestra.run.smithi106.stdout: 0, 2024-04-25T10:34:29.243 INFO:teuthology.orchestra.run.smithi106.stdout: 1, 2024-04-25T10:34:29.244 INFO:teuthology.orchestra.run.smithi106.stdout: 2 2024-04-25T10:34:29.244 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:34:29.244 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_age": 1, 2024-04-25T10:34:29.244 INFO:teuthology.orchestra.run.smithi106.stdout: "features": { 2024-04-25T10:34:29.244 INFO:teuthology.orchestra.run.smithi106.stdout: "required_con": "2450521708007866372", 2024-04-25T10:34:29.244 INFO:teuthology.orchestra.run.smithi106.stdout: "required_mon": [ 2024-04-25T10:34:29.244 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:34:29.244 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:34:29.244 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:34:29.244 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:34:29.244 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:34:29.244 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:34:29.244 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:34:29.244 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:34:29.244 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:34:29.245 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:34:29.245 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:34:29.245 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:34:29.245 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_con": "4540701547738038271", 2024-04-25T10:34:29.245 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_mon": [ 2024-04-25T10:34:29.245 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:34:29.245 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:34:29.245 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:34:29.245 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:34:29.245 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:34:29.245 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:34:29.245 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:34:29.245 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:34:29.245 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:34:29.246 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:34:29.246 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:34:29.246 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:34:29.246 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:34:29.246 INFO:teuthology.orchestra.run.smithi106.stdout: "outside_quorum": [], 2024-04-25T10:34:29.246 INFO:teuthology.orchestra.run.smithi106.stdout: "extra_probe_peers": [], 2024-04-25T10:34:29.246 INFO:teuthology.orchestra.run.smithi106.stdout: "sync_provider": [], 2024-04-25T10:34:29.246 INFO:teuthology.orchestra.run.smithi106.stdout: "monmap": { 2024-04-25T10:34:29.246 INFO:teuthology.orchestra.run.smithi106.stdout: "epoch": 1, 2024-04-25T10:34:29.246 INFO:teuthology.orchestra.run.smithi106.stdout: "fsid": "3ee77985-e8e4-41cf-81e9-7c774cf7891f", 2024-04-25T10:34:29.246 INFO:teuthology.orchestra.run.smithi106.stdout: "modified": "2024-04-25T10:29:33.446228Z", 2024-04-25T10:34:29.246 INFO:teuthology.orchestra.run.smithi106.stdout: "created": "2024-04-25T10:29:33.446228Z", 2024-04-25T10:34:29.246 INFO:teuthology.orchestra.run.smithi106.stdout: "min_mon_release": 19, 2024-04-25T10:34:29.247 INFO:teuthology.orchestra.run.smithi106.stdout: "min_mon_release_name": "squid", 2024-04-25T10:34:29.247 INFO:teuthology.orchestra.run.smithi106.stdout: "election_strategy": 1, 2024-04-25T10:34:29.247 INFO:teuthology.orchestra.run.smithi106.stdout: "disallowed_leaders: ": "", 2024-04-25T10:34:29.247 INFO:teuthology.orchestra.run.smithi106.stdout: "stretch_mode": false, 2024-04-25T10:34:29.247 INFO:teuthology.orchestra.run.smithi106.stdout: "tiebreaker_mon": "", 2024-04-25T10:34:29.247 INFO:teuthology.orchestra.run.smithi106.stdout: "removed_ranks: ": "", 2024-04-25T10:34:29.247 INFO:teuthology.orchestra.run.smithi106.stdout: "features": { 2024-04-25T10:34:29.247 INFO:teuthology.orchestra.run.smithi106.stdout: "persistent": [ 2024-04-25T10:34:29.247 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:34:29.247 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:34:29.247 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:34:29.247 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:34:29.247 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:34:29.247 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:34:29.247 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:34:29.247 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:34:29.248 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:34:29.248 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:34:29.248 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:34:29.248 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:34:29.248 INFO:teuthology.orchestra.run.smithi106.stdout: "optional": [] 2024-04-25T10:34:29.248 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:34:29.248 INFO:teuthology.orchestra.run.smithi106.stdout: "mons": [ 2024-04-25T10:34:29.248 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:34:29.248 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 0, 2024-04-25T10:34:29.248 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "a", 2024-04-25T10:34:29.248 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:34:29.248 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:34:29.248 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:34:29.248 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:34:29.249 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:3300", 2024-04-25T10:34:29.249 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:34:29.249 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:34:29.249 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:34:29.249 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:34:29.249 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:6789", 2024-04-25T10:34:29.249 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:34:29.249 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:34:29.249 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:34:29.249 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:34:29.249 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:6789/0", 2024-04-25T10:34:29.249 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.106:6789/0", 2024-04-25T10:34:29.249 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:34:29.249 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:34:29.249 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:34:29.250 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:34:29.250 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:34:29.250 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 1, 2024-04-25T10:34:29.250 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "b", 2024-04-25T10:34:29.250 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:34:29.250 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:34:29.250 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:34:29.250 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:34:29.250 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:3300", 2024-04-25T10:34:29.250 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:34:29.250 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:34:29.250 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:34:29.250 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:34:29.251 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6789", 2024-04-25T10:34:29.251 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:34:29.251 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:34:29.251 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:34:29.251 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:34:29.251 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6789/0", 2024-04-25T10:34:29.251 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.134:6789/0", 2024-04-25T10:34:29.251 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:34:29.251 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:34:29.251 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:34:29.251 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:34:29.251 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:34:29.251 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 2, 2024-04-25T10:34:29.251 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "c", 2024-04-25T10:34:29.252 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:34:29.252 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:34:29.252 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:34:29.252 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:34:29.252 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:3301", 2024-04-25T10:34:29.252 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:34:29.252 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:34:29.252 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:34:29.252 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:34:29.252 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6790", 2024-04-25T10:34:29.252 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:34:29.252 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:34:29.252 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:34:29.252 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:34:29.252 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6790/0", 2024-04-25T10:34:29.253 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.134:6790/0", 2024-04-25T10:34:29.253 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:34:29.253 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:34:29.253 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:34:29.253 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:34:29.253 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:34:29.253 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:34:29.253 INFO:teuthology.orchestra.run.smithi106.stdout: "feature_map": { 2024-04-25T10:34:29.253 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": [ 2024-04-25T10:34:29.253 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:34:29.253 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:34:29.253 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:34:29.253 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 1 2024-04-25T10:34:29.253 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:34:29.253 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:34:29.254 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": [ 2024-04-25T10:34:29.254 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:34:29.254 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:34:29.254 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:34:29.254 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 2 2024-04-25T10:34:29.254 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:34:29.254 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:34:29.254 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": [ 2024-04-25T10:34:29.254 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:34:29.254 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:34:29.254 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:34:29.254 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 3 2024-04-25T10:34:29.254 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:34:29.254 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:34:29.254 INFO:teuthology.orchestra.run.smithi106.stdout: "client": [ 2024-04-25T10:34:29.255 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:34:29.255 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:34:29.255 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:34:29.255 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 3 2024-04-25T10:34:29.255 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:34:29.255 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:34:29.255 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": [ 2024-04-25T10:34:29.255 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:34:29.255 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:34:29.255 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:34:29.255 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 1 2024-04-25T10:34:29.255 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:34:29.255 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:34:29.255 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:34:29.255 INFO:teuthology.orchestra.run.smithi106.stdout: "stretch_mode": false 2024-04-25T10:34:29.256 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-25T10:34:29.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.240+0000 7ff7c8157640 1 -- 172.21.15.106:0/3093305975 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7ff7a40744e0 msgr2=0x7ff7a40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:29.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.240+0000 7ff7c8157640 1 --2- 172.21.15.106:0/3093305975 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7ff7a40744e0 0x7ff7a40769f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff7c00b6250 tx=0x7ff7b800b040 comp rx=0 tx=0).stop 2024-04-25T10:34:29.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.240+0000 7ff7c8157640 1 -- 172.21.15.106:0/3093305975 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff7c00cc600 msgr2=0x7ff7c0151360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:29.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.240+0000 7ff7c8157640 1 --2- 172.21.15.106:0/3093305975 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff7c00cc600 0x7ff7c0151360 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff7b0009890 tx=0x7ff7b00046c0 comp rx=0 tx=0).stop 2024-04-25T10:34:29.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.240+0000 7ff7c66f4640 1 -- 172.21.15.106:0/3093305975 reap_dead start 2024-04-25T10:34:29.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.240+0000 7ff7c66f4640 1 -- 172.21.15.106:0/3093305975 reap_dead start 2024-04-25T10:34:29.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.240+0000 7ff7c8157640 1 -- 172.21.15.106:0/3093305975 shutdown_connections 2024-04-25T10:34:29.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.240+0000 7ff7c8157640 1 -- 172.21.15.106:0/3093305975 >> 172.21.15.106:0/3093305975 conn(0x7ff7c00727d0 msgr2=0x7ff7c00c65c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:29.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.240+0000 7ff7c8157640 1 -- 172.21.15.106:0/3093305975 shutdown_connections 2024-04-25T10:34:29.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.240+0000 7ff7c8157640 1 -- 172.21.15.106:0/3093305975 wait complete. 2024-04-25T10:34:29.258 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T10:34:29.258 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-04-25T10:34:29.258 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-25T10:34:29.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.364+0000 7ff9f68af640 1 Processor -- start 2024-04-25T10:34:29.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.364+0000 7ff9f68af640 1 -- start start 2024-04-25T10:34:29.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.365+0000 7ff9f68af640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff9f0156f70 0x7ff9f015ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:29.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.365+0000 7ff9f68af640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff9f015d340 0x7ff9f0167680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:29.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.365+0000 7ff9f68af640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9f0073980 con 0x7ff9f00ca170 2024-04-25T10:34:29.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.365+0000 7ff9f68af640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7ff9f0073f00 con 0x7ff9f0156f70 2024-04-25T10:34:29.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.365+0000 7ff9f68af640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7ff9f0074990 con 0x7ff9f015d340 2024-04-25T10:34:29.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.365+0000 7ff9ef7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff9f0156f70 0x7ff9f015ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:29.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.365+0000 7ff9effff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7ff9f00ca170 0x7ff9f006c5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:46750/0 (socket says 172.21.15.106:46750) 2024-04-25T10:34:29.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.365+0000 7ff9effff640 1 -- 172.21.15.106:0/2768719285 learned_addr learned my addr 172.21.15.106:0/2768719285 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:34:29.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.365+0000 7ff9ef7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff9f0156f70 0x7ff9f015ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60348/0 (socket says 172.21.15.106:60348) 2024-04-25T10:34:29.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.365+0000 7ff9f4e4c640 1 --2- 172.21.15.106:0/2768719285 >> v2:172.21.15.134:3300/0 conn(0x7ff9f015d340 0x7ff9f0167680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:29.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.366+0000 7ff9ef7fe640 1 -- 172.21.15.106:0/2768719285 >> v2:172.21.15.134:3300/0 conn(0x7ff9f015d340 msgr2=0x7ff9f0167680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:29.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.366+0000 7ff9ef7fe640 1 --2- 172.21.15.106:0/2768719285 >> v2:172.21.15.134:3300/0 conn(0x7ff9f015d340 0x7ff9f0167680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:29.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.366+0000 7ff9ef7fe640 1 -- 172.21.15.106:0/2768719285 >> v1:172.21.15.106:6789/0 conn(0x7ff9f00ca170 legacy=0x7ff9f006c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:29.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.366+0000 7ff9eeffd640 1 -- 172.21.15.106:0/2768719285 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 137560999 0 0) 0x7ff9f0073980 con 0x7ff9f00ca170 2024-04-25T10:34:29.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.366+0000 7ff9ef7fe640 1 -- 172.21.15.106:0/2768719285 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9f0160ea0 con 0x7ff9f0156f70 2024-04-25T10:34:29.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.366+0000 7ff9ef7fe640 1 --2- 172.21.15.106:0/2768719285 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9f0156f70 0x7ff9f015ce00 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff9f006d600 tx=0x7ff9e402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e0cd3bd40109459 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:29.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.366+0000 7ff9eeffd640 1 -- 172.21.15.106:0/2768719285 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e4037070 con 0x7ff9f0156f70 2024-04-25T10:34:29.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.367+0000 7ff9eeffd640 1 -- 172.21.15.106:0/2768719285 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9e4032440 con 0x7ff9f0156f70 2024-04-25T10:34:29.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.367+0000 7ff9eeffd640 1 -- 172.21.15.106:0/2768719285 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e4032810 con 0x7ff9f0156f70 2024-04-25T10:34:29.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.367+0000 7ff9f68af640 1 -- 172.21.15.106:0/2768719285 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9f0156f70 msgr2=0x7ff9f015ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:29.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.367+0000 7ff9f68af640 1 --2- 172.21.15.106:0/2768719285 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9f0156f70 0x7ff9f015ce00 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff9f006d600 tx=0x7ff9e402a230 comp rx=0 tx=0).stop 2024-04-25T10:34:29.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.367+0000 7ff9f68af640 1 -- 172.21.15.106:0/2768719285 shutdown_connections 2024-04-25T10:34:29.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.367+0000 7ff9f68af640 1 --2- 172.21.15.106:0/2768719285 >> v2:172.21.15.134:3300/0 conn(0x7ff9f015d340 0x7ff9f0167680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:29.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.367+0000 7ff9f68af640 1 --2- 172.21.15.106:0/2768719285 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9f0156f70 0x7ff9f015ce00 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:29.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.367+0000 7ff9f68af640 1 -- 172.21.15.106:0/2768719285 >> 172.21.15.106:0/2768719285 conn(0x7ff9f00cef20 msgr2=0x7ff9f00cf330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:29.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.367+0000 7ff9f68af640 1 -- 172.21.15.106:0/2768719285 shutdown_connections 2024-04-25T10:34:29.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.367+0000 7ff9f68af640 1 -- 172.21.15.106:0/2768719285 wait complete. 2024-04-25T10:34:29.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.369+0000 7ff9f68af640 1 Processor -- start 2024-04-25T10:34:29.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.369+0000 7ff9f68af640 1 -- start start 2024-04-25T10:34:29.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.370+0000 7ff9f68af640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9f00ca170 0x7ff9f01f2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:29.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.370+0000 7ff9f68af640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9f0156f70 0x7ff9f0071680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:29.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.370+0000 7ff9f68af640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9f015d340 0x7ff9f014e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:29.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.370+0000 7ff9f68af640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f0073c20 con 0x7ff9f00ca170 2024-04-25T10:34:29.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.370+0000 7ff9f68af640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f0074640 con 0x7ff9f015d340 2024-04-25T10:34:29.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.370+0000 7ff9f68af640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff9f0161140 con 0x7ff9f0156f70 2024-04-25T10:34:29.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.370+0000 7ff9effff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9f00ca170 0x7ff9f01f2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:29.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.370+0000 7ff9effff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9f00ca170 0x7ff9f01f2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60352/0 (socket says 172.21.15.106:60352) 2024-04-25T10:34:29.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.370+0000 7ff9effff640 1 -- 172.21.15.106:0/736078292 learned_addr learned my addr 172.21.15.106:0/736078292 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:34:29.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.371+0000 7ff9ef7fe640 1 --2- 172.21.15.106:0/736078292 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9f0156f70 0x7ff9f0071680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:29.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.371+0000 7ff9f4e4c640 1 --2- 172.21.15.106:0/736078292 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9f015d340 0x7ff9f014e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:29.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.371+0000 7ff9effff640 1 -- 172.21.15.106:0/736078292 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9f0156f70 msgr2=0x7ff9f0071680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:29.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.371+0000 7ff9effff640 1 --2- 172.21.15.106:0/736078292 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9f0156f70 0x7ff9f0071680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:29.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.371+0000 7ff9effff640 1 -- 172.21.15.106:0/736078292 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9f015d340 msgr2=0x7ff9f014e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:29.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.371+0000 7ff9effff640 1 --2- 172.21.15.106:0/736078292 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9f015d340 0x7ff9f014e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:29.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.371+0000 7ff9effff640 1 -- 172.21.15.106:0/736078292 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9f01f3390 con 0x7ff9f00ca170 2024-04-25T10:34:29.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.371+0000 7ff9effff640 1 --2- 172.21.15.106:0/736078292 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9f00ca170 0x7ff9f01f2e50 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff9e000c430 tx=0x7ff9e000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:29.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.371+0000 7ff9ecff9640 1 -- 172.21.15.106:0/736078292 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e0017070 con 0x7ff9f00ca170 2024-04-25T10:34:29.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.371+0000 7ff9ecff9640 1 -- 172.21.15.106:0/736078292 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9e0005ce0 con 0x7ff9f00ca170 2024-04-25T10:34:29.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.371+0000 7ff9f68af640 1 -- 172.21.15.106:0/736078292 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9f0153f30 con 0x7ff9f00ca170 2024-04-25T10:34:29.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.371+0000 7ff9ecff9640 1 -- 172.21.15.106:0/736078292 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e0007310 con 0x7ff9f00ca170 2024-04-25T10:34:29.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.371+0000 7ff9f68af640 1 -- 172.21.15.106:0/736078292 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9f00cb6b0 con 0x7ff9f00ca170 2024-04-25T10:34:29.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.372+0000 7ff9f68af640 1 --2- 172.21.15.106:0/736078292 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9f0154100 0x7ff9f01f6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:29.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.372+0000 7ff9f68af640 1 -- 172.21.15.106:0/736078292 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff9f015c020 con 0x7ff9f0154100 2024-04-25T10:34:29.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.372+0000 7ff9ef7fe640 1 --2- 172.21.15.106:0/736078292 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9f0154100 0x7ff9f01f6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:29.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.373+0000 7ff9ef7fe640 1 -- 172.21.15.106:0/736078292 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff9f00c70f0 con 0x7ff9f0154100 2024-04-25T10:34:29.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.373+0000 7ff9ecff9640 1 -- 172.21.15.106:0/736078292 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff9e001a070 con 0x7ff9f00ca170 2024-04-25T10:34:29.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.373+0000 7ff9ef7fe640 1 --2- 172.21.15.106:0/736078292 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9f0154100 0x7ff9f01f6d90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff9f00722a0 tx=0x7ff9e40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:29.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.373+0000 7ff9ecff9640 1 --2- 172.21.15.106:0/736078292 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7ff9dc0744e0 0x7ff9dc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:29.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.373+0000 7ff9f4e4c640 1 --2- 172.21.15.106:0/736078292 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7ff9dc0744e0 0x7ff9dc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:29.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.373+0000 7ff9ecff9640 1 -- 172.21.15.106:0/736078292 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7ff9e00944d0 con 0x7ff9f00ca170 2024-04-25T10:34:29.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.373+0000 7ff9ecff9640 1 -- 172.21.15.106:0/736078292 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e4042070 con 0x7ff9f0154100 2024-04-25T10:34:29.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.373+0000 7ff9ecff9640 1 -- 172.21.15.106:0/736078292 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7ff9e4032e20 con 0x7ff9f0154100 2024-04-25T10:34:29.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.373+0000 7ff9ecff9640 1 -- 172.21.15.106:0/736078292 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9f0154100 msgr2=0x7ff9f01f6d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:29.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.373+0000 7ff9ecff9640 1 --2- 172.21.15.106:0/736078292 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9f0154100 0x7ff9f01f6d90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff9f00722a0 tx=0x7ff9e40046c0 comp rx=0 tx=0).stop 2024-04-25T10:34:29.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.373+0000 7ff9ecff9640 1 -- 172.21.15.106:0/736078292 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9f0154100 msgr2=0x7ff9f01f6d90 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T10:34:29.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.373+0000 7ff9ecff9640 1 --2- 172.21.15.106:0/736078292 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9f0154100 0x7ff9f01f6d90 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:29.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.374+0000 7ff9f4e4c640 1 --2- 172.21.15.106:0/736078292 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7ff9dc0744e0 0x7ff9dc0769f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff9f0155c90 tx=0x7ff9e800b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:29.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.380+0000 7ff9f68af640 1 --2- 172.21.15.106:0/736078292 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9f01f3560 0x7ff9f014ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:29.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.380+0000 7ff9f68af640 1 -- 172.21.15.106:0/736078292 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff9f015c270 con 0x7ff9f01f3560 2024-04-25T10:34:29.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.380+0000 7ff9ef7fe640 1 --2- 172.21.15.106:0/736078292 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9f01f3560 0x7ff9f014ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:29.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.381+0000 7ff9ef7fe640 1 -- 172.21.15.106:0/736078292 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff9f006a3d0 con 0x7ff9f01f3560 2024-04-25T10:34:29.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.381+0000 7ff9ef7fe640 1 --2- 172.21.15.106:0/736078292 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9f01f3560 0x7ff9f014ed90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff9e4000c00 tx=0x7ff9e4008030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:29.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.381+0000 7ff9ecff9640 1 -- 172.21.15.106:0/736078292 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e4037040 con 0x7ff9f01f3560 2024-04-25T10:34:29.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.381+0000 7ff9ecff9640 1 -- 172.21.15.106:0/736078292 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5069 (secure 0 0 0) 0x7ff9e4048ad0 con 0x7ff9f01f3560 2024-04-25T10:34:29.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.381+0000 7ff9ecff9640 1 -- 172.21.15.106:0/736078292 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9f01f3560 msgr2=0x7ff9f014ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:29.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.381+0000 7ff9ecff9640 1 --2- 172.21.15.106:0/736078292 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9f01f3560 0x7ff9f014ed90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff9e4000c00 tx=0x7ff9e4008030 comp rx=0 tx=0).stop 2024-04-25T10:34:29.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.381+0000 7ff9ecff9640 1 -- 172.21.15.106:0/736078292 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9f01f3560 msgr2=0x7ff9f014ed90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T10:34:29.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.381+0000 7ff9ecff9640 1 --2- 172.21.15.106:0/736078292 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9f01f3560 0x7ff9f014ed90 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:29.384 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-25T10:34:29.384 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "c", 2024-04-25T10:34:29.384 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 2, 2024-04-25T10:34:29.384 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "peon", 2024-04-25T10:34:29.384 INFO:teuthology.orchestra.run.smithi106.stdout: "election_epoch": 28, 2024-04-25T10:34:29.385 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 108304, 2024-04-25T10:34:29.385 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum": [ 2024-04-25T10:34:29.385 INFO:teuthology.orchestra.run.smithi106.stdout: 0, 2024-04-25T10:34:29.385 INFO:teuthology.orchestra.run.smithi106.stdout: 1, 2024-04-25T10:34:29.385 INFO:teuthology.orchestra.run.smithi106.stdout: 2 2024-04-25T10:34:29.385 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:34:29.385 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_age": 1, 2024-04-25T10:34:29.385 INFO:teuthology.orchestra.run.smithi106.stdout: "features": { 2024-04-25T10:34:29.385 INFO:teuthology.orchestra.run.smithi106.stdout: "required_con": "2450521708007866372", 2024-04-25T10:34:29.385 INFO:teuthology.orchestra.run.smithi106.stdout: "required_mon": [ 2024-04-25T10:34:29.385 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:34:29.385 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:34:29.385 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:34:29.385 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:34:29.385 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:34:29.386 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:34:29.386 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:34:29.386 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:34:29.386 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:34:29.386 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:34:29.386 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:34:29.386 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:34:29.386 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_con": "4540701547738038271", 2024-04-25T10:34:29.386 INFO:teuthology.orchestra.run.smithi106.stdout: "quorum_mon": [ 2024-04-25T10:34:29.386 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:34:29.386 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:34:29.386 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:34:29.386 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:34:29.386 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:34:29.387 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:34:29.387 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:34:29.387 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:34:29.387 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:34:29.387 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:34:29.387 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:34:29.387 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:34:29.387 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:34:29.387 INFO:teuthology.orchestra.run.smithi106.stdout: "outside_quorum": [], 2024-04-25T10:34:29.387 INFO:teuthology.orchestra.run.smithi106.stdout: "extra_probe_peers": [], 2024-04-25T10:34:29.387 INFO:teuthology.orchestra.run.smithi106.stdout: "sync_provider": [], 2024-04-25T10:34:29.387 INFO:teuthology.orchestra.run.smithi106.stdout: "monmap": { 2024-04-25T10:34:29.387 INFO:teuthology.orchestra.run.smithi106.stdout: "epoch": 1, 2024-04-25T10:34:29.387 INFO:teuthology.orchestra.run.smithi106.stdout: "fsid": "3ee77985-e8e4-41cf-81e9-7c774cf7891f", 2024-04-25T10:34:29.387 INFO:teuthology.orchestra.run.smithi106.stdout: "modified": "2024-04-25T10:29:33.446228Z", 2024-04-25T10:34:29.387 INFO:teuthology.orchestra.run.smithi106.stdout: "created": "2024-04-25T10:29:33.446228Z", 2024-04-25T10:34:29.388 INFO:teuthology.orchestra.run.smithi106.stdout: "min_mon_release": 19, 2024-04-25T10:34:29.388 INFO:teuthology.orchestra.run.smithi106.stdout: "min_mon_release_name": "squid", 2024-04-25T10:34:29.388 INFO:teuthology.orchestra.run.smithi106.stdout: "election_strategy": 1, 2024-04-25T10:34:29.388 INFO:teuthology.orchestra.run.smithi106.stdout: "disallowed_leaders: ": "", 2024-04-25T10:34:29.388 INFO:teuthology.orchestra.run.smithi106.stdout: "stretch_mode": false, 2024-04-25T10:34:29.388 INFO:teuthology.orchestra.run.smithi106.stdout: "tiebreaker_mon": "", 2024-04-25T10:34:29.388 INFO:teuthology.orchestra.run.smithi106.stdout: "removed_ranks: ": "", 2024-04-25T10:34:29.388 INFO:teuthology.orchestra.run.smithi106.stdout: "features": { 2024-04-25T10:34:29.388 INFO:teuthology.orchestra.run.smithi106.stdout: "persistent": [ 2024-04-25T10:34:29.388 INFO:teuthology.orchestra.run.smithi106.stdout: "kraken", 2024-04-25T10:34:29.388 INFO:teuthology.orchestra.run.smithi106.stdout: "luminous", 2024-04-25T10:34:29.388 INFO:teuthology.orchestra.run.smithi106.stdout: "mimic", 2024-04-25T10:34:29.388 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap-prune", 2024-04-25T10:34:29.388 INFO:teuthology.orchestra.run.smithi106.stdout: "nautilus", 2024-04-25T10:34:29.388 INFO:teuthology.orchestra.run.smithi106.stdout: "octopus", 2024-04-25T10:34:29.388 INFO:teuthology.orchestra.run.smithi106.stdout: "pacific", 2024-04-25T10:34:29.388 INFO:teuthology.orchestra.run.smithi106.stdout: "elector-pinging", 2024-04-25T10:34:29.389 INFO:teuthology.orchestra.run.smithi106.stdout: "quincy", 2024-04-25T10:34:29.389 INFO:teuthology.orchestra.run.smithi106.stdout: "reef", 2024-04-25T10:34:29.389 INFO:teuthology.orchestra.run.smithi106.stdout: "squid" 2024-04-25T10:34:29.389 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:34:29.389 INFO:teuthology.orchestra.run.smithi106.stdout: "optional": [] 2024-04-25T10:34:29.389 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:34:29.389 INFO:teuthology.orchestra.run.smithi106.stdout: "mons": [ 2024-04-25T10:34:29.389 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:34:29.389 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 0, 2024-04-25T10:34:29.389 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "a", 2024-04-25T10:34:29.389 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:34:29.389 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:34:29.389 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:34:29.389 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:34:29.389 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:3300", 2024-04-25T10:34:29.390 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:34:29.390 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:34:29.390 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:34:29.390 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:34:29.390 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:6789", 2024-04-25T10:34:29.390 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:34:29.390 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:34:29.390 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:34:29.390 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:34:29.390 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.106:6789/0", 2024-04-25T10:34:29.390 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.106:6789/0", 2024-04-25T10:34:29.390 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:34:29.390 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:34:29.390 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:34:29.390 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:34:29.391 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:34:29.391 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 1, 2024-04-25T10:34:29.391 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "b", 2024-04-25T10:34:29.391 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:34:29.391 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:34:29.391 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:34:29.391 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:34:29.391 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:3300", 2024-04-25T10:34:29.391 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:34:29.391 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:34:29.391 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:34:29.391 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:34:29.391 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6789", 2024-04-25T10:34:29.391 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:34:29.391 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:34:29.392 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:34:29.392 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:34:29.392 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6789/0", 2024-04-25T10:34:29.392 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.134:6789/0", 2024-04-25T10:34:29.392 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:34:29.392 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:34:29.392 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:34:29.392 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:34:29.392 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:34:29.392 INFO:teuthology.orchestra.run.smithi106.stdout: "rank": 2, 2024-04-25T10:34:29.392 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "c", 2024-04-25T10:34:29.392 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addrs": { 2024-04-25T10:34:29.392 INFO:teuthology.orchestra.run.smithi106.stdout: "addrvec": [ 2024-04-25T10:34:29.392 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:34:29.393 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v2", 2024-04-25T10:34:29.393 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:3301", 2024-04-25T10:34:29.393 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:34:29.393 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:34:29.393 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:34:29.393 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-25T10:34:29.393 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6790", 2024-04-25T10:34:29.393 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 0 2024-04-25T10:34:29.393 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:34:29.393 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:34:29.393 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:34:29.393 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "172.21.15.134:6790/0", 2024-04-25T10:34:29.393 INFO:teuthology.orchestra.run.smithi106.stdout: "public_addr": "172.21.15.134:6790/0", 2024-04-25T10:34:29.393 INFO:teuthology.orchestra.run.smithi106.stdout: "priority": 0, 2024-04-25T10:34:29.393 INFO:teuthology.orchestra.run.smithi106.stdout: "weight": 0, 2024-04-25T10:34:29.394 INFO:teuthology.orchestra.run.smithi106.stdout: "crush_location": "{}" 2024-04-25T10:34:29.394 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:34:29.394 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:34:29.394 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:34:29.394 INFO:teuthology.orchestra.run.smithi106.stdout: "feature_map": { 2024-04-25T10:34:29.394 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": [ 2024-04-25T10:34:29.394 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:34:29.394 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:34:29.394 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:34:29.394 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 1 2024-04-25T10:34:29.394 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:34:29.394 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:34:29.394 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": [ 2024-04-25T10:34:29.394 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:34:29.394 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:34:29.394 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:34:29.395 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 4 2024-04-25T10:34:29.395 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:34:29.395 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:34:29.395 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": [ 2024-04-25T10:34:29.395 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:34:29.395 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:34:29.395 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:34:29.395 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 5 2024-04-25T10:34:29.395 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:34:29.395 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:34:29.395 INFO:teuthology.orchestra.run.smithi106.stdout: "client": [ 2024-04-25T10:34:29.395 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:34:29.395 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:34:29.395 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:34:29.395 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 4 2024-04-25T10:34:29.395 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:34:29.395 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2024-04-25T10:34:29.396 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": [ 2024-04-25T10:34:29.396 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-25T10:34:29.396 INFO:teuthology.orchestra.run.smithi106.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T10:34:29.396 INFO:teuthology.orchestra.run.smithi106.stdout: "release": "luminous", 2024-04-25T10:34:29.396 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 1 2024-04-25T10:34:29.396 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T10:34:29.396 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T10:34:29.396 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T10:34:29.396 INFO:teuthology.orchestra.run.smithi106.stdout: "stretch_mode": false 2024-04-25T10:34:29.396 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-25T10:34:29.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.381+0000 7ff9f68af640 1 -- 172.21.15.106:0/736078292 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7ff9dc0744e0 msgr2=0x7ff9dc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:29.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.381+0000 7ff9f68af640 1 --2- 172.21.15.106:0/736078292 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7ff9dc0744e0 0x7ff9dc0769f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff9f0155c90 tx=0x7ff9e800b040 comp rx=0 tx=0).stop 2024-04-25T10:34:29.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.381+0000 7ff9f68af640 1 -- 172.21.15.106:0/736078292 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9f00ca170 msgr2=0x7ff9f01f2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:29.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.382+0000 7ff9f68af640 1 --2- 172.21.15.106:0/736078292 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9f00ca170 0x7ff9f01f2e50 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff9e000c430 tx=0x7ff9e000c8f0 comp rx=0 tx=0).stop 2024-04-25T10:34:29.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.382+0000 7ff9f4e4c640 1 -- 172.21.15.106:0/736078292 reap_dead start 2024-04-25T10:34:29.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.382+0000 7ff9f4e4c640 1 -- 172.21.15.106:0/736078292 reap_dead start 2024-04-25T10:34:29.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.382+0000 7ff9f68af640 1 -- 172.21.15.106:0/736078292 shutdown_connections 2024-04-25T10:34:29.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.382+0000 7ff9f68af640 1 -- 172.21.15.106:0/736078292 >> 172.21.15.106:0/736078292 conn(0x7ff9f00cef20 msgr2=0x7ff9f00c5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:29.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.382+0000 7ff9f68af640 1 -- 172.21.15.106:0/736078292 shutdown_connections 2024-04-25T10:34:29.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.382+0000 7ff9f68af640 1 -- 172.21.15.106:0/736078292 wait complete. 2024-04-25T10:34:29.398 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T10:34:29.398 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-04-25T10:34:29.398 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-04-25T10:34:29.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.503+0000 7fb566ed8640 1 Processor -- start 2024-04-25T10:34:29.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.503+0000 7fb566ed8640 1 -- start start 2024-04-25T10:34:29.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.504+0000 7fb566ed8640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5600ca170 0x7fb560156f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:29.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.504+0000 7fb566ed8640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fb56015d510 0x7fb56006c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:29.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.504+0000 7fb566ed8640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb560073980 con 0x7fb56006c5c0 2024-04-25T10:34:29.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.505+0000 7fb566ed8640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fb560073f00 con 0x7fb56015d510 2024-04-25T10:34:29.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.505+0000 7fb566ed8640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb560074990 con 0x7fb5600ca170 2024-04-25T10:34:29.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.505+0000 7fb55ffff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fb56006c5c0 0x7fb56015ce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:46756/0 (socket says 172.21.15.106:46756) 2024-04-25T10:34:29.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.505+0000 7fb55ffff640 1 -- 172.21.15.106:0/805222263 learned_addr learned my addr 172.21.15.106:0/805222263 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:34:29.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.505+0000 7fb565475640 1 --2- 172.21.15.106:0/805222263 >> v2:172.21.15.134:3300/0 conn(0x7fb56015d510 0x7fb56006c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:29.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.505+0000 7fb55f7fe640 1 -- 172.21.15.106:0/805222263 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3601145910 0 0) 0x7fb560073980 con 0x7fb56006c5c0 2024-04-25T10:34:29.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.505+0000 7fb564c74640 1 --2- 172.21.15.106:0/805222263 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5600ca170 0x7fb560156f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:29.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.505+0000 7fb55f7fe640 1 -- 172.21.15.106:0/805222263 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb54c003520 con 0x7fb56006c5c0 2024-04-25T10:34:29.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.505+0000 7fb55f7fe640 1 -- 172.21.15.106:0/805222263 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 358317945 0 0) 0x7fb54c003520 con 0x7fb56006c5c0 2024-04-25T10:34:29.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.505+0000 7fb55f7fe640 1 -- 172.21.15.106:0/805222263 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5600ca170 msgr2=0x7fb560156f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:29.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.505+0000 7fb55f7fe640 1 --2- 172.21.15.106:0/805222263 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5600ca170 0x7fb560156f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:29.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.505+0000 7fb55f7fe640 1 -- 172.21.15.106:0/805222263 >> v2:172.21.15.134:3300/0 conn(0x7fb56015d510 msgr2=0x7fb56006c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:29.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.505+0000 7fb55f7fe640 1 --2- 172.21.15.106:0/805222263 >> v2:172.21.15.134:3300/0 conn(0x7fb56015d510 0x7fb56006c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:29.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.505+0000 7fb55f7fe640 1 -- 172.21.15.106:0/805222263 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb560160ea0 con 0x7fb56006c5c0 2024-04-25T10:34:29.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.505+0000 7fb55f7fe640 1 -- 172.21.15.106:0/805222263 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3776459350 0 0) 0x7fb5500019f0 con 0x7fb56006c5c0 2024-04-25T10:34:29.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.505+0000 7fb55f7fe640 1 -- 172.21.15.106:0/805222263 >> v1:172.21.15.106:6789/0 conn(0x7fb56006c5c0 legacy=0x7fb56015ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:29.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.505+0000 7fb55f7fe640 1 --2- 172.21.15.106:0/805222263 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5600ca170 0x7fb54c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:29.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.505+0000 7fb55f7fe640 1 --2- 172.21.15.106:0/805222263 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb54c003f80 0x7fb54c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:29.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.505+0000 7fb55f7fe640 1 --2- 172.21.15.106:0/805222263 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb54c004d40 0x7fb54c005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:29.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.505+0000 7fb55f7fe640 1 -- 172.21.15.106:0/805222263 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb560073f00 con 0x7fb54c004d40 2024-04-25T10:34:29.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.505+0000 7fb55f7fe640 1 -- 172.21.15.106:0/805222263 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb560074990 con 0x7fb54c003f80 2024-04-25T10:34:29.511 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.505+0000 7fb55f7fe640 1 -- 172.21.15.106:0/805222263 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb54c00baf0 con 0x7fb5600ca170 2024-04-25T10:34:29.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.505+0000 7fb565475640 1 --2- 172.21.15.106:0/805222263 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb54c004d40 0x7fb54c005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:29.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.505+0000 7fb564c74640 1 --2- 172.21.15.106:0/805222263 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5600ca170 0x7fb54c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:29.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.505+0000 7fb55ffff640 1 --2- 172.21.15.106:0/805222263 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb54c003f80 0x7fb54c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:29.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.506+0000 7fb565475640 1 -- 172.21.15.106:0/805222263 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5600ca170 msgr2=0x7fb54c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:29.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.506+0000 7fb565475640 1 --2- 172.21.15.106:0/805222263 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5600ca170 0x7fb54c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:29.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.506+0000 7fb565475640 1 -- 172.21.15.106:0/805222263 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb54c003f80 msgr2=0x7fb54c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:29.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.506+0000 7fb565475640 1 --2- 172.21.15.106:0/805222263 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb54c003f80 0x7fb54c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:29.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.506+0000 7fb565475640 1 -- 172.21.15.106:0/805222263 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb54c003520 con 0x7fb54c004d40 2024-04-25T10:34:29.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.506+0000 7fb565475640 1 --2- 172.21.15.106:0/805222263 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb54c004d40 0x7fb54c005150 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb5600c5f50 tx=0x7fb54c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76cf33015f00450c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:29.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.507+0000 7fb55f7fe640 1 -- 172.21.15.106:0/805222263 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb548036070 con 0x7fb54c004d40 2024-04-25T10:34:29.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.507+0000 7fb55f7fe640 1 -- 172.21.15.106:0/805222263 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb548031400 con 0x7fb54c004d40 2024-04-25T10:34:29.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.507+0000 7fb55f7fe640 1 -- 172.21.15.106:0/805222263 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb548009690 con 0x7fb54c004d40 2024-04-25T10:34:29.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.507+0000 7fb566ed8640 1 -- 172.21.15.106:0/805222263 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb54c004d40 msgr2=0x7fb54c005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:29.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.507+0000 7fb566ed8640 1 --2- 172.21.15.106:0/805222263 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb54c004d40 0x7fb54c005150 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb5600c5f50 tx=0x7fb54c005830 comp rx=0 tx=0).stop 2024-04-25T10:34:29.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.507+0000 7fb565475640 1 -- 172.21.15.106:0/805222263 reap_dead start 2024-04-25T10:34:29.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.507+0000 7fb566ed8640 1 -- 172.21.15.106:0/805222263 shutdown_connections 2024-04-25T10:34:29.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.507+0000 7fb566ed8640 1 -- 172.21.15.106:0/805222263 >> 172.21.15.106:0/805222263 conn(0x7fb5600cef20 msgr2=0x7fb5600cf330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:29.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.507+0000 7fb566ed8640 1 -- 172.21.15.106:0/805222263 shutdown_connections 2024-04-25T10:34:29.513 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.507+0000 7fb566ed8640 1 -- 172.21.15.106:0/805222263 wait complete. 2024-04-25T10:34:29.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.509+0000 7fb566ed8640 1 Processor -- start 2024-04-25T10:34:29.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.509+0000 7fb566ed8640 1 -- start start 2024-04-25T10:34:29.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.510+0000 7fb566ed8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb54c004d40 0x7fb5601e6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:29.515 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.510+0000 7fb566ed8640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb560158f30 0x7fb56015bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:29.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.510+0000 7fb566ed8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5600ccd20 0x7fb5600cd130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:29.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.510+0000 7fb566ed8640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb560161140 con 0x7fb54c004d40 2024-04-25T10:34:29.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.510+0000 7fb566ed8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb560073c20 con 0x7fb5600ccd20 2024-04-25T10:34:29.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.510+0000 7fb566ed8640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb5600c9cd0 con 0x7fb560158f30 2024-04-25T10:34:29.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.511+0000 7fb564c74640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb54c004d40 0x7fb5601e6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:29.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.511+0000 7fb564c74640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb54c004d40 0x7fb5601e6130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60384/0 (socket says 172.21.15.106:60384) 2024-04-25T10:34:29.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.511+0000 7fb564c74640 1 -- 172.21.15.106:0/1920396440 learned_addr learned my addr 172.21.15.106:0/1920396440 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:34:29.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.511+0000 7fb564c74640 1 -- 172.21.15.106:0/1920396440 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb560158f30 msgr2=0x7fb56015bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:29.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.511+0000 7fb564c74640 1 --2- 172.21.15.106:0/1920396440 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb560158f30 0x7fb56015bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:29.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.511+0000 7fb564c74640 1 -- 172.21.15.106:0/1920396440 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5600ccd20 msgr2=0x7fb5600cd130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:29.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.511+0000 7fb564c74640 1 --2- 172.21.15.106:0/1920396440 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5600ccd20 0x7fb5600cd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:29.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.511+0000 7fb564c74640 1 -- 172.21.15.106:0/1920396440 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb56015a6c0 con 0x7fb54c004d40 2024-04-25T10:34:29.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.511+0000 7fb564c74640 1 --2- 172.21.15.106:0/1920396440 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb54c004d40 0x7fb5601e6130 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb5580119d0 tx=0x7fb558011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:29.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.511+0000 7fb55d7fa640 1 -- 172.21.15.106:0/1920396440 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb55800fad0 con 0x7fb54c004d40 2024-04-25T10:34:29.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.511+0000 7fb55d7fa640 1 -- 172.21.15.106:0/1920396440 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb55800fc70 con 0x7fb54c004d40 2024-04-25T10:34:29.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.511+0000 7fb566ed8640 1 -- 172.21.15.106:0/1920396440 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb560157e20 con 0x7fb54c004d40 2024-04-25T10:34:29.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.511+0000 7fb55d7fa640 1 -- 172.21.15.106:0/1920396440 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb558014440 con 0x7fb54c004d40 2024-04-25T10:34:29.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.511+0000 7fb566ed8640 1 -- 172.21.15.106:0/1920396440 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5600ceb20 con 0x7fb54c004d40 2024-04-25T10:34:29.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.512+0000 7fb566ed8640 1 -- 172.21.15.106:0/1920396440 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb56006a3d0 con 0x7fb54c004d40 2024-04-25T10:34:29.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.513+0000 7fb55d7fa640 1 -- 172.21.15.106:0/1920396440 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb558012040 con 0x7fb54c004d40 2024-04-25T10:34:29.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.513+0000 7fb55d7fa640 1 --2- 172.21.15.106:0/1920396440 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fb5400744b0 0x7fb5400769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:29.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.513+0000 7fb55d7fa640 1 -- 172.21.15.106:0/1920396440 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fb558095950 con 0x7fb54c004d40 2024-04-25T10:34:29.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.513+0000 7fb55ffff640 1 --2- 172.21.15.106:0/1920396440 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fb5400744b0 0x7fb5400769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:29.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.514+0000 7fb55ffff640 1 --2- 172.21.15.106:0/1920396440 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fb5400744b0 0x7fb5400769c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb550001b60 tx=0x7fb550001810 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:29.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.518+0000 7fb55d7fa640 1 -- 172.21.15.106:0/1920396440 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb558019030 con 0x7fb54c004d40 2024-04-25T10:34:29.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.763+0000 7fb566ed8640 1 -- 172.21.15.106:0/1920396440 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7fb560074420 con 0x7fb54c004d40 2024-04-25T10:34:29.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.764+0000 7fb55d7fa640 1 -- 172.21.15.106:0/1920396440 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7fb558060060 con 0x7fb54c004d40 2024-04-25T10:34:29.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.766+0000 7fb566ed8640 1 -- 172.21.15.106:0/1920396440 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fb5400744b0 msgr2=0x7fb5400769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:29.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.766+0000 7fb566ed8640 1 --2- 172.21.15.106:0/1920396440 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fb5400744b0 0x7fb5400769c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb550001b60 tx=0x7fb550001810 comp rx=0 tx=0).stop 2024-04-25T10:34:29.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.766+0000 7fb566ed8640 1 -- 172.21.15.106:0/1920396440 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb54c004d40 msgr2=0x7fb5601e6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:29.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.766+0000 7fb566ed8640 1 --2- 172.21.15.106:0/1920396440 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb54c004d40 0x7fb5601e6130 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb5580119d0 tx=0x7fb558011e90 comp rx=0 tx=0).stop 2024-04-25T10:34:29.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.766+0000 7fb566ed8640 1 -- 172.21.15.106:0/1920396440 shutdown_connections 2024-04-25T10:34:29.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.766+0000 7fb566ed8640 1 --2- 172.21.15.106:0/1920396440 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fb5400744b0 0x7fb5400769c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:29.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.766+0000 7fb566ed8640 1 --2- 172.21.15.106:0/1920396440 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5600ccd20 0x7fb5600cd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:29.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.766+0000 7fb566ed8640 1 --2- 172.21.15.106:0/1920396440 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb560158f30 0x7fb56015bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:29.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.766+0000 7fb566ed8640 1 --2- 172.21.15.106:0/1920396440 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb54c004d40 0x7fb5601e6130 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:29.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.766+0000 7fb566ed8640 1 -- 172.21.15.106:0/1920396440 >> 172.21.15.106:0/1920396440 conn(0x7fb5600cef20 msgr2=0x7fb5600ca8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:29.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.766+0000 7fb566ed8640 1 -- 172.21.15.106:0/1920396440 shutdown_connections 2024-04-25T10:34:29.770 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:29.766+0000 7fb566ed8640 1 -- 172.21.15.106:0/1920396440 wait complete. 2024-04-25T10:34:29.781 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-04-25T10:34:32.894 DEBUG:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:34:32.897 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:34:39.782 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:34:39.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:39.903+0000 7efda30f9640 1 Processor -- start 2024-04-25T10:34:39.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:39.904+0000 7efda30f9640 1 -- start start 2024-04-25T10:34:39.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:39.905+0000 7efda30f9640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7efd9c072870 0x7efd9c15a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:39.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:39.905+0000 7efda30f9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efd9c0c5280 0x7efd9c15d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:39.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:39.905+0000 7efda30f9640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd9c0c5690 con 0x7efd9c0cae80 2024-04-25T10:34:39.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:39.905+0000 7efda30f9640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7efd9c0c7c40 con 0x7efd9c072870 2024-04-25T10:34:39.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:39.905+0000 7efda30f9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7efd9c0c7f20 con 0x7efd9c0c5280 2024-04-25T10:34:39.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:39.905+0000 7efda0e95640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7efd9c072870 0x7efd9c15a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:39.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:39.905+0000 7efda1696640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7efd9c0cae80 0x7efd9c153710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:45214/0 (socket says 172.21.15.106:45214) 2024-04-25T10:34:39.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:39.905+0000 7efda1696640 1 -- 172.21.15.106:0/952816037 learned_addr learned my addr 172.21.15.106:0/952816037 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:34:39.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:39.905+0000 7efd9bfff640 1 --2- 172.21.15.106:0/952816037 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efd9c0c5280 0x7efd9c15d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:39.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:39.905+0000 7efda0e95640 1 -- 172.21.15.106:0/952816037 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efd9c0c5280 msgr2=0x7efd9c15d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:39.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:39.905+0000 7efda0e95640 1 --2- 172.21.15.106:0/952816037 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efd9c0c5280 0x7efd9c15d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:39.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:39.905+0000 7efda0e95640 1 -- 172.21.15.106:0/952816037 >> v1:172.21.15.106:6789/0 conn(0x7efd9c0cae80 legacy=0x7efd9c153710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:39.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:39.905+0000 7efda0e95640 1 -- 172.21.15.106:0/952816037 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd9c0c8200 con 0x7efd9c072870 2024-04-25T10:34:39.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:39.905+0000 7efd9bfff640 1 --2- 172.21.15.106:0/952816037 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efd9c0c5280 0x7efd9c15d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:34:39.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:39.906+0000 7efda0e95640 1 --2- 172.21.15.106:0/952816037 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd9c072870 0x7efd9c15a590 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7efd8c0097a0 tx=0x7efd8c02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=690018d3e6bd5d3b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:39.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:39.906+0000 7efd9b7fe640 1 -- 172.21.15.106:0/952816037 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd8c037070 con 0x7efd9c072870 2024-04-25T10:34:39.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:39.906+0000 7efd9b7fe640 1 -- 172.21.15.106:0/952816037 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd8c02ae50 con 0x7efd9c072870 2024-04-25T10:34:39.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:39.906+0000 7efd9b7fe640 1 -- 172.21.15.106:0/952816037 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd8c032660 con 0x7efd9c072870 2024-04-25T10:34:39.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:39.907+0000 7efda30f9640 1 -- 172.21.15.106:0/952816037 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd9c072870 msgr2=0x7efd9c15a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:39.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:39.907+0000 7efda30f9640 1 --2- 172.21.15.106:0/952816037 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd9c072870 0x7efd9c15a590 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7efd8c0097a0 tx=0x7efd8c02a100 comp rx=0 tx=0).stop 2024-04-25T10:34:39.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:39.907+0000 7efda30f9640 1 -- 172.21.15.106:0/952816037 shutdown_connections 2024-04-25T10:34:39.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:39.907+0000 7efda30f9640 1 --2- 172.21.15.106:0/952816037 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efd9c0c5280 0x7efd9c15d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:39.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:39.907+0000 7efda30f9640 1 --2- 172.21.15.106:0/952816037 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd9c072870 0x7efd9c15a590 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:39.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:39.907+0000 7efda30f9640 1 -- 172.21.15.106:0/952816037 >> 172.21.15.106:0/952816037 conn(0x7efd9c06c9b0 msgr2=0x7efd9c06cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:39.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:39.907+0000 7efda30f9640 1 -- 172.21.15.106:0/952816037 shutdown_connections 2024-04-25T10:34:39.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:39.907+0000 7efda30f9640 1 -- 172.21.15.106:0/952816037 wait complete. 2024-04-25T10:34:39.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:39.909+0000 7efda30f9640 1 Processor -- start 2024-04-25T10:34:39.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:39.909+0000 7efda30f9640 1 -- start start 2024-04-25T10:34:39.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:39.910+0000 7efda30f9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efd9c072870 0x7efd9c153e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:39.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:39.910+0000 7efda30f9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd9c0c5280 0x7efd9c15d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:39.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:39.910+0000 7efda30f9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efd9c0cae80 0x7efd9c155e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:39.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:39.910+0000 7efda30f9640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7efd9c15a070 con 0x7efd9c0c5280 2024-04-25T10:34:39.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:39.910+0000 7efda30f9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efd9c159d90 con 0x7efd9c0cae80 2024-04-25T10:34:39.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:39.910+0000 7efda30f9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7efd9c0c7920 con 0x7efd9c072870 2024-04-25T10:34:39.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:39.910+0000 7efd9bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd9c0c5280 0x7efd9c15d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:39.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:39.910+0000 7efd9bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd9c0c5280 0x7efd9c15d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:37244/0 (socket says 172.21.15.106:37244) 2024-04-25T10:34:39.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:39.910+0000 7efd9bfff640 1 -- 172.21.15.106:0/1781405991 learned_addr learned my addr 172.21.15.106:0/1781405991 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:34:39.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:39.910+0000 7efda0e95640 1 --2- 172.21.15.106:0/1781405991 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efd9c072870 0x7efd9c153e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:39.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:39.910+0000 7efda1696640 1 --2- 172.21.15.106:0/1781405991 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efd9c0cae80 0x7efd9c155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:39.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:39.911+0000 7efd9bfff640 1 -- 172.21.15.106:0/1781405991 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efd9c072870 msgr2=0x7efd9c153e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:39.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:39.911+0000 7efd9bfff640 1 --2- 172.21.15.106:0/1781405991 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efd9c072870 0x7efd9c153e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:39.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:39.911+0000 7efd9bfff640 1 -- 172.21.15.106:0/1781405991 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efd9c0cae80 msgr2=0x7efd9c155e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:39.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:39.911+0000 7efd9bfff640 1 --2- 172.21.15.106:0/1781405991 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efd9c0cae80 0x7efd9c155e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:39.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:39.911+0000 7efd9bfff640 1 -- 172.21.15.106:0/1781405991 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd9c154370 con 0x7efd9c0c5280 2024-04-25T10:34:39.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:39.911+0000 7efd9bfff640 1 --2- 172.21.15.106:0/1781405991 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd9c0c5280 0x7efd9c15d120 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7efd9000d830 tx=0x7efd9000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:39.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:39.911+0000 7efd997fa640 1 -- 172.21.15.106:0/1781405991 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd900096a0 con 0x7efd9c0c5280 2024-04-25T10:34:39.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:39.911+0000 7efd997fa640 1 -- 172.21.15.106:0/1781405991 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd90019070 con 0x7efd9c0c5280 2024-04-25T10:34:39.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:39.911+0000 7efd997fa640 1 -- 172.21.15.106:0/1781405991 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd90009a80 con 0x7efd9c0c5280 2024-04-25T10:34:39.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:39.911+0000 7efda30f9640 1 -- 172.21.15.106:0/1781405991 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd9c0cd9d0 con 0x7efd9c0c5280 2024-04-25T10:34:39.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:39.911+0000 7efda30f9640 1 -- 172.21.15.106:0/1781405991 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd9c0cd330 con 0x7efd9c0c5280 2024-04-25T10:34:39.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:39.912+0000 7efda30f9640 1 -- 172.21.15.106:0/1781405991 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd9c06a3d0 con 0x7efd9c0c5280 2024-04-25T10:34:39.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:39.913+0000 7efd997fa640 1 -- 172.21.15.106:0/1781405991 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7efd9001e460 con 0x7efd9c0c5280 2024-04-25T10:34:39.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:39.913+0000 7efd997fa640 1 --2- 172.21.15.106:0/1781405991 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7efd880744b0 0x7efd880769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:39.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:39.913+0000 7efd997fa640 1 -- 172.21.15.106:0/1781405991 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7efd90095750 con 0x7efd9c0c5280 2024-04-25T10:34:39.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:39.913+0000 7efda0e95640 1 --2- 172.21.15.106:0/1781405991 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7efd880744b0 0x7efd880769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:39.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:39.914+0000 7efda0e95640 1 --2- 172.21.15.106:0/1781405991 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7efd880744b0 0x7efd880769c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7efd8c034000 tx=0x7efd8c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:39.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:39.918+0000 7efd997fa640 1 -- 172.21.15.106:0/1781405991 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efd9005f8a0 con 0x7efd9c0c5280 2024-04-25T10:34:40.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.134+0000 7efda30f9640 1 -- 172.21.15.106:0/1781405991 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efd9c0c9d00 con 0x7efd9c0c5280 2024-04-25T10:34:40.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.135+0000 7efd997fa640 1 -- 172.21.15.106:0/1781405991 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+6922 (secure 0 0 0) 0x7efd90063ba0 con 0x7efd9c0c5280 2024-04-25T10:34:40.136 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:34:40.136 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/2834155852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":2834155852},{"type":"v1","addr":"172.21.15.106:6835","nonce":2834155852}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:30:31.761107+0000","modified":"2024-04-25T10:31:00.589522+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4290,"mds_1":4287,"mds_2":4284,"mds_3":4269,"mds_4":4266},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4266":{"gid":4266,"name":"c","rank":4,"incarnation":16,"state":"up:active","state_seq":10,"addr":"172.21.15.106:6837/2632121774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":2632121774},{"type":"v1","addr":"172.21.15.106:6837","nonce":2632121774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4269":{"gid":4269,"name":"e","rank":3,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.106:6839/486339834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":486339834},{"type":"v1","addr":"172.21.15.106:6839","nonce":486339834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4284":{"gid":4284,"name":"b","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.134:6836/2781656864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6833","nonce":2781656864},{"type":"v1","addr":"172.21.15.134:6836","nonce":2781656864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4287":{"gid":4287,"name":"f","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.134:6835/1993918351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1993918351},{"type":"v1","addr":"172.21.15.134:6835","nonce":1993918351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4290":{"gid":4290,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.134:6837/2662003420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2662003420},{"type":"v1","addr":"172.21.15.134:6837","nonce":2662003420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4290,"qdb_cluster":[4266,4269,4284,4287,4290]},"id":1}]} 2024-04-25T10:34:40.136 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 17 2024-04-25T10:34:40.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.137+0000 7efda30f9640 1 -- 172.21.15.106:0/1781405991 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7efd880744b0 msgr2=0x7efd880769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:40.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.137+0000 7efda30f9640 1 --2- 172.21.15.106:0/1781405991 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7efd880744b0 0x7efd880769c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7efd8c034000 tx=0x7efd8c02e000 comp rx=0 tx=0).stop 2024-04-25T10:34:40.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.137+0000 7efda30f9640 1 -- 172.21.15.106:0/1781405991 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd9c0c5280 msgr2=0x7efd9c15d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:40.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.137+0000 7efda30f9640 1 --2- 172.21.15.106:0/1781405991 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd9c0c5280 0x7efd9c15d120 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7efd9000d830 tx=0x7efd9000dcf0 comp rx=0 tx=0).stop 2024-04-25T10:34:40.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.137+0000 7efda30f9640 1 -- 172.21.15.106:0/1781405991 shutdown_connections 2024-04-25T10:34:40.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.137+0000 7efda30f9640 1 --2- 172.21.15.106:0/1781405991 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7efd880744b0 0x7efd880769c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:40.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.137+0000 7efda30f9640 1 --2- 172.21.15.106:0/1781405991 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efd9c0cae80 0x7efd9c155e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:40.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.137+0000 7efda30f9640 1 --2- 172.21.15.106:0/1781405991 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd9c0c5280 0x7efd9c15d120 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:40.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.137+0000 7efda30f9640 1 --2- 172.21.15.106:0/1781405991 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efd9c072870 0x7efd9c153e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:40.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.137+0000 7efda30f9640 1 -- 172.21.15.106:0/1781405991 >> 172.21.15.106:0/1781405991 conn(0x7efd9c06c9b0 msgr2=0x7efd9c14a330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:40.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.137+0000 7efda30f9640 1 -- 172.21.15.106:0/1781405991 shutdown_connections 2024-04-25T10:34:40.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.137+0000 7efda30f9640 1 -- 172.21.15.106:0/1781405991 wait complete. 2024-04-25T10:34:40.150 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-04-25T10:34:40.150 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T10:34:40.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.258+0000 7efeb5182640 1 Processor -- start 2024-04-25T10:34:40.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.258+0000 7efeb5182640 1 -- start start 2024-04-25T10:34:40.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.259+0000 7efeb5182640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7efeb00c7e30 0x7efeb00c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:40.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.259+0000 7efeb5182640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efeb00b4ce0 0x7efeb015ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:40.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.259+0000 7efeb5182640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efeb00ce630 con 0x7efeb0156ef0 2024-04-25T10:34:40.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.259+0000 7efeb5182640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7efeb00b50f0 con 0x7efeb00c7e30 2024-04-25T10:34:40.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.259+0000 7efeb5182640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7efeb00c7c60 con 0x7efeb00b4ce0 2024-04-25T10:34:40.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.259+0000 7efeaed9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7efeb00c7e30 0x7efeb00c8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:40.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.259+0000 7efeaed9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7efeb00c7e30 0x7efeb00c8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:37260/0 (socket says 172.21.15.106:37260) 2024-04-25T10:34:40.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.260+0000 7efeaed9d640 1 -- 172.21.15.106:0/2296550921 learned_addr learned my addr 172.21.15.106:0/2296550921 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:34:40.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.260+0000 7efeae59c640 1 --1- 172.21.15.106:0/2296550921 >> v1:172.21.15.106:6789/0 conn(0x7efeb0156ef0 0x7efeb006d000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:45230/0 (socket says 172.21.15.106:45230) 2024-04-25T10:34:40.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.260+0000 7efeaf59e640 1 --2- 172.21.15.106:0/2296550921 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efeb00b4ce0 0x7efeb015ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:40.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.260+0000 7efeaed9d640 1 -- 172.21.15.106:0/2296550921 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efeb00b4ce0 msgr2=0x7efeb015ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:40.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.260+0000 7efeaed9d640 1 --2- 172.21.15.106:0/2296550921 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efeb00b4ce0 0x7efeb015ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:40.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.260+0000 7efeaed9d640 1 -- 172.21.15.106:0/2296550921 >> v1:172.21.15.106:6789/0 conn(0x7efeb0156ef0 legacy=0x7efeb006d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:40.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.260+0000 7efeaed9d640 1 -- 172.21.15.106:0/2296550921 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeb00c7980 con 0x7efeb00c7e30 2024-04-25T10:34:40.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.260+0000 7efeaf59e640 1 --2- 172.21.15.106:0/2296550921 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efeb00b4ce0 0x7efeb015ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:34:40.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.260+0000 7efeaed9d640 1 --2- 172.21.15.106:0/2296550921 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efeb00c7e30 0x7efeb00c8d90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7efeb00c7de0 tx=0x7efea002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dab52eb9ef4ca3f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:40.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.261+0000 7efeadd9b640 1 -- 172.21.15.106:0/2296550921 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea0036070 con 0x7efeb00c7e30 2024-04-25T10:34:40.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.261+0000 7efeadd9b640 1 -- 172.21.15.106:0/2296550921 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efea0031600 con 0x7efeb00c7e30 2024-04-25T10:34:40.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.261+0000 7efeadd9b640 1 -- 172.21.15.106:0/2296550921 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea00319d0 con 0x7efeb00c7e30 2024-04-25T10:34:40.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.261+0000 7efeb5182640 1 -- 172.21.15.106:0/2296550921 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efeb00c7e30 msgr2=0x7efeb00c8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:40.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.261+0000 7efeb5182640 1 --2- 172.21.15.106:0/2296550921 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efeb00c7e30 0x7efeb00c8d90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7efeb00c7de0 tx=0x7efea002a680 comp rx=0 tx=0).stop 2024-04-25T10:34:40.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.261+0000 7efeb5182640 1 -- 172.21.15.106:0/2296550921 shutdown_connections 2024-04-25T10:34:40.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.261+0000 7efeb5182640 1 --2- 172.21.15.106:0/2296550921 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efeb00b4ce0 0x7efeb015ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:40.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.261+0000 7efeb5182640 1 --2- 172.21.15.106:0/2296550921 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efeb00c7e30 0x7efeb00c8d90 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:40.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.261+0000 7efeb5182640 1 -- 172.21.15.106:0/2296550921 >> 172.21.15.106:0/2296550921 conn(0x7efeb0068b10 msgr2=0x7efeb00d1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:40.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.261+0000 7efeb5182640 1 -- 172.21.15.106:0/2296550921 shutdown_connections 2024-04-25T10:34:40.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.261+0000 7efeb5182640 1 -- 172.21.15.106:0/2296550921 wait complete. 2024-04-25T10:34:40.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.263+0000 7efeb5182640 1 Processor -- start 2024-04-25T10:34:40.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.263+0000 7efeb5182640 1 -- start start 2024-04-25T10:34:40.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.265+0000 7efeb5182640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efeb00b4ce0 0x7efeb0155670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:40.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.265+0000 7efeb5182640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efeb00c7e30 0x7efeb014d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:40.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.265+0000 7efeb5182640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efeb0156ef0 0x7efeb014dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:40.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.265+0000 7efeb5182640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7efeb006d710 con 0x7efeb00b4ce0 2024-04-25T10:34:40.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.265+0000 7efeb5182640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efeb00c72f0 con 0x7efeb00c7e30 2024-04-25T10:34:40.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.265+0000 7efeb5182640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7efeb006d890 con 0x7efeb0156ef0 2024-04-25T10:34:40.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.265+0000 7efeaed9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efeb00b4ce0 0x7efeb0155670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:40.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.265+0000 7efeaed9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efeb00b4ce0 0x7efeb0155670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:37272/0 (socket says 172.21.15.106:37272) 2024-04-25T10:34:40.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.265+0000 7efeaed9d640 1 -- 172.21.15.106:0/2717480507 learned_addr learned my addr 172.21.15.106:0/2717480507 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:34:40.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.265+0000 7efeaf59e640 1 --2- 172.21.15.106:0/2717480507 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efeb0156ef0 0x7efeb014dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:40.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.265+0000 7efeae59c640 1 --2- 172.21.15.106:0/2717480507 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efeb00c7e30 0x7efeb014d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:40.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.265+0000 7efeaed9d640 1 -- 172.21.15.106:0/2717480507 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efeb0156ef0 msgr2=0x7efeb014dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:40.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.265+0000 7efeaed9d640 1 --2- 172.21.15.106:0/2717480507 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efeb0156ef0 0x7efeb014dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:40.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.265+0000 7efeaed9d640 1 -- 172.21.15.106:0/2717480507 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efeb00c7e30 msgr2=0x7efeb014d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:40.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.265+0000 7efeaed9d640 1 --2- 172.21.15.106:0/2717480507 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efeb00c7e30 0x7efeb014d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:40.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.265+0000 7efeaed9d640 1 -- 172.21.15.106:0/2717480507 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeb01e9200 con 0x7efeb00b4ce0 2024-04-25T10:34:40.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.265+0000 7efeaed9d640 1 --2- 172.21.15.106:0/2717480507 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efeb00b4ce0 0x7efeb0155670 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7efea0009f40 tx=0x7efea0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:40.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.265+0000 7efe977fe640 1 -- 172.21.15.106:0/2717480507 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea003f020 con 0x7efeb00b4ce0 2024-04-25T10:34:40.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.265+0000 7efe977fe640 1 -- 172.21.15.106:0/2717480507 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efea0037040 con 0x7efeb00b4ce0 2024-04-25T10:34:40.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.265+0000 7efeb5182640 1 -- 172.21.15.106:0/2717480507 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efeb01f67d0 con 0x7efeb00b4ce0 2024-04-25T10:34:40.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.265+0000 7efe977fe640 1 -- 172.21.15.106:0/2717480507 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea0036070 con 0x7efeb00b4ce0 2024-04-25T10:34:40.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.265+0000 7efeb5182640 1 -- 172.21.15.106:0/2717480507 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efeb01f3400 con 0x7efeb00b4ce0 2024-04-25T10:34:40.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.266+0000 7efeb5182640 1 -- 172.21.15.106:0/2717480507 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efeb006a3d0 con 0x7efeb00b4ce0 2024-04-25T10:34:40.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.267+0000 7efe977fe640 1 -- 172.21.15.106:0/2717480507 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7efea0042020 con 0x7efeb00b4ce0 2024-04-25T10:34:40.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.268+0000 7efe977fe640 1 --2- 172.21.15.106:0/2717480507 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7efe9c0744b0 0x7efe9c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:40.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.268+0000 7efeae59c640 1 --2- 172.21.15.106:0/2717480507 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7efe9c0744b0 0x7efe9c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:40.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.268+0000 7efe977fe640 1 -- 172.21.15.106:0/2717480507 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7efea00b5800 con 0x7efeb00b4ce0 2024-04-25T10:34:40.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.268+0000 7efeae59c640 1 --2- 172.21.15.106:0/2717480507 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7efe9c0744b0 0x7efe9c0769c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7efea4007fd0 tx=0x7efea4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:40.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.272+0000 7efe977fe640 1 -- 172.21.15.106:0/2717480507 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efea0049030 con 0x7efeb00b4ce0 2024-04-25T10:34:40.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.518+0000 7efeb5182640 1 -- 172.21.15.106:0/2717480507 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7efeb015a5a0 con 0x7efeb00b4ce0 2024-04-25T10:34:40.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.518+0000 7efe977fe640 1 -- 172.21.15.106:0/2717480507 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1569 (secure 0 0 0) 0x7efea007ff10 con 0x7efeb00b4ce0 2024-04-25T10:34:40.520 INFO:teuthology.orchestra.run.smithi106.stdout:{"election_epoch":28,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"3ee77985-e8e4-41cf-81e9-7c774cf7891f","modified":"2024-04-25T10:29:33.446228Z","created":"2024-04-25T10:29:33.446228Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:3300","nonce":0},{"type":"v1","addr":"172.21.15.106:6789","nonce":0}]},"addr":"172.21.15.106:6789/0","public_addr":"172.21.15.106:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T10:34:40.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.520+0000 7efeb5182640 1 -- 172.21.15.106:0/2717480507 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7efe9c0744b0 msgr2=0x7efe9c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:40.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.520+0000 7efeb5182640 1 --2- 172.21.15.106:0/2717480507 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7efe9c0744b0 0x7efe9c0769c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7efea4007fd0 tx=0x7efea4009040 comp rx=0 tx=0).stop 2024-04-25T10:34:40.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.520+0000 7efeb5182640 1 -- 172.21.15.106:0/2717480507 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efeb00b4ce0 msgr2=0x7efeb0155670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:40.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.520+0000 7efeb5182640 1 --2- 172.21.15.106:0/2717480507 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efeb00b4ce0 0x7efeb0155670 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7efea0009f40 tx=0x7efea0004a40 comp rx=0 tx=0).stop 2024-04-25T10:34:40.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.520+0000 7efeb5182640 1 -- 172.21.15.106:0/2717480507 shutdown_connections 2024-04-25T10:34:40.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.520+0000 7efeb5182640 1 --2- 172.21.15.106:0/2717480507 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7efe9c0744b0 0x7efe9c0769c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:40.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.520+0000 7efeb5182640 1 --2- 172.21.15.106:0/2717480507 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efeb0156ef0 0x7efeb014dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:40.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.520+0000 7efeb5182640 1 --2- 172.21.15.106:0/2717480507 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efeb00c7e30 0x7efeb014d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:40.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.520+0000 7efeb5182640 1 --2- 172.21.15.106:0/2717480507 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efeb00b4ce0 0x7efeb0155670 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:40.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.520+0000 7efeb5182640 1 -- 172.21.15.106:0/2717480507 >> 172.21.15.106:0/2717480507 conn(0x7efeb0068b10 msgr2=0x7efeb015a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:40.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.520+0000 7efeb5182640 1 -- 172.21.15.106:0/2717480507 shutdown_connections 2024-04-25T10:34:40.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.520+0000 7efeb5182640 1 -- 172.21.15.106:0/2717480507 wait complete. 2024-04-25T10:34:40.533 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-04-25T10:34:40.534 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-04-25T10:34:40.543 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-04-25T10:34:40.543 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T10:34:40.543 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:34:40.559 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T10:34:40.559 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:34:40.614 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:34:40.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.760+0000 7fc63759d640 1 Processor -- start 2024-04-25T10:34:40.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.760+0000 7fc63759d640 1 -- start start 2024-04-25T10:34:40.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.761+0000 7fc63759d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc630156b60 0x7fc6300ce580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:40.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.761+0000 7fc63759d640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fc6300ceac0 0x7fc6301672e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:40.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.761+0000 7fc63759d640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6300cc870 con 0x7fc6300cd210 2024-04-25T10:34:40.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.761+0000 7fc63759d640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fc63015d3e0 con 0x7fc630156b60 2024-04-25T10:34:40.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.761+0000 7fc63759d640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fc63015d6c0 con 0x7fc6300ceac0 2024-04-25T10:34:40.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.761+0000 7fc635d9a640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc630156b60 0x7fc6300ce580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:40.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.761+0000 7fc63659b640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fc6300cd210 0x7fc6300c7080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:45246/0 (socket says 172.21.15.106:45246) 2024-04-25T10:34:40.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.762+0000 7fc63659b640 1 -- 172.21.15.106:0/2320334893 learned_addr learned my addr 172.21.15.106:0/2320334893 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:34:40.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.762+0000 7fc636d9c640 1 --2- 172.21.15.106:0/2320334893 >> v2:172.21.15.134:3300/0 conn(0x7fc6300ceac0 0x7fc6301672e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:40.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.762+0000 7fc635d9a640 1 -- 172.21.15.106:0/2320334893 >> v2:172.21.15.134:3300/0 conn(0x7fc6300ceac0 msgr2=0x7fc6301672e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:40.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.762+0000 7fc635d9a640 1 --2- 172.21.15.106:0/2320334893 >> v2:172.21.15.134:3300/0 conn(0x7fc6300ceac0 0x7fc6301672e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:40.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.762+0000 7fc635d9a640 1 -- 172.21.15.106:0/2320334893 >> v1:172.21.15.106:6789/0 conn(0x7fc6300cd210 legacy=0x7fc6300c7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:40.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.762+0000 7fc635599640 1 -- 172.21.15.106:0/2320334893 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 830154053 0 0) 0x7fc62c001110 con 0x7fc6300cd210 2024-04-25T10:34:40.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.762+0000 7fc635d9a640 1 -- 172.21.15.106:0/2320334893 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6300d1f80 con 0x7fc630156b60 2024-04-25T10:34:40.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.763+0000 7fc635d9a640 1 --2- 172.21.15.106:0/2320334893 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc630156b60 0x7fc6300ce580 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc63008fda0 tx=0x7fc62002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b215d62aa6ea6ba5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:40.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.763+0000 7fc635599640 1 -- 172.21.15.106:0/2320334893 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc620037070 con 0x7fc630156b60 2024-04-25T10:34:40.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.763+0000 7fc635599640 1 -- 172.21.15.106:0/2320334893 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc620032440 con 0x7fc630156b60 2024-04-25T10:34:40.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.763+0000 7fc635599640 1 -- 172.21.15.106:0/2320334893 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc620032810 con 0x7fc630156b60 2024-04-25T10:34:40.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.763+0000 7fc63759d640 1 -- 172.21.15.106:0/2320334893 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc630156b60 msgr2=0x7fc6300ce580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:40.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.763+0000 7fc63759d640 1 --2- 172.21.15.106:0/2320334893 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc630156b60 0x7fc6300ce580 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc63008fda0 tx=0x7fc62002a230 comp rx=0 tx=0).stop 2024-04-25T10:34:40.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.763+0000 7fc63759d640 1 -- 172.21.15.106:0/2320334893 shutdown_connections 2024-04-25T10:34:40.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.763+0000 7fc63759d640 1 --2- 172.21.15.106:0/2320334893 >> v2:172.21.15.134:3300/0 conn(0x7fc6300ceac0 0x7fc6301672e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:40.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.763+0000 7fc63759d640 1 --2- 172.21.15.106:0/2320334893 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc630156b60 0x7fc6300ce580 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:40.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.763+0000 7fc63759d640 1 -- 172.21.15.106:0/2320334893 >> 172.21.15.106:0/2320334893 conn(0x7fc63006e650 msgr2=0x7fc630073cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:40.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.763+0000 7fc63759d640 1 -- 172.21.15.106:0/2320334893 shutdown_connections 2024-04-25T10:34:40.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.764+0000 7fc63759d640 1 -- 172.21.15.106:0/2320334893 wait complete. 2024-04-25T10:34:40.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.765+0000 7fc63759d640 1 Processor -- start 2024-04-25T10:34:40.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.765+0000 7fc63759d640 1 -- start start 2024-04-25T10:34:40.772 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.766+0000 7fc63759d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc6300cd210 0x7fc630157d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:40.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.766+0000 7fc63759d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc6300ceac0 0x7fc6300d0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:40.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.766+0000 7fc63759d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc630156b60 0x7fc63015dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:40.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.766+0000 7fc63759d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc630156f70 con 0x7fc6300cd210 2024-04-25T10:34:40.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.766+0000 7fc63759d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc6300d2540 con 0x7fc6300ceac0 2024-04-25T10:34:40.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.766+0000 7fc63759d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc6300d2a60 con 0x7fc630156b60 2024-04-25T10:34:40.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.766+0000 7fc63659b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc6300cd210 0x7fc630157d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:40.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.766+0000 7fc63659b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc6300cd210 0x7fc630157d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:37296/0 (socket says 172.21.15.106:37296) 2024-04-25T10:34:40.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.767+0000 7fc63659b640 1 -- 172.21.15.106:0/3918291128 learned_addr learned my addr 172.21.15.106:0/3918291128 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:34:40.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.767+0000 7fc635d9a640 1 --2- 172.21.15.106:0/3918291128 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc6300ceac0 0x7fc6300d0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:40.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.767+0000 7fc636d9c640 1 --2- 172.21.15.106:0/3918291128 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc630156b60 0x7fc63015dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:40.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.767+0000 7fc63659b640 1 -- 172.21.15.106:0/3918291128 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc630156b60 msgr2=0x7fc63015dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:40.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.767+0000 7fc63659b640 1 --2- 172.21.15.106:0/3918291128 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc630156b60 0x7fc63015dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:40.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.767+0000 7fc63659b640 1 -- 172.21.15.106:0/3918291128 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc6300ceac0 msgr2=0x7fc6300d0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:40.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.767+0000 7fc63659b640 1 --2- 172.21.15.106:0/3918291128 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc6300ceac0 0x7fc6300d0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:40.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.767+0000 7fc63659b640 1 -- 172.21.15.106:0/3918291128 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6300cfc80 con 0x7fc6300cd210 2024-04-25T10:34:40.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.767+0000 7fc635d9a640 1 --2- 172.21.15.106:0/3918291128 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc6300ceac0 0x7fc6300d0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:34:40.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.767+0000 7fc636d9c640 1 --2- 172.21.15.106:0/3918291128 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc630156b60 0x7fc63015dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:34:40.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.767+0000 7fc63659b640 1 --2- 172.21.15.106:0/3918291128 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc6300cd210 0x7fc630157d40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc62c00c440 tx=0x7fc62c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:40.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.767+0000 7fc626ffd640 1 -- 172.21.15.106:0/3918291128 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc62c018070 con 0x7fc6300cd210 2024-04-25T10:34:40.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.767+0000 7fc626ffd640 1 -- 172.21.15.106:0/3918291128 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc62c0074e0 con 0x7fc6300cd210 2024-04-25T10:34:40.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.767+0000 7fc63759d640 1 -- 172.21.15.106:0/3918291128 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6300d11f0 con 0x7fc6300cd210 2024-04-25T10:34:40.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.767+0000 7fc626ffd640 1 -- 172.21.15.106:0/3918291128 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc62c004a20 con 0x7fc6300cd210 2024-04-25T10:34:40.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.767+0000 7fc63759d640 1 -- 172.21.15.106:0/3918291128 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6300d01d0 con 0x7fc6300cd210 2024-04-25T10:34:40.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.768+0000 7fc63759d640 1 -- 172.21.15.106:0/3918291128 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6300b71c0 con 0x7fc6300cd210 2024-04-25T10:34:40.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.769+0000 7fc626ffd640 1 -- 172.21.15.106:0/3918291128 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc62c01b070 con 0x7fc6300cd210 2024-04-25T10:34:40.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.769+0000 7fc626ffd640 1 --2- 172.21.15.106:0/3918291128 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fc6140744b0 0x7fc6140769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:40.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.769+0000 7fc626ffd640 1 -- 172.21.15.106:0/3918291128 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fc62c094480 con 0x7fc6300cd210 2024-04-25T10:34:40.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.769+0000 7fc635d9a640 1 --2- 172.21.15.106:0/3918291128 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fc6140744b0 0x7fc6140769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:40.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.769+0000 7fc635d9a640 1 --2- 172.21.15.106:0/3918291128 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fc6140744b0 0x7fc6140769c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc620034000 tx=0x7fc62002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:40.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.774+0000 7fc626ffd640 1 -- 172.21.15.106:0/3918291128 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc6300b71c0 con 0x7fc6300cd210 2024-04-25T10:34:40.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.954+0000 7fc63759d640 1 -- 172.21.15.106:0/3918291128 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc6300b6f50 con 0x7fc6300cd210 2024-04-25T10:34:40.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.955+0000 7fc626ffd640 1 -- 172.21.15.106:0/3918291128 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16890 (secure 0 0 0) 0x7fc6300cc870 con 0x7fc6300cd210 2024-04-25T10:34:40.957 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:34:40.957 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":22,"fsid":"3ee77985-e8e4-41cf-81e9-7c774cf7891f","created":"2024-04-25T10:30:07.981054+0000","modified":"2024-04-25T10:31:13.621853+0000","last_up_change":"2024-04-25T10:30:19.184325+0000","last_in_change":"2024-04-25T10:30:10.217553+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:30:20.111046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:30:30.314914+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:30:30.715500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a5962cb1-dbbd-47b1-8d15-9bae114bfdf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1240717079},{"type":"v1","addr":"172.21.15.106:6801","nonce":1240717079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1240717079},{"type":"v1","addr":"172.21.15.106:6803","nonce":1240717079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1240717079},{"type":"v1","addr":"172.21.15.106:6807","nonce":1240717079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1240717079},{"type":"v1","addr":"172.21.15.106:6805","nonce":1240717079}]},"public_addr":"172.21.15.106:6801/1240717079","cluster_addr":"172.21.15.106:6803/1240717079","heartbeat_back_addr":"172.21.15.106:6807/1240717079","heartbeat_front_addr":"172.21.15.106:6805/1240717079","state":["exists","up"]},{"osd":1,"uuid":"94436e13-4769-4752-8397-b5e396a6f7ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4089653051},{"type":"v1","addr":"172.21.15.106:6825","nonce":4089653051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4089653051},{"type":"v1","addr":"172.21.15.106:6827","nonce":4089653051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4089653051},{"type":"v1","addr":"172.21.15.106:6831","nonce":4089653051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4089653051},{"type":"v1","addr":"172.21.15.106:6829","nonce":4089653051}]},"public_addr":"172.21.15.106:6825/4089653051","cluster_addr":"172.21.15.106:6827/4089653051","heartbeat_back_addr":"172.21.15.106:6831/4089653051","heartbeat_front_addr":"172.21.15.106:6829/4089653051","state":["exists","up"]},{"osd":2,"uuid":"429aae0b-70c6-4e7b-9043-6a11334c7efc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":1441018239},{"type":"v1","addr":"172.21.15.106:6809","nonce":1441018239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1441018239},{"type":"v1","addr":"172.21.15.106:6811","nonce":1441018239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1441018239},{"type":"v1","addr":"172.21.15.106:6815","nonce":1441018239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":1441018239},{"type":"v1","addr":"172.21.15.106:6813","nonce":1441018239}]},"public_addr":"172.21.15.106:6809/1441018239","cluster_addr":"172.21.15.106:6811/1441018239","heartbeat_back_addr":"172.21.15.106:6815/1441018239","heartbeat_front_addr":"172.21.15.106:6813/1441018239","state":["exists","up"]},{"osd":3,"uuid":"9593a4d1-772f-4667-ab0d-2982f841e342","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":1846968711},{"type":"v1","addr":"172.21.15.106:6817","nonce":1846968711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1846968711},{"type":"v1","addr":"172.21.15.106:6819","nonce":1846968711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":1846968711},{"type":"v1","addr":"172.21.15.106:6823","nonce":1846968711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":1846968711},{"type":"v1","addr":"172.21.15.106:6821","nonce":1846968711}]},"public_addr":"172.21.15.106:6817/1846968711","cluster_addr":"172.21.15.106:6819/1846968711","heartbeat_back_addr":"172.21.15.106:6823/1846968711","heartbeat_front_addr":"172.21.15.106:6821/1846968711","state":["exists","up"]},{"osd":4,"uuid":"1b61f16d-5c60-480a-b449-c745aa3f6b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":729224739},{"type":"v1","addr":"172.21.15.134:6825","nonce":729224739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":729224739},{"type":"v1","addr":"172.21.15.134:6827","nonce":729224739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":729224739},{"type":"v1","addr":"172.21.15.134:6831","nonce":729224739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":729224739},{"type":"v1","addr":"172.21.15.134:6829","nonce":729224739}]},"public_addr":"172.21.15.134:6825/729224739","cluster_addr":"172.21.15.134:6827/729224739","heartbeat_back_addr":"172.21.15.134:6831/729224739","heartbeat_front_addr":"172.21.15.134:6829/729224739","state":["exists","up"]},{"osd":5,"uuid":"944cdd7f-47c0-4b71-b7e0-d093542f4528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3132275652},{"type":"v1","addr":"172.21.15.134:6809","nonce":3132275652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3132275652},{"type":"v1","addr":"172.21.15.134:6811","nonce":3132275652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3132275652},{"type":"v1","addr":"172.21.15.134:6815","nonce":3132275652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3132275652},{"type":"v1","addr":"172.21.15.134:6813","nonce":3132275652}]},"public_addr":"172.21.15.134:6809/3132275652","cluster_addr":"172.21.15.134:6811/3132275652","heartbeat_back_addr":"172.21.15.134:6815/3132275652","heartbeat_front_addr":"172.21.15.134:6813/3132275652","state":["exists","up"]},{"osd":6,"uuid":"cc2446e2-1d21-4135-ac04-6af7c990058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":992702668},{"type":"v1","addr":"172.21.15.134:6801","nonce":992702668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":992702668},{"type":"v1","addr":"172.21.15.134:6803","nonce":992702668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":992702668},{"type":"v1","addr":"172.21.15.134:6807","nonce":992702668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":992702668},{"type":"v1","addr":"172.21.15.134:6805","nonce":992702668}]},"public_addr":"172.21.15.134:6801/992702668","cluster_addr":"172.21.15.134:6803/992702668","heartbeat_back_addr":"172.21.15.134:6807/992702668","heartbeat_front_addr":"172.21.15.134:6805/992702668","state":["exists","up"]},{"osd":7,"uuid":"3df1fc7a-82fc-4257-8629-98027b6e2aed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":3291062284},{"type":"v1","addr":"172.21.15.134:6817","nonce":3291062284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":3291062284},{"type":"v1","addr":"172.21.15.134:6819","nonce":3291062284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":3291062284},{"type":"v1","addr":"172.21.15.134:6823","nonce":3291062284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":3291062284},{"type":"v1","addr":"172.21.15.134:6821","nonce":3291062284}]},"public_addr":"172.21.15.134:6817/3291062284","cluster_addr":"172.21.15.134:6819/3291062284","heartbeat_back_addr":"172.21.15.134:6823/3291062284","heartbeat_front_addr":"172.21.15.134:6821/3291062284","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.403941+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.382563+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.411544+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.330444+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.405430+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.387737+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.408922+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.454450+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":2}]},{"pgid":"3.2b","mappings":[{"from":0,"to":1}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:34:40.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.957+0000 7fc63759d640 1 -- 172.21.15.106:0/3918291128 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fc6140744b0 msgr2=0x7fc6140769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:40.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.957+0000 7fc63759d640 1 --2- 172.21.15.106:0/3918291128 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fc6140744b0 0x7fc6140769c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc620034000 tx=0x7fc62002e000 comp rx=0 tx=0).stop 2024-04-25T10:34:40.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.957+0000 7fc63759d640 1 -- 172.21.15.106:0/3918291128 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc6300cd210 msgr2=0x7fc630157d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:40.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.957+0000 7fc63759d640 1 --2- 172.21.15.106:0/3918291128 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc6300cd210 0x7fc630157d40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc62c00c440 tx=0x7fc62c00c900 comp rx=0 tx=0).stop 2024-04-25T10:34:40.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.957+0000 7fc63759d640 1 -- 172.21.15.106:0/3918291128 shutdown_connections 2024-04-25T10:34:40.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.957+0000 7fc63759d640 1 --2- 172.21.15.106:0/3918291128 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fc6140744b0 0x7fc6140769c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:40.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.957+0000 7fc63759d640 1 --2- 172.21.15.106:0/3918291128 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc630156b60 0x7fc63015dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:40.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.957+0000 7fc63759d640 1 --2- 172.21.15.106:0/3918291128 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc6300ceac0 0x7fc6300d0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:40.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.957+0000 7fc63759d640 1 --2- 172.21.15.106:0/3918291128 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc6300cd210 0x7fc630157d40 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:40.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.957+0000 7fc63759d640 1 -- 172.21.15.106:0/3918291128 >> 172.21.15.106:0/3918291128 conn(0x7fc63006e650 msgr2=0x7fc630155df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:40.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.957+0000 7fc63759d640 1 -- 172.21.15.106:0/3918291128 shutdown_connections 2024-04-25T10:34:40.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:40.957+0000 7fc63759d640 1 -- 172.21.15.106:0/3918291128 wait complete. 2024-04-25T10:34:40.975 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:30:20.111046+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:30:30.314914+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:30:30.715500+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:34:40.975 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:34:41.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.084+0000 7fead12b6640 1 Processor -- start 2024-04-25T10:34:41.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.084+0000 7fead12b6640 1 -- start start 2024-04-25T10:34:41.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.085+0000 7fead12b6640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7feacc0c7e30 0x7feacc0c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:41.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.085+0000 7fead12b6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feacc0b4ce0 0x7feacc15ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:41.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.085+0000 7fead12b6640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feacc0ce630 con 0x7feacc156ef0 2024-04-25T10:34:41.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.085+0000 7fead12b6640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7feacc0b50f0 con 0x7feacc0c7e30 2024-04-25T10:34:41.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.085+0000 7fead12b6640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7feacc0c7c60 con 0x7feacc0b4ce0 2024-04-25T10:34:41.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.086+0000 7feacaffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7feacc0c7e30 0x7feacc0c8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:41.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.086+0000 7feacaffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7feacc0c7e30 0x7feacc0c8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:37300/0 (socket says 172.21.15.106:37300) 2024-04-25T10:34:41.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.086+0000 7feacaffd640 1 -- 172.21.15.106:0/3594568928 learned_addr learned my addr 172.21.15.106:0/3594568928 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:34:41.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.086+0000 7feacb7fe640 1 --2- 172.21.15.106:0/3594568928 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feacc0b4ce0 0x7feacc15ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:41.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.086+0000 7feacaffd640 1 -- 172.21.15.106:0/3594568928 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feacc0b4ce0 msgr2=0x7feacc15ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:41.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.086+0000 7feacaffd640 1 --2- 172.21.15.106:0/3594568928 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feacc0b4ce0 0x7feacc15ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:41.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.086+0000 7feacaffd640 1 -- 172.21.15.106:0/3594568928 >> v1:172.21.15.134:6789/0 conn(0x7feacc156ef0 legacy=0x7feacc06d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:41.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.086+0000 7feacaffd640 1 -- 172.21.15.106:0/3594568928 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feacc0c7980 con 0x7feacc0c7e30 2024-04-25T10:34:41.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.087+0000 7feacaffd640 1 --2- 172.21.15.106:0/3594568928 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feacc0c7e30 0x7feacc0c8d90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7feacc0c7de0 tx=0x7feabc02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa585842f2b6884a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:41.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.087+0000 7feac9ffb640 1 -- 172.21.15.106:0/3594568928 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feabc036070 con 0x7feacc0c7e30 2024-04-25T10:34:41.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.087+0000 7feac9ffb640 1 -- 172.21.15.106:0/3594568928 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feabc031600 con 0x7feacc0c7e30 2024-04-25T10:34:41.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.087+0000 7feac9ffb640 1 -- 172.21.15.106:0/3594568928 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feabc0319d0 con 0x7feacc0c7e30 2024-04-25T10:34:41.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.087+0000 7fead12b6640 1 -- 172.21.15.106:0/3594568928 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feacc0c7e30 msgr2=0x7feacc0c8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:41.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.087+0000 7fead12b6640 1 --2- 172.21.15.106:0/3594568928 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feacc0c7e30 0x7feacc0c8d90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7feacc0c7de0 tx=0x7feabc02a680 comp rx=0 tx=0).stop 2024-04-25T10:34:41.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.087+0000 7fead12b6640 1 -- 172.21.15.106:0/3594568928 shutdown_connections 2024-04-25T10:34:41.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.087+0000 7fead12b6640 1 --2- 172.21.15.106:0/3594568928 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feacc0b4ce0 0x7feacc15ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:41.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.087+0000 7fead12b6640 1 --2- 172.21.15.106:0/3594568928 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feacc0c7e30 0x7feacc0c8d90 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:41.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.087+0000 7fead12b6640 1 -- 172.21.15.106:0/3594568928 >> 172.21.15.106:0/3594568928 conn(0x7feacc068b10 msgr2=0x7feacc0d1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:41.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.088+0000 7fead12b6640 1 -- 172.21.15.106:0/3594568928 shutdown_connections 2024-04-25T10:34:41.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.088+0000 7fead12b6640 1 -- 172.21.15.106:0/3594568928 wait complete. 2024-04-25T10:34:41.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.089+0000 7fead12b6640 1 Processor -- start 2024-04-25T10:34:41.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.089+0000 7fead12b6640 1 -- start start 2024-04-25T10:34:41.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.091+0000 7fead12b6640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feacc0b4ce0 0x7feacc14ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:41.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.091+0000 7fead12b6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feacc0c7e30 0x7feacc0d0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:41.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.091+0000 7fead12b6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feacc156ef0 0x7feacc154f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:41.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.091+0000 7fead12b6640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7feacc06d710 con 0x7feacc0b4ce0 2024-04-25T10:34:41.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.091+0000 7fead12b6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7feacc0c72f0 con 0x7feacc156ef0 2024-04-25T10:34:41.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.091+0000 7fead12b6640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7feacc06d890 con 0x7feacc0c7e30 2024-04-25T10:34:41.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.091+0000 7feacaffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feacc0b4ce0 0x7feacc14ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:41.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.091+0000 7feacaffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feacc0b4ce0 0x7feacc14ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:37304/0 (socket says 172.21.15.106:37304) 2024-04-25T10:34:41.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.091+0000 7feacaffd640 1 -- 172.21.15.106:0/3563996859 learned_addr learned my addr 172.21.15.106:0/3563996859 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:34:41.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.091+0000 7feaca7fc640 1 --2- 172.21.15.106:0/3563996859 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feacc0c7e30 0x7feacc0d0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:41.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.091+0000 7feacb7fe640 1 --2- 172.21.15.106:0/3563996859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feacc156ef0 0x7feacc154f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:41.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.091+0000 7feacaffd640 1 -- 172.21.15.106:0/3563996859 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feacc0c7e30 msgr2=0x7feacc0d0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:41.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.091+0000 7feacaffd640 1 --2- 172.21.15.106:0/3563996859 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feacc0c7e30 0x7feacc0d0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:41.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.091+0000 7feacaffd640 1 -- 172.21.15.106:0/3563996859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feacc156ef0 msgr2=0x7feacc154f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:41.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.091+0000 7feacaffd640 1 --2- 172.21.15.106:0/3563996859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feacc156ef0 0x7feacc154f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:41.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.091+0000 7feacaffd640 1 -- 172.21.15.106:0/3563996859 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feacc0d13f0 con 0x7feacc0b4ce0 2024-04-25T10:34:41.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.091+0000 7feacaffd640 1 --2- 172.21.15.106:0/3563996859 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feacc0b4ce0 0x7feacc14ff10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7feabc034040 tx=0x7feabc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:41.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.091+0000 7feab37fe640 1 -- 172.21.15.106:0/3563996859 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feabc036050 con 0x7feacc0b4ce0 2024-04-25T10:34:41.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.091+0000 7feab37fe640 1 -- 172.21.15.106:0/3563996859 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feabc042070 con 0x7feacc0b4ce0 2024-04-25T10:34:41.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.091+0000 7fead12b6640 1 -- 172.21.15.106:0/3563996859 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feacc1f39e0 con 0x7feacc0b4ce0 2024-04-25T10:34:41.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.091+0000 7feab37fe640 1 -- 172.21.15.106:0/3563996859 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feabc009510 con 0x7feacc0b4ce0 2024-04-25T10:34:41.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.091+0000 7fead12b6640 1 -- 172.21.15.106:0/3563996859 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feacc1f3ca0 con 0x7feacc0b4ce0 2024-04-25T10:34:41.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.092+0000 7fead12b6640 1 -- 172.21.15.106:0/3563996859 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feacc06a3d0 con 0x7feacc0b4ce0 2024-04-25T10:34:41.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.093+0000 7feab37fe640 1 -- 172.21.15.106:0/3563996859 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7feabc007c50 con 0x7feacc0b4ce0 2024-04-25T10:34:41.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.093+0000 7feab37fe640 1 --2- 172.21.15.106:0/3563996859 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7feab80744e0 0x7feab80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:41.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.094+0000 7feab37fe640 1 -- 172.21.15.106:0/3563996859 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7feabc0b6140 con 0x7feacc0b4ce0 2024-04-25T10:34:41.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.094+0000 7feaca7fc640 1 --2- 172.21.15.106:0/3563996859 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7feab80744e0 0x7feab80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:41.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.094+0000 7feaca7fc640 1 --2- 172.21.15.106:0/3563996859 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7feab80744e0 0x7feab80769f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7feacc1f0af0 tx=0x7feac0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:41.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.098+0000 7feab37fe640 1 -- 172.21.15.106:0/3563996859 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7feabc080850 con 0x7feacc0b4ce0 2024-04-25T10:34:41.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.272+0000 7fead12b6640 1 -- 172.21.15.106:0/3563996859 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7feacc0ce2e0 con 0x7feacc0b4ce0 2024-04-25T10:34:41.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.273+0000 7feab37fe640 1 -- 172.21.15.106:0/3563996859 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7feabc080150 con 0x7feacc0b4ce0 2024-04-25T10:34:41.274 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-25T10:34:41.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.274+0000 7fead12b6640 1 -- 172.21.15.106:0/3563996859 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7feab80744e0 msgr2=0x7feab80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:41.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.274+0000 7fead12b6640 1 --2- 172.21.15.106:0/3563996859 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7feab80744e0 0x7feab80769f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7feacc1f0af0 tx=0x7feac0009040 comp rx=0 tx=0).stop 2024-04-25T10:34:41.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.274+0000 7fead12b6640 1 -- 172.21.15.106:0/3563996859 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feacc0b4ce0 msgr2=0x7feacc14ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:41.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.274+0000 7fead12b6640 1 --2- 172.21.15.106:0/3563996859 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feacc0b4ce0 0x7feacc14ff10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7feabc034040 tx=0x7feabc004a40 comp rx=0 tx=0).stop 2024-04-25T10:34:41.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.274+0000 7fead12b6640 1 -- 172.21.15.106:0/3563996859 shutdown_connections 2024-04-25T10:34:41.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.274+0000 7fead12b6640 1 --2- 172.21.15.106:0/3563996859 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7feab80744e0 0x7feab80769f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:41.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.274+0000 7fead12b6640 1 --2- 172.21.15.106:0/3563996859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feacc156ef0 0x7feacc154f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:41.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.274+0000 7fead12b6640 1 --2- 172.21.15.106:0/3563996859 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feacc0c7e30 0x7feacc0d0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:41.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.274+0000 7fead12b6640 1 --2- 172.21.15.106:0/3563996859 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feacc0b4ce0 0x7feacc14ff10 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:41.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.275+0000 7fead12b6640 1 -- 172.21.15.106:0/3563996859 >> 172.21.15.106:0/3563996859 conn(0x7feacc068b10 msgr2=0x7feacc15cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:41.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.275+0000 7fead12b6640 1 -- 172.21.15.106:0/3563996859 shutdown_connections 2024-04-25T10:34:41.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.275+0000 7fead12b6640 1 -- 172.21.15.106:0/3563996859 wait complete. 2024-04-25T10:34:41.288 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:34:41.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.397+0000 7f591b24a640 1 Processor -- start 2024-04-25T10:34:41.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.397+0000 7f591b24a640 1 -- start start 2024-04-25T10:34:41.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.398+0000 7f591b24a640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f5914154c60 0x7f5914070ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:41.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.398+0000 7f591b24a640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f591415e020 0x7f591415e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:41.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.398+0000 7f591b24a640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59140caa00 con 0x7f5914071220 2024-04-25T10:34:41.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.398+0000 7f591b24a640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f59140cb880 con 0x7f5914154c60 2024-04-25T10:34:41.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.398+0000 7f591b24a640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f59140cac70 con 0x7f591415e020 2024-04-25T10:34:41.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.398+0000 7f5918fe6640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f5914154c60 0x7f5914070ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:41.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.398+0000 7f5918fe6640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f5914154c60 0x7f5914070ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:37316/0 (socket says 172.21.15.106:37316) 2024-04-25T10:34:41.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.398+0000 7f5913fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f5914071220 0x7f59140c5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:45258/0 (socket says 172.21.15.106:45258) 2024-04-25T10:34:41.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.398+0000 7f5918fe6640 1 -- 172.21.15.106:0/2664584641 learned_addr learned my addr 172.21.15.106:0/2664584641 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:34:41.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.398+0000 7f59197e7640 1 --2- 172.21.15.106:0/2664584641 >> v2:172.21.15.134:3300/0 conn(0x7f591415e020 0x7f591415e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:41.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.398+0000 7f5918fe6640 1 -- 172.21.15.106:0/2664584641 >> v2:172.21.15.134:3300/0 conn(0x7f591415e020 msgr2=0x7f591415e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:41.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.398+0000 7f5918fe6640 1 --2- 172.21.15.106:0/2664584641 >> v2:172.21.15.134:3300/0 conn(0x7f591415e020 0x7f591415e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:41.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.398+0000 7f5918fe6640 1 -- 172.21.15.106:0/2664584641 >> v1:172.21.15.106:6789/0 conn(0x7f5914071220 legacy=0x7f59140c5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:41.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.399+0000 7f5918fe6640 1 -- 172.21.15.106:0/2664584641 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59140caf50 con 0x7f5914154c60 2024-04-25T10:34:41.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.398+0000 7f59137fe640 1 -- 172.21.15.106:0/2664584641 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2968495948 0 0) 0x7f59140caa00 con 0x7f5914071220 2024-04-25T10:34:41.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.399+0000 7f5918fe6640 1 --2- 172.21.15.106:0/2664584641 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5914154c60 0x7f5914070ce0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5914074a40 tx=0x7f590402e570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fea428dc9c60be38 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:41.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.399+0000 7f59137fe640 1 -- 172.21.15.106:0/2664584641 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f590403a070 con 0x7f5914154c60 2024-04-25T10:34:41.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.399+0000 7f59137fe640 1 -- 172.21.15.106:0/2664584641 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f590400b650 con 0x7f5914154c60 2024-04-25T10:34:41.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.399+0000 7f59137fe640 1 -- 172.21.15.106:0/2664584641 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f590400ba20 con 0x7f5914154c60 2024-04-25T10:34:41.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.400+0000 7f591b24a640 1 -- 172.21.15.106:0/2664584641 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5914154c60 msgr2=0x7f5914070ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:41.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.400+0000 7f591b24a640 1 --2- 172.21.15.106:0/2664584641 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5914154c60 0x7f5914070ce0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5914074a40 tx=0x7f590402e570 comp rx=0 tx=0).stop 2024-04-25T10:34:41.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.400+0000 7f591b24a640 1 -- 172.21.15.106:0/2664584641 shutdown_connections 2024-04-25T10:34:41.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.400+0000 7f591b24a640 1 --2- 172.21.15.106:0/2664584641 >> v2:172.21.15.134:3300/0 conn(0x7f591415e020 0x7f591415e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:41.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.400+0000 7f591b24a640 1 --2- 172.21.15.106:0/2664584641 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5914154c60 0x7f5914070ce0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:41.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.400+0000 7f591b24a640 1 -- 172.21.15.106:0/2664584641 >> 172.21.15.106:0/2664584641 conn(0x7f59140cff60 msgr2=0x7f59140d0370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:41.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.400+0000 7f591b24a640 1 -- 172.21.15.106:0/2664584641 shutdown_connections 2024-04-25T10:34:41.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.400+0000 7f591b24a640 1 -- 172.21.15.106:0/2664584641 wait complete. 2024-04-25T10:34:41.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.402+0000 7f591b24a640 1 Processor -- start 2024-04-25T10:34:41.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.402+0000 7f591b24a640 1 -- start start 2024-04-25T10:34:41.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.403+0000 7f591b24a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5914071220 0x7f59141eef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:41.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.403+0000 7f591b24a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5914154c60 0x7f591414ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:41.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.403+0000 7f591b24a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f591415e020 0x7f591414cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:41.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.403+0000 7f591b24a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f59140c5b10 con 0x7f591415e020 2024-04-25T10:34:41.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.403+0000 7f591b24a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f59140cb510 con 0x7f5914071220 2024-04-25T10:34:41.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.403+0000 7f591b24a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f59140c5c90 con 0x7f5914154c60 2024-04-25T10:34:41.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.403+0000 7f59197e7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f591415e020 0x7f591414cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:41.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.403+0000 7f59197e7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f591415e020 0x7f591414cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:37326/0 (socket says 172.21.15.106:37326) 2024-04-25T10:34:41.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.403+0000 7f59197e7640 1 -- 172.21.15.106:0/3532846856 learned_addr learned my addr 172.21.15.106:0/3532846856 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:34:41.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.403+0000 7f5918fe6640 1 --2- 172.21.15.106:0/3532846856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5914071220 0x7f59141eef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:41.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.404+0000 7f59197e7640 1 -- 172.21.15.106:0/3532846856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5914154c60 msgr2=0x7f591414ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:41.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.404+0000 7f59197e7640 1 --2- 172.21.15.106:0/3532846856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5914154c60 0x7f591414ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:41.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.404+0000 7f59197e7640 1 -- 172.21.15.106:0/3532846856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5914071220 msgr2=0x7f59141eef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:41.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.404+0000 7f59197e7640 1 --2- 172.21.15.106:0/3532846856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5914071220 0x7f59141eef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:41.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.404+0000 7f59197e7640 1 -- 172.21.15.106:0/3532846856 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59140ceab0 con 0x7f591415e020 2024-04-25T10:34:41.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.404+0000 7f59197e7640 1 --2- 172.21.15.106:0/3532846856 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f591415e020 0x7f591414cfa0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f590c007d80 tx=0x7f590c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:41.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.404+0000 7f59117fa640 1 -- 172.21.15.106:0/3532846856 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f590c014420 con 0x7f591415e020 2024-04-25T10:34:41.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.404+0000 7f59117fa640 1 -- 172.21.15.106:0/3532846856 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f590c019070 con 0x7f591415e020 2024-04-25T10:34:41.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.404+0000 7f591b24a640 1 -- 172.21.15.106:0/3532846856 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59141ef470 con 0x7f591415e020 2024-04-25T10:34:41.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.404+0000 7f59117fa640 1 -- 172.21.15.106:0/3532846856 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f590c014800 con 0x7f591415e020 2024-04-25T10:34:41.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.404+0000 7f591b24a640 1 -- 172.21.15.106:0/3532846856 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59140ce310 con 0x7f591415e020 2024-04-25T10:34:41.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.405+0000 7f591b24a640 1 -- 172.21.15.106:0/3532846856 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f591406a3d0 con 0x7f591415e020 2024-04-25T10:34:41.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.406+0000 7f59117fa640 1 -- 172.21.15.106:0/3532846856 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f590c0149a0 con 0x7f591415e020 2024-04-25T10:34:41.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.406+0000 7f59117fa640 1 --2- 172.21.15.106:0/3532846856 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f59000744b0 0x7f59000769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:41.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.406+0000 7f59117fa640 1 -- 172.21.15.106:0/3532846856 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f590c0959f0 con 0x7f591415e020 2024-04-25T10:34:41.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.406+0000 7f5918fe6640 1 --2- 172.21.15.106:0/3532846856 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f59000744b0 0x7f59000769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:41.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.407+0000 7f5918fe6640 1 --2- 172.21.15.106:0/3532846856 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f59000744b0 0x7f59000769c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5904037000 tx=0x7f590400d000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:41.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.411+0000 7f59117fa640 1 -- 172.21.15.106:0/3532846856 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f590c060190 con 0x7f591415e020 2024-04-25T10:34:41.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.586+0000 7f591b24a640 1 -- 172.21.15.106:0/3532846856 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f59140ce0a0 con 0x7f591415e020 2024-04-25T10:34:41.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.586+0000 7f59117fa640 1 -- 172.21.15.106:0/3532846856 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f590c05fa90 con 0x7f591415e020 2024-04-25T10:34:41.589 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-25T10:34:41.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.589+0000 7f591b24a640 1 -- 172.21.15.106:0/3532846856 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f59000744b0 msgr2=0x7f59000769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:41.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.589+0000 7f591b24a640 1 --2- 172.21.15.106:0/3532846856 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f59000744b0 0x7f59000769c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5904037000 tx=0x7f590400d000 comp rx=0 tx=0).stop 2024-04-25T10:34:41.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.589+0000 7f591b24a640 1 -- 172.21.15.106:0/3532846856 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f591415e020 msgr2=0x7f591414cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:41.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.589+0000 7f591b24a640 1 --2- 172.21.15.106:0/3532846856 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f591415e020 0x7f591414cfa0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f590c007d80 tx=0x7f590c00e4b0 comp rx=0 tx=0).stop 2024-04-25T10:34:41.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.589+0000 7f591b24a640 1 -- 172.21.15.106:0/3532846856 shutdown_connections 2024-04-25T10:34:41.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.589+0000 7f591b24a640 1 --2- 172.21.15.106:0/3532846856 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f59000744b0 0x7f59000769c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:41.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.589+0000 7f591b24a640 1 --2- 172.21.15.106:0/3532846856 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f591415e020 0x7f591414cfa0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:41.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.589+0000 7f591b24a640 1 --2- 172.21.15.106:0/3532846856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5914154c60 0x7f591414ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:41.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.589+0000 7f591b24a640 1 --2- 172.21.15.106:0/3532846856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5914071220 0x7f59141eef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:41.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.589+0000 7f591b24a640 1 -- 172.21.15.106:0/3532846856 >> 172.21.15.106:0/3532846856 conn(0x7f59140cff60 msgr2=0x7f5914153740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:41.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.589+0000 7f591b24a640 1 -- 172.21.15.106:0/3532846856 shutdown_connections 2024-04-25T10:34:41.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.589+0000 7f591b24a640 1 -- 172.21.15.106:0/3532846856 wait complete. 2024-04-25T10:34:41.602 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:34:41.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.712+0000 7fb27fdce640 1 Processor -- start 2024-04-25T10:34:41.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.712+0000 7fb27fdce640 1 -- start start 2024-04-25T10:34:41.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.713+0000 7fb27fdce640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fb2780cbff0 0x7fb2780cb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:41.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.713+0000 7fb27fdce640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb278156ec0 0x7fb2780c61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:41.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.713+0000 7fb27fdce640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb27815dae0 con 0x7fb2780c66e0 2024-04-25T10:34:41.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.713+0000 7fb27fdce640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fb278161270 con 0x7fb278156ec0 2024-04-25T10:34:41.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.713+0000 7fb27fdce640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fb2781604b0 con 0x7fb2780cbff0 2024-04-25T10:34:41.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.713+0000 7fb27d369640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb278156ec0 0x7fb2780c61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:41.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.713+0000 7fb27d369640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb278156ec0 0x7fb2780c61a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:37340/0 (socket says 172.21.15.106:37340) 2024-04-25T10:34:41.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.713+0000 7fb27d369640 1 -- 172.21.15.106:0/1224938929 learned_addr learned my addr 172.21.15.106:0/1224938929 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:34:41.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.714+0000 7fb27db6a640 1 --2- 172.21.15.106:0/1224938929 >> v2:172.21.15.134:3300/0 conn(0x7fb2780cbff0 0x7fb2780cb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:41.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.714+0000 7fb27d369640 1 -- 172.21.15.106:0/1224938929 >> v2:172.21.15.134:3300/0 conn(0x7fb2780cbff0 msgr2=0x7fb2780cb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:41.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.714+0000 7fb27d369640 1 --2- 172.21.15.106:0/1224938929 >> v2:172.21.15.134:3300/0 conn(0x7fb2780cbff0 0x7fb2780cb270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:41.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.714+0000 7fb27d369640 1 -- 172.21.15.106:0/1224938929 >> v1:172.21.15.134:6789/0 conn(0x7fb2780c66e0 legacy=0x7fb2780cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:41.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.714+0000 7fb27d369640 1 -- 172.21.15.106:0/1224938929 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb278160790 con 0x7fb278156ec0 2024-04-25T10:34:41.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.714+0000 7fb27d369640 1 --2- 172.21.15.106:0/1224938929 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb278156ec0 0x7fb2780c61a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb2780749c0 tx=0x7fb26802a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5162df831f23a46 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:41.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.715+0000 7fb27cb68640 1 -- 172.21.15.106:0/1224938929 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb268037070 con 0x7fb278156ec0 2024-04-25T10:34:41.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.715+0000 7fb27cb68640 1 -- 172.21.15.106:0/1224938929 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb268032440 con 0x7fb278156ec0 2024-04-25T10:34:41.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.715+0000 7fb27cb68640 1 -- 172.21.15.106:0/1224938929 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb268032810 con 0x7fb278156ec0 2024-04-25T10:34:41.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.715+0000 7fb27fdce640 1 -- 172.21.15.106:0/1224938929 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb278156ec0 msgr2=0x7fb2780c61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:41.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.715+0000 7fb27fdce640 1 --2- 172.21.15.106:0/1224938929 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb278156ec0 0x7fb2780c61a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb2780749c0 tx=0x7fb26802a290 comp rx=0 tx=0).stop 2024-04-25T10:34:41.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.715+0000 7fb27fdce640 1 -- 172.21.15.106:0/1224938929 shutdown_connections 2024-04-25T10:34:41.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.715+0000 7fb27fdce640 1 --2- 172.21.15.106:0/1224938929 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb278156ec0 0x7fb2780c61a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:41.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.715+0000 7fb27fdce640 1 --2- 172.21.15.106:0/1224938929 >> v2:172.21.15.134:3300/0 conn(0x7fb2780cbff0 0x7fb2780cb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:41.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.715+0000 7fb27fdce640 1 -- 172.21.15.106:0/1224938929 >> 172.21.15.106:0/1224938929 conn(0x7fb2780727d0 msgr2=0x7fb278072c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:41.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.715+0000 7fb27fdce640 1 -- 172.21.15.106:0/1224938929 shutdown_connections 2024-04-25T10:34:41.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.715+0000 7fb27fdce640 1 -- 172.21.15.106:0/1224938929 wait complete. 2024-04-25T10:34:41.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.717+0000 7fb27fdce640 1 Processor -- start 2024-04-25T10:34:41.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.717+0000 7fb27fdce640 1 -- start start 2024-04-25T10:34:41.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.718+0000 7fb27fdce640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb2780c66e0 0x7fb2781552e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:41.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.718+0000 7fb27fdce640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb278156ec0 0x7fb27814d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:41.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.718+0000 7fb27fdce640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb27814b650 0x7fb27814ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:41.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.718+0000 7fb27fdce640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb2781572d0 con 0x7fb27814b650 2024-04-25T10:34:41.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.718+0000 7fb27fdce640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb278160d50 con 0x7fb278156ec0 2024-04-25T10:34:41.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.718+0000 7fb27fdce640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb27815d730 con 0x7fb2780c66e0 2024-04-25T10:34:41.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.718+0000 7fb27e36b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb27814b650 0x7fb27814ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:41.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.718+0000 7fb27e36b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb27814b650 0x7fb27814ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:37350/0 (socket says 172.21.15.106:37350) 2024-04-25T10:34:41.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.718+0000 7fb27e36b640 1 -- 172.21.15.106:0/1014859191 learned_addr learned my addr 172.21.15.106:0/1014859191 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:34:41.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.718+0000 7fb27d369640 1 --2- 172.21.15.106:0/1014859191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb278156ec0 0x7fb27814d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:41.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.718+0000 7fb27db6a640 1 --2- 172.21.15.106:0/1014859191 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb2780c66e0 0x7fb2781552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:41.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.719+0000 7fb27e36b640 1 -- 172.21.15.106:0/1014859191 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb2780c66e0 msgr2=0x7fb2781552e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:41.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.719+0000 7fb27e36b640 1 --2- 172.21.15.106:0/1014859191 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb2780c66e0 0x7fb2781552e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:41.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.719+0000 7fb27e36b640 1 -- 172.21.15.106:0/1014859191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb278156ec0 msgr2=0x7fb27814d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:41.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.719+0000 7fb27e36b640 1 --2- 172.21.15.106:0/1014859191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb278156ec0 0x7fb27814d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:41.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.719+0000 7fb27e36b640 1 -- 172.21.15.106:0/1014859191 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb278155820 con 0x7fb27814b650 2024-04-25T10:34:41.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.719+0000 7fb27e36b640 1 --2- 172.21.15.106:0/1014859191 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb27814b650 0x7fb27814ba60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb27000b9a0 tx=0x7fb27000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:41.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.719+0000 7fb26e7fc640 1 -- 172.21.15.106:0/1014859191 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb270019070 con 0x7fb27814b650 2024-04-25T10:34:41.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.719+0000 7fb26e7fc640 1 -- 172.21.15.106:0/1014859191 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb27000cba0 con 0x7fb27814b650 2024-04-25T10:34:41.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.719+0000 7fb27fdce640 1 -- 172.21.15.106:0/1014859191 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb27814f150 con 0x7fb27814b650 2024-04-25T10:34:41.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.719+0000 7fb26e7fc640 1 -- 172.21.15.106:0/1014859191 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb270014440 con 0x7fb27814b650 2024-04-25T10:34:41.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.719+0000 7fb27fdce640 1 -- 172.21.15.106:0/1014859191 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb278154ae0 con 0x7fb27814b650 2024-04-25T10:34:41.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.720+0000 7fb27fdce640 1 -- 172.21.15.106:0/1014859191 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb27815cf90 con 0x7fb27814b650 2024-04-25T10:34:41.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.721+0000 7fb26e7fc640 1 -- 172.21.15.106:0/1014859191 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb27001c070 con 0x7fb27814b650 2024-04-25T10:34:41.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.721+0000 7fb26e7fc640 1 --2- 172.21.15.106:0/1014859191 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fb25c0744e0 0x7fb25c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:41.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.721+0000 7fb26e7fc640 1 -- 172.21.15.106:0/1014859191 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fb270094e70 con 0x7fb27814b650 2024-04-25T10:34:41.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.721+0000 7fb27db6a640 1 --2- 172.21.15.106:0/1014859191 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fb25c0744e0 0x7fb25c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:41.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.722+0000 7fb27db6a640 1 --2- 172.21.15.106:0/1014859191 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fb25c0744e0 0x7fb25c0769f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb2740079a0 tx=0x7fb274008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:41.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.726+0000 7fb26e7fc640 1 -- 172.21.15.106:0/1014859191 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb27815cf90 con 0x7fb27814b650 2024-04-25T10:34:41.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.900+0000 7fb27fdce640 1 -- 172.21.15.106:0/1014859191 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb27806a3d0 con 0x7fb27814b650 2024-04-25T10:34:41.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.901+0000 7fb26e7fc640 1 -- 172.21.15.106:0/1014859191 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7fb27005efc0 con 0x7fb27814b650 2024-04-25T10:34:41.902 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-25T10:34:41.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.903+0000 7fb27fdce640 1 -- 172.21.15.106:0/1014859191 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fb25c0744e0 msgr2=0x7fb25c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:41.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.903+0000 7fb27fdce640 1 --2- 172.21.15.106:0/1014859191 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fb25c0744e0 0x7fb25c0769f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb2740079a0 tx=0x7fb274008040 comp rx=0 tx=0).stop 2024-04-25T10:34:41.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.903+0000 7fb27fdce640 1 -- 172.21.15.106:0/1014859191 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb27814b650 msgr2=0x7fb27814ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:41.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.903+0000 7fb27fdce640 1 --2- 172.21.15.106:0/1014859191 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb27814b650 0x7fb27814ba60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb27000b9a0 tx=0x7fb27000be60 comp rx=0 tx=0).stop 2024-04-25T10:34:41.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.903+0000 7fb27fdce640 1 -- 172.21.15.106:0/1014859191 shutdown_connections 2024-04-25T10:34:41.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.903+0000 7fb27fdce640 1 --2- 172.21.15.106:0/1014859191 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fb25c0744e0 0x7fb25c0769f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:41.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.903+0000 7fb27fdce640 1 --2- 172.21.15.106:0/1014859191 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb27814b650 0x7fb27814ba60 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:41.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.903+0000 7fb27fdce640 1 --2- 172.21.15.106:0/1014859191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb278156ec0 0x7fb27814d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:41.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.903+0000 7fb27fdce640 1 --2- 172.21.15.106:0/1014859191 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb2780c66e0 0x7fb2781552e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:41.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.903+0000 7fb27fdce640 1 -- 172.21.15.106:0/1014859191 >> 172.21.15.106:0/1014859191 conn(0x7fb2780727d0 msgr2=0x7fb278072be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:41.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.903+0000 7fb27fdce640 1 -- 172.21.15.106:0/1014859191 shutdown_connections 2024-04-25T10:34:41.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:41.903+0000 7fb27fdce640 1 -- 172.21.15.106:0/1014859191 wait complete. 2024-04-25T10:34:41.916 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:34:42.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.022+0000 7f8801c8c640 1 Processor -- start 2024-04-25T10:34:42.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.023+0000 7f8801c8c640 1 -- start start 2024-04-25T10:34:42.026 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.024+0000 7f8801c8c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f87fc072870 0x7f87fc15a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:42.026 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.024+0000 7f8801c8c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f87fc15d9a0 0x7f87fc153710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:42.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.024+0000 7f8801c8c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87fc0c5690 con 0x7f87fc0c5280 2024-04-25T10:34:42.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.024+0000 7f8801c8c640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f87fc0cb590 con 0x7f87fc072870 2024-04-25T10:34:42.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.024+0000 7f8801c8c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f87fc0c7c40 con 0x7f87fc15d9a0 2024-04-25T10:34:42.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.024+0000 7f87fb7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f87fc072870 0x7f87fc15a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:42.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.024+0000 7f87faffd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f87fc0c5280 0x7f87fc0cae80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:45272/0 (socket says 172.21.15.106:45272) 2024-04-25T10:34:42.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.024+0000 7f87faffd640 1 -- 172.21.15.106:0/2208958151 learned_addr learned my addr 172.21.15.106:0/2208958151 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:34:42.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.024+0000 7f87fbfff640 1 --2- 172.21.15.106:0/2208958151 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f87fc15d9a0 0x7f87fc153710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:42.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.024+0000 7f87fa7fc640 1 -- 172.21.15.106:0/2208958151 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1897881430 0 0) 0x7f87f0001110 con 0x7f87fc0c5280 2024-04-25T10:34:42.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.024+0000 7f87fa7fc640 1 -- 172.21.15.106:0/2208958151 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f87e4003520 con 0x7f87fc0c5280 2024-04-25T10:34:42.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.024+0000 7f87fb7fe640 1 -- 172.21.15.106:0/2208958151 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f87fc15d9a0 msgr2=0x7f87fc153710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:42.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.024+0000 7f87fb7fe640 1 --2- 172.21.15.106:0/2208958151 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f87fc15d9a0 0x7f87fc153710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:42.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.024+0000 7f87fb7fe640 1 -- 172.21.15.106:0/2208958151 >> v1:172.21.15.106:6789/0 conn(0x7f87fc0c5280 legacy=0x7f87fc0cae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:42.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.024+0000 7f87fb7fe640 1 -- 172.21.15.106:0/2208958151 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87fc0c7f20 con 0x7f87fc072870 2024-04-25T10:34:42.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.025+0000 7f87fb7fe640 1 --2- 172.21.15.106:0/2208958151 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f87fc072870 0x7f87fc15a590 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f87ec0097a0 tx=0x7f87ec02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=436f82c80e297195 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:42.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.025+0000 7f87fa7fc640 1 -- 172.21.15.106:0/2208958151 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ec037070 con 0x7f87fc072870 2024-04-25T10:34:42.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.025+0000 7f87fa7fc640 1 -- 172.21.15.106:0/2208958151 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87ec02ae50 con 0x7f87fc072870 2024-04-25T10:34:42.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.025+0000 7f87fa7fc640 1 -- 172.21.15.106:0/2208958151 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ec032660 con 0x7f87fc072870 2024-04-25T10:34:42.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.026+0000 7f8801c8c640 1 -- 172.21.15.106:0/2208958151 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f87fc072870 msgr2=0x7f87fc15a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:42.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.026+0000 7f8801c8c640 1 --2- 172.21.15.106:0/2208958151 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f87fc072870 0x7f87fc15a590 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f87ec0097a0 tx=0x7f87ec02a100 comp rx=0 tx=0).stop 2024-04-25T10:34:42.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.026+0000 7f8801c8c640 1 -- 172.21.15.106:0/2208958151 shutdown_connections 2024-04-25T10:34:42.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.026+0000 7f8801c8c640 1 --2- 172.21.15.106:0/2208958151 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f87fc15d9a0 0x7f87fc153710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:42.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.026+0000 7f8801c8c640 1 --2- 172.21.15.106:0/2208958151 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f87fc072870 0x7f87fc15a590 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:42.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.026+0000 7f8801c8c640 1 -- 172.21.15.106:0/2208958151 >> 172.21.15.106:0/2208958151 conn(0x7f87fc06c9b0 msgr2=0x7f87fc06cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:42.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.026+0000 7f8801c8c640 1 -- 172.21.15.106:0/2208958151 shutdown_connections 2024-04-25T10:34:42.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.026+0000 7f8801c8c640 1 -- 172.21.15.106:0/2208958151 wait complete. 2024-04-25T10:34:42.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.028+0000 7f8801c8c640 1 Processor -- start 2024-04-25T10:34:42.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.028+0000 7f8801c8c640 1 -- start start 2024-04-25T10:34:42.032 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.029+0000 7f8801c8c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87fc072870 0x7f87fc14a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:42.032 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.029+0000 7f8801c8c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f87fc0c5280 0x7f87fc1f6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:42.032 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.029+0000 7f8801c8c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f87fc15d9a0 0x7f87fc158610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:42.032 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.029+0000 7f8801c8c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f87fc159d90 con 0x7f87fc0c5280 2024-04-25T10:34:42.032 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.029+0000 7f8801c8c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f87fc159ab0 con 0x7f87fc072870 2024-04-25T10:34:42.032 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.029+0000 7f8801c8c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f87fc15a070 con 0x7f87fc15d9a0 2024-04-25T10:34:42.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.029+0000 7f87faffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f87fc0c5280 0x7f87fc1f6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:42.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.029+0000 7f87faffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f87fc0c5280 0x7f87fc1f6e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:37362/0 (socket says 172.21.15.106:37362) 2024-04-25T10:34:42.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.029+0000 7f87faffd640 1 -- 172.21.15.106:0/1864161678 learned_addr learned my addr 172.21.15.106:0/1864161678 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:34:42.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.029+0000 7f87fb7fe640 1 --2- 172.21.15.106:0/1864161678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87fc072870 0x7f87fc14a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:42.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.029+0000 7f87fbfff640 1 --2- 172.21.15.106:0/1864161678 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f87fc15d9a0 0x7f87fc158610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:42.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.029+0000 7f87faffd640 1 -- 172.21.15.106:0/1864161678 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f87fc15d9a0 msgr2=0x7f87fc158610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:42.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.029+0000 7f87faffd640 1 --2- 172.21.15.106:0/1864161678 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f87fc15d9a0 0x7f87fc158610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:42.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.029+0000 7f87faffd640 1 -- 172.21.15.106:0/1864161678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87fc072870 msgr2=0x7f87fc14a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:42.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.029+0000 7f87faffd640 1 --2- 172.21.15.106:0/1864161678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87fc072870 0x7f87fc14a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:42.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.030+0000 7f87faffd640 1 -- 172.21.15.106:0/1864161678 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87fc15c510 con 0x7f87fc0c5280 2024-04-25T10:34:42.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.030+0000 7f87faffd640 1 --2- 172.21.15.106:0/1864161678 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f87fc0c5280 0x7f87fc1f6e70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f87f000c370 tx=0x7f87f000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:42.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.030+0000 7f87e3fff640 1 -- 172.21.15.106:0/1864161678 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f0017070 con 0x7f87fc0c5280 2024-04-25T10:34:42.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.030+0000 7f87e3fff640 1 -- 172.21.15.106:0/1864161678 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87f0005ce0 con 0x7f87fc0c5280 2024-04-25T10:34:42.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.030+0000 7f8801c8c640 1 -- 172.21.15.106:0/1864161678 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87fc15c7b0 con 0x7f87fc0c5280 2024-04-25T10:34:42.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.030+0000 7f87e3fff640 1 -- 172.21.15.106:0/1864161678 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f00072f0 con 0x7f87fc0c5280 2024-04-25T10:34:42.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.030+0000 7f8801c8c640 1 -- 172.21.15.106:0/1864161678 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87fc15ca70 con 0x7f87fc0c5280 2024-04-25T10:34:42.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.031+0000 7f8801c8c640 1 -- 172.21.15.106:0/1864161678 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87fc14a540 con 0x7f87fc0c5280 2024-04-25T10:34:42.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.031+0000 7f87e3fff640 1 -- 172.21.15.106:0/1864161678 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f87f001a070 con 0x7f87fc0c5280 2024-04-25T10:34:42.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.032+0000 7f87e3fff640 1 --2- 172.21.15.106:0/1864161678 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f87e80744b0 0x7f87e80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:42.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.032+0000 7f87fb7fe640 1 --2- 172.21.15.106:0/1864161678 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f87e80744b0 0x7f87e80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:42.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.032+0000 7f87e3fff640 1 -- 172.21.15.106:0/1864161678 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f87f0094560 con 0x7f87fc0c5280 2024-04-25T10:34:42.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.032+0000 7f87fb7fe640 1 --2- 172.21.15.106:0/1864161678 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f87e80744b0 0x7f87e80769c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f87ec034000 tx=0x7f87ec02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:42.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.037+0000 7f87e3fff640 1 -- 172.21.15.106:0/1864161678 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f87fc0c5690 con 0x7f87fc0c5280 2024-04-25T10:34:42.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.228+0000 7f8801c8c640 1 -- 172.21.15.106:0/1864161678 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f87fc06a3d0 con 0x7f87fc0c5280 2024-04-25T10:34:42.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.228+0000 7f87e3fff640 1 -- 172.21.15.106:0/1864161678 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f87f005e6b0 con 0x7f87fc0c5280 2024-04-25T10:34:42.230 INFO:teuthology.orchestra.run.smithi106.stderr:listed 0 entries 2024-04-25T10:34:42.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.230+0000 7f8801c8c640 1 -- 172.21.15.106:0/1864161678 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f87e80744b0 msgr2=0x7f87e80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:42.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.230+0000 7f8801c8c640 1 --2- 172.21.15.106:0/1864161678 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f87e80744b0 0x7f87e80769c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f87ec034000 tx=0x7f87ec02e000 comp rx=0 tx=0).stop 2024-04-25T10:34:42.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.230+0000 7f8801c8c640 1 -- 172.21.15.106:0/1864161678 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f87fc0c5280 msgr2=0x7f87fc1f6e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:42.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.230+0000 7f8801c8c640 1 --2- 172.21.15.106:0/1864161678 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f87fc0c5280 0x7f87fc1f6e70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f87f000c370 tx=0x7f87f000c830 comp rx=0 tx=0).stop 2024-04-25T10:34:42.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.230+0000 7f8801c8c640 1 -- 172.21.15.106:0/1864161678 shutdown_connections 2024-04-25T10:34:42.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.230+0000 7f8801c8c640 1 --2- 172.21.15.106:0/1864161678 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f87e80744b0 0x7f87e80769c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:42.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.230+0000 7f8801c8c640 1 --2- 172.21.15.106:0/1864161678 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f87fc15d9a0 0x7f87fc158610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:42.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.230+0000 7f8801c8c640 1 --2- 172.21.15.106:0/1864161678 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f87fc0c5280 0x7f87fc1f6e70 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:42.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.230+0000 7f8801c8c640 1 --2- 172.21.15.106:0/1864161678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87fc072870 0x7f87fc14a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:42.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.230+0000 7f8801c8c640 1 -- 172.21.15.106:0/1864161678 >> 172.21.15.106:0/1864161678 conn(0x7f87fc06c9b0 msgr2=0x7f87fc15cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:42.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.230+0000 7f8801c8c640 1 -- 172.21.15.106:0/1864161678 shutdown_connections 2024-04-25T10:34:42.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.230+0000 7f8801c8c640 1 -- 172.21.15.106:0/1864161678 wait complete. 2024-04-25T10:34:42.243 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T10:34:42.244 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:34:42.259 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:34:42.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.406+0000 7f4767e13640 1 Processor -- start 2024-04-25T10:34:42.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.406+0000 7f4767e13640 1 -- start start 2024-04-25T10:34:42.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.408+0000 7f4767e13640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47600cb270 0x7f4760156ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:42.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.408+0000 7f4767e13640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f476015dae0 con 0x7f47600cbff0 2024-04-25T10:34:42.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.408+0000 7f4767e13640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47601604b0 con 0x7f47600cce00 2024-04-25T10:34:42.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.408+0000 7f4767e13640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f4760161270 con 0x7f47600cb270 2024-04-25T10:34:42.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.408+0000 7f4765baf640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f47600cbff0 0x7f47600c61a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:45282/0 (socket says 172.21.15.106:45282) 2024-04-25T10:34:42.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.408+0000 7f4765baf640 1 -- 172.21.15.106:0/670565039 learned_addr learned my addr 172.21.15.106:0/670565039 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:34:42.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.408+0000 7f47653ae640 1 --2- 172.21.15.106:0/670565039 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47600cb270 0x7f4760156ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:42.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.408+0000 7f4764bad640 1 -- 172.21.15.106:0/670565039 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2972506587 0 0) 0x7f476015dae0 con 0x7f47600cbff0 2024-04-25T10:34:42.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.408+0000 7f4764bad640 1 -- 172.21.15.106:0/670565039 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f474c003520 con 0x7f47600cbff0 2024-04-25T10:34:42.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.408+0000 7f4764bad640 1 -- 172.21.15.106:0/670565039 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3332006513 0 0) 0x7f47601604b0 con 0x7f47600cce00 2024-04-25T10:34:42.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.408+0000 7f4764bad640 1 -- 172.21.15.106:0/670565039 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f476015dae0 con 0x7f47600cce00 2024-04-25T10:34:42.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.408+0000 7f4764bad640 1 -- 172.21.15.106:0/670565039 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 552690115 0 0) 0x7f474c003520 con 0x7f47600cbff0 2024-04-25T10:34:42.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.408+0000 7f4764bad640 1 -- 172.21.15.106:0/670565039 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47600cb270 msgr2=0x7f4760156ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:42.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.408+0000 7f4764bad640 1 --2- 172.21.15.106:0/670565039 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47600cb270 0x7f4760156ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:42.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.408+0000 7f4764bad640 1 -- 172.21.15.106:0/670565039 >> v1:172.21.15.134:6789/0 conn(0x7f47600cce00 legacy=0x7f4760167630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:42.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.408+0000 7f4764bad640 1 -- 172.21.15.106:0/670565039 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4760160790 con 0x7f47600cbff0 2024-04-25T10:34:42.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.408+0000 7f4764bad640 1 -- 172.21.15.106:0/670565039 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3776459350 0 0) 0x7f475c001a10 con 0x7f47600cbff0 2024-04-25T10:34:42.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.408+0000 7f4764bad640 1 -- 172.21.15.106:0/670565039 >> v1:172.21.15.106:6789/0 conn(0x7f47600cbff0 legacy=0x7f47600c61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:42.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.408+0000 7f4764bad640 1 --2- 172.21.15.106:0/670565039 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f474c003b70 0x7f474c003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:42.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.408+0000 7f4764bad640 1 --2- 172.21.15.106:0/670565039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f474c004800 0x7f474c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:42.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.408+0000 7f4764bad640 1 --2- 172.21.15.106:0/670565039 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47600cb270 0x7f474c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:42.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.408+0000 7f4764bad640 1 -- 172.21.15.106:0/670565039 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4760161270 con 0x7f474c003b70 2024-04-25T10:34:42.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.408+0000 7f4764bad640 1 -- 172.21.15.106:0/670565039 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f474c00bb00 con 0x7f474c004800 2024-04-25T10:34:42.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.408+0000 7f4764bad640 1 -- 172.21.15.106:0/670565039 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f474c00bd00 con 0x7f47600cb270 2024-04-25T10:34:42.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.408+0000 7f4765baf640 1 --2- 172.21.15.106:0/670565039 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f474c003b70 0x7f474c003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:42.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.409+0000 7f47653ae640 1 --2- 172.21.15.106:0/670565039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f474c004800 0x7f474c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:42.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.409+0000 7f47663b0640 1 --2- 172.21.15.106:0/670565039 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47600cb270 0x7f474c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:42.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.409+0000 7f4765baf640 1 -- 172.21.15.106:0/670565039 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47600cb270 msgr2=0x7f474c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:42.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.409+0000 7f4765baf640 1 --2- 172.21.15.106:0/670565039 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47600cb270 0x7f474c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:42.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.409+0000 7f4765baf640 1 -- 172.21.15.106:0/670565039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f474c004800 msgr2=0x7f474c004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:42.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.409+0000 7f4765baf640 1 --2- 172.21.15.106:0/670565039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f474c004800 0x7f474c004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:42.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.409+0000 7f4765baf640 1 -- 172.21.15.106:0/670565039 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f474c003520 con 0x7f474c003b70 2024-04-25T10:34:42.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.409+0000 7f47663b0640 1 --2- 172.21.15.106:0/670565039 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47600cb270 0x7f474c005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:34:42.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.409+0000 7f4765baf640 1 --2- 172.21.15.106:0/670565039 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f474c003b70 0x7f474c003fc0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f475c00a800 tx=0x7f474c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=401c17b0e842451 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:42.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.410+0000 7f4764bad640 1 -- 172.21.15.106:0/670565039 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475c00a450 con 0x7f474c003b70 2024-04-25T10:34:42.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.410+0000 7f4764bad640 1 -- 172.21.15.106:0/670565039 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f475c02f070 con 0x7f474c003b70 2024-04-25T10:34:42.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.410+0000 7f4764bad640 1 -- 172.21.15.106:0/670565039 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475c02bc30 con 0x7f474c003b70 2024-04-25T10:34:42.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.410+0000 7f4767e13640 1 -- 172.21.15.106:0/670565039 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f474c003b70 msgr2=0x7f474c003fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:42.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.410+0000 7f4767e13640 1 --2- 172.21.15.106:0/670565039 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f474c003b70 0x7f474c003fc0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f475c00a800 tx=0x7f474c005830 comp rx=0 tx=0).stop 2024-04-25T10:34:42.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.410+0000 7f47663b0640 1 -- 172.21.15.106:0/670565039 reap_dead start 2024-04-25T10:34:42.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.410+0000 7f4767e13640 1 -- 172.21.15.106:0/670565039 shutdown_connections 2024-04-25T10:34:42.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.410+0000 7f4767e13640 1 -- 172.21.15.106:0/670565039 >> 172.21.15.106:0/670565039 conn(0x7f47600727d0 msgr2=0x7f4760072c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:42.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.410+0000 7f4767e13640 1 -- 172.21.15.106:0/670565039 shutdown_connections 2024-04-25T10:34:42.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.410+0000 7f4767e13640 1 -- 172.21.15.106:0/670565039 wait complete. 2024-04-25T10:34:42.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.412+0000 7f4767e13640 1 Processor -- start 2024-04-25T10:34:42.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.412+0000 7f4767e13640 1 -- start start 2024-04-25T10:34:42.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.413+0000 7f4767e13640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f474c003b70 0x7f476015ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:42.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.413+0000 7f4767e13640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f47600d0b10 0x7f476015f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:42.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.413+0000 7f4767e13640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47600d05c0 0x7f476014da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:42.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.413+0000 7f4767e13640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f476015d730 con 0x7f47600d0b10 2024-04-25T10:34:42.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.413+0000 7f4767e13640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4760160d50 con 0x7f47600d05c0 2024-04-25T10:34:42.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.413+0000 7f4767e13640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f476015b690 con 0x7f474c003b70 2024-04-25T10:34:42.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.413+0000 7f47653ae640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f47600d0b10 0x7f476015f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:42.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.413+0000 7f47653ae640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f47600d0b10 0x7f476015f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:37392/0 (socket says 172.21.15.106:37392) 2024-04-25T10:34:42.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.413+0000 7f47653ae640 1 -- 172.21.15.106:0/3892718035 learned_addr learned my addr 172.21.15.106:0/3892718035 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:34:42.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.414+0000 7f4765baf640 1 --2- 172.21.15.106:0/3892718035 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f474c003b70 0x7f476015ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:42.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.414+0000 7f47663b0640 1 --2- 172.21.15.106:0/3892718035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47600d05c0 0x7f476014da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:42.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.414+0000 7f47653ae640 1 -- 172.21.15.106:0/3892718035 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f474c003b70 msgr2=0x7f476015ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:42.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.414+0000 7f47653ae640 1 --2- 172.21.15.106:0/3892718035 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f474c003b70 0x7f476015ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:42.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.414+0000 7f47653ae640 1 -- 172.21.15.106:0/3892718035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47600d05c0 msgr2=0x7f476014da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:42.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.414+0000 7f47653ae640 1 --2- 172.21.15.106:0/3892718035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47600d05c0 0x7f476014da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:42.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.414+0000 7f47653ae640 1 -- 172.21.15.106:0/3892718035 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f476015b1c0 con 0x7f47600d0b10 2024-04-25T10:34:42.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.414+0000 7f47653ae640 1 --2- 172.21.15.106:0/3892718035 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f47600d0b10 0x7f476015f2b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f47500119a0 tx=0x7f4750011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:42.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.414+0000 7f47567fc640 1 -- 172.21.15.106:0/3892718035 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475000faf0 con 0x7f47600d0b10 2024-04-25T10:34:42.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.414+0000 7f47567fc640 1 -- 172.21.15.106:0/3892718035 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f475000fc90 con 0x7f47600d0b10 2024-04-25T10:34:42.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.414+0000 7f4767e13640 1 -- 172.21.15.106:0/3892718035 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47600ce3c0 con 0x7f47600d0b10 2024-04-25T10:34:42.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.414+0000 7f47567fc640 1 -- 172.21.15.106:0/3892718035 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4750014440 con 0x7f47600d0b10 2024-04-25T10:34:42.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.414+0000 7f4767e13640 1 -- 172.21.15.106:0/3892718035 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47601f2cd0 con 0x7f47600d0b10 2024-04-25T10:34:42.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.415+0000 7f4767e13640 1 -- 172.21.15.106:0/3892718035 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f476006a3d0 con 0x7f47600d0b10 2024-04-25T10:34:42.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.415+0000 7f47567fc640 1 -- 172.21.15.106:0/3892718035 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4750012040 con 0x7f47600d0b10 2024-04-25T10:34:42.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.416+0000 7f47567fc640 1 --2- 172.21.15.106:0/3892718035 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f47440744b0 0x7f47440769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:42.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.416+0000 7f4765baf640 1 --2- 172.21.15.106:0/3892718035 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f47440744b0 0x7f47440769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:42.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.416+0000 7f47567fc640 1 -- 172.21.15.106:0/3892718035 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f47500958e0 con 0x7f47600d0b10 2024-04-25T10:34:42.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.417+0000 7f4765baf640 1 --2- 172.21.15.106:0/3892718035 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f47440744b0 0x7f47440769c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f475c033000 tx=0x7f475c02d000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:42.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.421+0000 7f47567fc640 1 -- 172.21.15.106:0/3892718035 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4750019030 con 0x7f47600d0b10 2024-04-25T10:34:42.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.599+0000 7f4767e13640 1 -- 172.21.15.106:0/3892718035 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f476015ce50 con 0x7f47600d0b10 2024-04-25T10:34:42.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.600+0000 7f47567fc640 1 -- 172.21.15.106:0/3892718035 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16890 (secure 0 0 0) 0x7f47500600f0 con 0x7f47600d0b10 2024-04-25T10:34:42.601 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:34:42.602 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":22,"fsid":"3ee77985-e8e4-41cf-81e9-7c774cf7891f","created":"2024-04-25T10:30:07.981054+0000","modified":"2024-04-25T10:31:13.621853+0000","last_up_change":"2024-04-25T10:30:19.184325+0000","last_in_change":"2024-04-25T10:30:10.217553+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:30:20.111046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:30:30.314914+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:30:30.715500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a5962cb1-dbbd-47b1-8d15-9bae114bfdf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1240717079},{"type":"v1","addr":"172.21.15.106:6801","nonce":1240717079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1240717079},{"type":"v1","addr":"172.21.15.106:6803","nonce":1240717079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1240717079},{"type":"v1","addr":"172.21.15.106:6807","nonce":1240717079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1240717079},{"type":"v1","addr":"172.21.15.106:6805","nonce":1240717079}]},"public_addr":"172.21.15.106:6801/1240717079","cluster_addr":"172.21.15.106:6803/1240717079","heartbeat_back_addr":"172.21.15.106:6807/1240717079","heartbeat_front_addr":"172.21.15.106:6805/1240717079","state":["exists","up"]},{"osd":1,"uuid":"94436e13-4769-4752-8397-b5e396a6f7ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4089653051},{"type":"v1","addr":"172.21.15.106:6825","nonce":4089653051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4089653051},{"type":"v1","addr":"172.21.15.106:6827","nonce":4089653051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4089653051},{"type":"v1","addr":"172.21.15.106:6831","nonce":4089653051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4089653051},{"type":"v1","addr":"172.21.15.106:6829","nonce":4089653051}]},"public_addr":"172.21.15.106:6825/4089653051","cluster_addr":"172.21.15.106:6827/4089653051","heartbeat_back_addr":"172.21.15.106:6831/4089653051","heartbeat_front_addr":"172.21.15.106:6829/4089653051","state":["exists","up"]},{"osd":2,"uuid":"429aae0b-70c6-4e7b-9043-6a11334c7efc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":1441018239},{"type":"v1","addr":"172.21.15.106:6809","nonce":1441018239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1441018239},{"type":"v1","addr":"172.21.15.106:6811","nonce":1441018239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1441018239},{"type":"v1","addr":"172.21.15.106:6815","nonce":1441018239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":1441018239},{"type":"v1","addr":"172.21.15.106:6813","nonce":1441018239}]},"public_addr":"172.21.15.106:6809/1441018239","cluster_addr":"172.21.15.106:6811/1441018239","heartbeat_back_addr":"172.21.15.106:6815/1441018239","heartbeat_front_addr":"172.21.15.106:6813/1441018239","state":["exists","up"]},{"osd":3,"uuid":"9593a4d1-772f-4667-ab0d-2982f841e342","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":1846968711},{"type":"v1","addr":"172.21.15.106:6817","nonce":1846968711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1846968711},{"type":"v1","addr":"172.21.15.106:6819","nonce":1846968711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":1846968711},{"type":"v1","addr":"172.21.15.106:6823","nonce":1846968711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":1846968711},{"type":"v1","addr":"172.21.15.106:6821","nonce":1846968711}]},"public_addr":"172.21.15.106:6817/1846968711","cluster_addr":"172.21.15.106:6819/1846968711","heartbeat_back_addr":"172.21.15.106:6823/1846968711","heartbeat_front_addr":"172.21.15.106:6821/1846968711","state":["exists","up"]},{"osd":4,"uuid":"1b61f16d-5c60-480a-b449-c745aa3f6b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":729224739},{"type":"v1","addr":"172.21.15.134:6825","nonce":729224739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":729224739},{"type":"v1","addr":"172.21.15.134:6827","nonce":729224739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":729224739},{"type":"v1","addr":"172.21.15.134:6831","nonce":729224739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":729224739},{"type":"v1","addr":"172.21.15.134:6829","nonce":729224739}]},"public_addr":"172.21.15.134:6825/729224739","cluster_addr":"172.21.15.134:6827/729224739","heartbeat_back_addr":"172.21.15.134:6831/729224739","heartbeat_front_addr":"172.21.15.134:6829/729224739","state":["exists","up"]},{"osd":5,"uuid":"944cdd7f-47c0-4b71-b7e0-d093542f4528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3132275652},{"type":"v1","addr":"172.21.15.134:6809","nonce":3132275652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3132275652},{"type":"v1","addr":"172.21.15.134:6811","nonce":3132275652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3132275652},{"type":"v1","addr":"172.21.15.134:6815","nonce":3132275652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3132275652},{"type":"v1","addr":"172.21.15.134:6813","nonce":3132275652}]},"public_addr":"172.21.15.134:6809/3132275652","cluster_addr":"172.21.15.134:6811/3132275652","heartbeat_back_addr":"172.21.15.134:6815/3132275652","heartbeat_front_addr":"172.21.15.134:6813/3132275652","state":["exists","up"]},{"osd":6,"uuid":"cc2446e2-1d21-4135-ac04-6af7c990058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":992702668},{"type":"v1","addr":"172.21.15.134:6801","nonce":992702668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":992702668},{"type":"v1","addr":"172.21.15.134:6803","nonce":992702668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":992702668},{"type":"v1","addr":"172.21.15.134:6807","nonce":992702668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":992702668},{"type":"v1","addr":"172.21.15.134:6805","nonce":992702668}]},"public_addr":"172.21.15.134:6801/992702668","cluster_addr":"172.21.15.134:6803/992702668","heartbeat_back_addr":"172.21.15.134:6807/992702668","heartbeat_front_addr":"172.21.15.134:6805/992702668","state":["exists","up"]},{"osd":7,"uuid":"3df1fc7a-82fc-4257-8629-98027b6e2aed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":3291062284},{"type":"v1","addr":"172.21.15.134:6817","nonce":3291062284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":3291062284},{"type":"v1","addr":"172.21.15.134:6819","nonce":3291062284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":3291062284},{"type":"v1","addr":"172.21.15.134:6823","nonce":3291062284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":3291062284},{"type":"v1","addr":"172.21.15.134:6821","nonce":3291062284}]},"public_addr":"172.21.15.134:6817/3291062284","cluster_addr":"172.21.15.134:6819/3291062284","heartbeat_back_addr":"172.21.15.134:6823/3291062284","heartbeat_front_addr":"172.21.15.134:6821/3291062284","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.403941+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.382563+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.411544+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.330444+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.405430+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.387737+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.408922+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.454450+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":2}]},{"pgid":"3.2b","mappings":[{"from":0,"to":1}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:34:42.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.602+0000 7f4767e13640 1 -- 172.21.15.106:0/3892718035 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f47440744b0 msgr2=0x7f47440769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:42.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.602+0000 7f4767e13640 1 --2- 172.21.15.106:0/3892718035 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f47440744b0 0x7f47440769c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f475c033000 tx=0x7f475c02d000 comp rx=0 tx=0).stop 2024-04-25T10:34:42.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.602+0000 7f4767e13640 1 -- 172.21.15.106:0/3892718035 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f47600d0b10 msgr2=0x7f476015f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:42.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.602+0000 7f4767e13640 1 --2- 172.21.15.106:0/3892718035 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f47600d0b10 0x7f476015f2b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f47500119a0 tx=0x7f4750011e60 comp rx=0 tx=0).stop 2024-04-25T10:34:42.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.602+0000 7f4767e13640 1 -- 172.21.15.106:0/3892718035 shutdown_connections 2024-04-25T10:34:42.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.602+0000 7f4767e13640 1 --2- 172.21.15.106:0/3892718035 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f47440744b0 0x7f47440769c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:42.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.602+0000 7f4767e13640 1 --2- 172.21.15.106:0/3892718035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47600d05c0 0x7f476014da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:42.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.602+0000 7f4767e13640 1 --2- 172.21.15.106:0/3892718035 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f47600d0b10 0x7f476015f2b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:42.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.602+0000 7f4767e13640 1 --2- 172.21.15.106:0/3892718035 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f474c003b70 0x7f476015ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:42.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.602+0000 7f4767e13640 1 -- 172.21.15.106:0/3892718035 >> 172.21.15.106:0/3892718035 conn(0x7f47600727d0 msgr2=0x7f4760072c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:42.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.602+0000 7f4767e13640 1 -- 172.21.15.106:0/3892718035 shutdown_connections 2024-04-25T10:34:42.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.602+0000 7f4767e13640 1 -- 172.21.15.106:0/3892718035 wait complete. 2024-04-25T10:34:42.615 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:30:20.111046+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:30:30.314914+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:30:30.715500+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:34:42.616 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:34:42.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.724+0000 7f6d42a15640 1 Processor -- start 2024-04-25T10:34:42.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.724+0000 7f6d42a15640 1 -- start start 2024-04-25T10:34:42.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.725+0000 7f6d42a15640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6d3c06dc00 0x7f6d3c15d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:42.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.725+0000 7f6d42a15640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f6d3c0c61a0 0x7f6d3c156ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:42.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.725+0000 7f6d42a15640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6d3c1601c0 0x7f6d3c0ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:42.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.725+0000 7f6d42a15640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f6d3c0c65b0 con 0x7f6d3c1601c0 2024-04-25T10:34:42.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.725+0000 7f6d42a15640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f6d3c160ae0 con 0x7f6d3c0c61a0 2024-04-25T10:34:42.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.725+0000 7f6d42a15640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6d3c160dc0 con 0x7f6d3c06dc00 2024-04-25T10:34:42.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.726+0000 7f6d40fb2640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6d3c1601c0 0x7f6d3c0ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:42.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.726+0000 7f6d40fb2640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6d3c1601c0 0x7f6d3c0ccde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:37398/0 (socket says 172.21.15.106:37398) 2024-04-25T10:34:42.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.726+0000 7f6d40fb2640 1 -- 172.21.15.106:0/1033098247 learned_addr learned my addr 172.21.15.106:0/1033098247 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:34:42.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.726+0000 7f6d3bfff640 1 --2- 172.21.15.106:0/1033098247 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6d3c06dc00 0x7f6d3c15d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:42.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.726+0000 7f6d3b7fe640 1 --2- 172.21.15.106:0/1033098247 >> v2:172.21.15.134:3300/0 conn(0x7f6d3c0c61a0 0x7f6d3c156ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:42.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.726+0000 7f6d40fb2640 1 -- 172.21.15.106:0/1033098247 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6d3c06dc00 msgr2=0x7f6d3c15d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:42.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.726+0000 7f6d40fb2640 1 --2- 172.21.15.106:0/1033098247 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6d3c06dc00 0x7f6d3c15d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:42.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.726+0000 7f6d40fb2640 1 -- 172.21.15.106:0/1033098247 >> v2:172.21.15.134:3300/0 conn(0x7f6d3c0c61a0 msgr2=0x7f6d3c156ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:42.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.726+0000 7f6d40fb2640 1 --2- 172.21.15.106:0/1033098247 >> v2:172.21.15.134:3300/0 conn(0x7f6d3c0c61a0 0x7f6d3c156ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:42.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.726+0000 7f6d40fb2640 1 -- 172.21.15.106:0/1033098247 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d3c1610a0 con 0x7f6d3c1601c0 2024-04-25T10:34:42.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.726+0000 7f6d3bfff640 1 --2- 172.21.15.106:0/1033098247 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6d3c06dc00 0x7f6d3c15d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:34:42.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.726+0000 7f6d3b7fe640 1 --2- 172.21.15.106:0/1033098247 >> v2:172.21.15.134:3300/0 conn(0x7f6d3c0c61a0 0x7f6d3c156ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:34:42.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.727+0000 7f6d40fb2640 1 --2- 172.21.15.106:0/1033098247 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d3c1601c0 0x7f6d3c0ccde0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c0cd930 tx=0x7f6d3402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79a97bc2655412c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:42.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.727+0000 7f6d3affd640 1 -- 172.21.15.106:0/1033098247 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d34036070 con 0x7f6d3c1601c0 2024-04-25T10:34:42.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.727+0000 7f6d3affd640 1 -- 172.21.15.106:0/1033098247 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d34031600 con 0x7f6d3c1601c0 2024-04-25T10:34:42.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.727+0000 7f6d3affd640 1 -- 172.21.15.106:0/1033098247 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d340319d0 con 0x7f6d3c1601c0 2024-04-25T10:34:42.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.727+0000 7f6d42a15640 1 -- 172.21.15.106:0/1033098247 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d3c1601c0 msgr2=0x7f6d3c0ccde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:42.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.727+0000 7f6d42a15640 1 --2- 172.21.15.106:0/1033098247 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d3c1601c0 0x7f6d3c0ccde0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c0cd930 tx=0x7f6d3402a6a0 comp rx=0 tx=0).stop 2024-04-25T10:34:42.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.728+0000 7f6d42a15640 1 -- 172.21.15.106:0/1033098247 shutdown_connections 2024-04-25T10:34:42.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.728+0000 7f6d42a15640 1 --2- 172.21.15.106:0/1033098247 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d3c1601c0 0x7f6d3c0ccde0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:42.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.728+0000 7f6d42a15640 1 --2- 172.21.15.106:0/1033098247 >> v2:172.21.15.134:3300/0 conn(0x7f6d3c0c61a0 0x7f6d3c156ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:42.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.728+0000 7f6d42a15640 1 --2- 172.21.15.106:0/1033098247 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6d3c06dc00 0x7f6d3c15d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:42.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.728+0000 7f6d42a15640 1 -- 172.21.15.106:0/1033098247 >> 172.21.15.106:0/1033098247 conn(0x7f6d3c0727d0 msgr2=0x7f6d3c072c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:42.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.728+0000 7f6d42a15640 1 -- 172.21.15.106:0/1033098247 shutdown_connections 2024-04-25T10:34:42.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.728+0000 7f6d42a15640 1 -- 172.21.15.106:0/1033098247 wait complete. 2024-04-25T10:34:42.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.729+0000 7f6d42a15640 1 Processor -- start 2024-04-25T10:34:42.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.729+0000 7f6d42a15640 1 -- start start 2024-04-25T10:34:42.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.731+0000 7f6d42a15640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d3c06dc00 0x7f6d3c151aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:42.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.731+0000 7f6d42a15640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6d3c0c61a0 0x7f6d3c14fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:42.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.731+0000 7f6d42a15640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d3c1601c0 0x7f6d3c14d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:42.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.731+0000 7f6d42a15640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6d3c15d2a0 con 0x7f6d3c06dc00 2024-04-25T10:34:42.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.731+0000 7f6d42a15640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6d3c15cfc0 con 0x7f6d3c1601c0 2024-04-25T10:34:42.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.731+0000 7f6d42a15640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6d3c06d740 con 0x7f6d3c0c61a0 2024-04-25T10:34:42.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.731+0000 7f6d3bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d3c06dc00 0x7f6d3c151aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:42.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.731+0000 7f6d3bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d3c06dc00 0x7f6d3c151aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:37406/0 (socket says 172.21.15.106:37406) 2024-04-25T10:34:42.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.731+0000 7f6d3bfff640 1 -- 172.21.15.106:0/60030113 learned_addr learned my addr 172.21.15.106:0/60030113 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:34:42.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.731+0000 7f6d40fb2640 1 --2- 172.21.15.106:0/60030113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d3c1601c0 0x7f6d3c14d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:42.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.731+0000 7f6d3b7fe640 1 --2- 172.21.15.106:0/60030113 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6d3c0c61a0 0x7f6d3c14fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:42.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.731+0000 7f6d3bfff640 1 -- 172.21.15.106:0/60030113 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6d3c0c61a0 msgr2=0x7f6d3c14fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:42.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.731+0000 7f6d3bfff640 1 --2- 172.21.15.106:0/60030113 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6d3c0c61a0 0x7f6d3c14fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:42.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.731+0000 7f6d3bfff640 1 -- 172.21.15.106:0/60030113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d3c1601c0 msgr2=0x7f6d3c14d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:42.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.731+0000 7f6d3bfff640 1 --2- 172.21.15.106:0/60030113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d3c1601c0 0x7f6d3c14d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:42.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.731+0000 7f6d3bfff640 1 -- 172.21.15.106:0/60030113 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d3c150010 con 0x7f6d3c06dc00 2024-04-25T10:34:42.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.731+0000 7f6d40fb2640 1 --2- 172.21.15.106:0/60030113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d3c1601c0 0x7f6d3c14d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:34:42.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.731+0000 7f6d3bfff640 1 --2- 172.21.15.106:0/60030113 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d3c06dc00 0x7f6d3c151aa0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c00d7d0 tx=0x7f6d2c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:42.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.731+0000 7f6d38ff9640 1 -- 172.21.15.106:0/60030113 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d2c004030 con 0x7f6d3c06dc00 2024-04-25T10:34:42.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.731+0000 7f6d38ff9640 1 -- 172.21.15.106:0/60030113 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d2c0041d0 con 0x7f6d3c06dc00 2024-04-25T10:34:42.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.731+0000 7f6d42a15640 1 -- 172.21.15.106:0/60030113 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d3c06f390 con 0x7f6d3c06dc00 2024-04-25T10:34:42.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.731+0000 7f6d38ff9640 1 -- 172.21.15.106:0/60030113 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d2c00b6c0 con 0x7f6d3c06dc00 2024-04-25T10:34:42.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.731+0000 7f6d42a15640 1 -- 172.21.15.106:0/60030113 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d3c1f35c0 con 0x7f6d3c06dc00 2024-04-25T10:34:42.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.733+0000 7f6d42a15640 1 -- 172.21.15.106:0/60030113 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d3c06a3d0 con 0x7f6d3c06dc00 2024-04-25T10:34:42.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.733+0000 7f6d38ff9640 1 -- 172.21.15.106:0/60030113 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6d2c00b860 con 0x7f6d3c06dc00 2024-04-25T10:34:42.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.733+0000 7f6d38ff9640 1 --2- 172.21.15.106:0/60030113 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f6d280744b0 0x7f6d280769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:42.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.734+0000 7f6d38ff9640 1 -- 172.21.15.106:0/60030113 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f6d2c0954a0 con 0x7f6d3c06dc00 2024-04-25T10:34:42.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.734+0000 7f6d3b7fe640 1 --2- 172.21.15.106:0/60030113 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f6d280744b0 0x7f6d280769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:42.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.734+0000 7f6d3b7fe640 1 --2- 172.21.15.106:0/60030113 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f6d280744b0 0x7f6d280769c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c0ca4b0 tx=0x7f6d30008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:42.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.738+0000 7f6d38ff9640 1 -- 172.21.15.106:0/60030113 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6d2c05fcd0 con 0x7f6d3c06dc00 2024-04-25T10:34:42.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.910+0000 7f6d42a15640 1 -- 172.21.15.106:0/60030113 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6d3c156700 con 0x7f6d3c06dc00 2024-04-25T10:34:42.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.910+0000 7f6d38ff9640 1 -- 172.21.15.106:0/60030113 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f6d2c05f5d0 con 0x7f6d3c06dc00 2024-04-25T10:34:42.912 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-25T10:34:42.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.912+0000 7f6d42a15640 1 -- 172.21.15.106:0/60030113 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f6d280744b0 msgr2=0x7f6d280769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:42.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.912+0000 7f6d42a15640 1 --2- 172.21.15.106:0/60030113 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f6d280744b0 0x7f6d280769c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c0ca4b0 tx=0x7f6d30008040 comp rx=0 tx=0).stop 2024-04-25T10:34:42.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.912+0000 7f6d42a15640 1 -- 172.21.15.106:0/60030113 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d3c06dc00 msgr2=0x7f6d3c151aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:42.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.912+0000 7f6d42a15640 1 --2- 172.21.15.106:0/60030113 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d3c06dc00 0x7f6d3c151aa0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c00d7d0 tx=0x7f6d2c00dc90 comp rx=0 tx=0).stop 2024-04-25T10:34:42.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.912+0000 7f6d42a15640 1 -- 172.21.15.106:0/60030113 shutdown_connections 2024-04-25T10:34:42.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.912+0000 7f6d42a15640 1 --2- 172.21.15.106:0/60030113 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f6d280744b0 0x7f6d280769c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:42.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.912+0000 7f6d42a15640 1 --2- 172.21.15.106:0/60030113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d3c1601c0 0x7f6d3c14d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:42.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.912+0000 7f6d42a15640 1 --2- 172.21.15.106:0/60030113 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6d3c0c61a0 0x7f6d3c14fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:42.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.912+0000 7f6d42a15640 1 --2- 172.21.15.106:0/60030113 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d3c06dc00 0x7f6d3c151aa0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:42.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.912+0000 7f6d42a15640 1 -- 172.21.15.106:0/60030113 >> 172.21.15.106:0/60030113 conn(0x7f6d3c0727d0 msgr2=0x7f6d3c154770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:42.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.912+0000 7f6d42a15640 1 -- 172.21.15.106:0/60030113 shutdown_connections 2024-04-25T10:34:42.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:42.912+0000 7f6d42a15640 1 -- 172.21.15.106:0/60030113 wait complete. 2024-04-25T10:34:42.925 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:34:43.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.034+0000 7f2733640640 1 Processor -- start 2024-04-25T10:34:43.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.034+0000 7f2733640640 1 -- start start 2024-04-25T10:34:43.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.036+0000 7f2733640640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f272c0cb2b0 0x7f272c0cce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:43.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.036+0000 7f2733640640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f272c154ac0 0x7f272c0cd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:43.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.036+0000 7f2733640640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f272c15b8c0 con 0x7f272c0cc600 2024-04-25T10:34:43.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.036+0000 7f2733640640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f272c15eea0 con 0x7f272c154ac0 2024-04-25T10:34:43.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.036+0000 7f2733640640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f272c15b5e0 con 0x7f272c0cb2b0 2024-04-25T10:34:43.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.036+0000 7f2730bdb640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f272c0cb2b0 0x7f272c0cce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:43.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.036+0000 7f2731bdd640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f272c154ac0 0x7f272c0cd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:43.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.036+0000 7f2730bdb640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f272c0cb2b0 0x7f272c0cce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.106:33250/0 (socket says 172.21.15.106:33250) 2024-04-25T10:34:43.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.036+0000 7f2730bdb640 1 -- 172.21.15.106:0/1676153521 learned_addr learned my addr 172.21.15.106:0/1676153521 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:34:43.039 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.036+0000 7f2723fff640 1 -- 172.21.15.106:0/1676153521 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4221311229 0 0) 0x7f272c15b8c0 con 0x7f272c0cc600 2024-04-25T10:34:43.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.036+0000 7f2730bdb640 1 -- 172.21.15.106:0/1676153521 >> v2:172.21.15.134:3300/0 conn(0x7f272c154ac0 msgr2=0x7f272c0cd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:43.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.036+0000 7f2730bdb640 1 --2- 172.21.15.106:0/1676153521 >> v2:172.21.15.134:3300/0 conn(0x7f272c154ac0 0x7f272c0cd3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:43.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.036+0000 7f2730bdb640 1 -- 172.21.15.106:0/1676153521 >> v1:172.21.15.134:6789/0 conn(0x7f272c0cc600 legacy=0x7f272c0c5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:43.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.036+0000 7f2730bdb640 1 -- 172.21.15.106:0/1676153521 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f272c15e0e0 con 0x7f272c0cb2b0 2024-04-25T10:34:43.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.036+0000 7f2731bdd640 1 --2- 172.21.15.106:0/1676153521 >> v2:172.21.15.134:3300/0 conn(0x7f272c154ac0 0x7f272c0cd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:34:43.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.037+0000 7f2730bdb640 1 --2- 172.21.15.106:0/1676153521 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f272c0cb2b0 0x7f272c0cce90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f271c009870 tx=0x7f271c02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7fc02e2d45306cf0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:43.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.037+0000 7f2723fff640 1 -- 172.21.15.106:0/1676153521 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f271c037070 con 0x7f272c0cb2b0 2024-04-25T10:34:43.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.038+0000 7f2723fff640 1 -- 172.21.15.106:0/1676153521 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f271c032440 con 0x7f272c0cb2b0 2024-04-25T10:34:43.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.038+0000 7f2723fff640 1 -- 172.21.15.106:0/1676153521 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f271c032810 con 0x7f272c0cb2b0 2024-04-25T10:34:43.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.038+0000 7f2733640640 1 -- 172.21.15.106:0/1676153521 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f272c0cb2b0 msgr2=0x7f272c0cce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:43.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.038+0000 7f2733640640 1 --2- 172.21.15.106:0/1676153521 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f272c0cb2b0 0x7f272c0cce90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f271c009870 tx=0x7f271c02a270 comp rx=0 tx=0).stop 2024-04-25T10:34:43.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.038+0000 7f2733640640 1 -- 172.21.15.106:0/1676153521 shutdown_connections 2024-04-25T10:34:43.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.038+0000 7f2733640640 1 --2- 172.21.15.106:0/1676153521 >> v2:172.21.15.134:3300/0 conn(0x7f272c154ac0 0x7f272c0cd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:43.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.038+0000 7f2733640640 1 --2- 172.21.15.106:0/1676153521 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f272c0cb2b0 0x7f272c0cce90 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:43.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.038+0000 7f2733640640 1 -- 172.21.15.106:0/1676153521 >> 172.21.15.106:0/1676153521 conn(0x7f272c0727d0 msgr2=0x7f272c072c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:43.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.038+0000 7f2733640640 1 -- 172.21.15.106:0/1676153521 shutdown_connections 2024-04-25T10:34:43.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.038+0000 7f2733640640 1 -- 172.21.15.106:0/1676153521 wait complete. 2024-04-25T10:34:43.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.040+0000 7f2733640640 1 Processor -- start 2024-04-25T10:34:43.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.040+0000 7f2733640640 1 -- start start 2024-04-25T10:34:43.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.041+0000 7f2733640640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f272c0cb2b0 0x7f272c1f3150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:43.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.041+0000 7f2733640640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f272c0cc600 0x7f272c158fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:43.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.041+0000 7f2733640640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f272c154ac0 0x7f272c14f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:43.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.041+0000 7f2733640640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f272c15e980 con 0x7f272c154ac0 2024-04-25T10:34:43.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.041+0000 7f2733640640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f272c15e6a0 con 0x7f272c0cb2b0 2024-04-25T10:34:43.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.041+0000 7f2733640640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f272c0cbff0 con 0x7f272c0cc600 2024-04-25T10:34:43.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.042+0000 7f2731bdd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f272c154ac0 0x7f272c14f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:43.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.042+0000 7f2731bdd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f272c154ac0 0x7f272c14f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:37416/0 (socket says 172.21.15.106:37416) 2024-04-25T10:34:43.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.042+0000 7f2731bdd640 1 -- 172.21.15.106:0/3757908397 learned_addr learned my addr 172.21.15.106:0/3757908397 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:34:43.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.042+0000 7f2730bdb640 1 --2- 172.21.15.106:0/3757908397 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f272c0cc600 0x7f272c158fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:43.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.042+0000 7f27313dc640 1 --2- 172.21.15.106:0/3757908397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f272c0cb2b0 0x7f272c1f3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:43.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.042+0000 7f2731bdd640 1 -- 172.21.15.106:0/3757908397 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f272c0cc600 msgr2=0x7f272c158fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:43.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.042+0000 7f2731bdd640 1 --2- 172.21.15.106:0/3757908397 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f272c0cc600 0x7f272c158fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:43.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.042+0000 7f2731bdd640 1 -- 172.21.15.106:0/3757908397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f272c0cb2b0 msgr2=0x7f272c1f3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:43.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.042+0000 7f2731bdd640 1 --2- 172.21.15.106:0/3757908397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f272c0cb2b0 0x7f272c1f3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:43.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.042+0000 7f2731bdd640 1 -- 172.21.15.106:0/3757908397 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f272c1f3690 con 0x7f272c154ac0 2024-04-25T10:34:43.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.042+0000 7f2731bdd640 1 --2- 172.21.15.106:0/3757908397 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f272c154ac0 0x7f272c14f320 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f272400e880 tx=0x7f272400ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:43.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.042+0000 7f2721ffb640 1 -- 172.21.15.106:0/3757908397 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2724017070 con 0x7f272c154ac0 2024-04-25T10:34:43.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.042+0000 7f2721ffb640 1 -- 172.21.15.106:0/3757908397 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2724004020 con 0x7f272c154ac0 2024-04-25T10:34:43.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.042+0000 7f2733640640 1 -- 172.21.15.106:0/3757908397 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f272c1f2990 con 0x7f272c154ac0 2024-04-25T10:34:43.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.042+0000 7f2721ffb640 1 -- 172.21.15.106:0/3757908397 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27240043f0 con 0x7f272c154ac0 2024-04-25T10:34:43.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.042+0000 7f2733640640 1 -- 172.21.15.106:0/3757908397 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f272c1f6980 con 0x7f272c154ac0 2024-04-25T10:34:43.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.044+0000 7f2733640640 1 -- 172.21.15.106:0/3757908397 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f272c06a3d0 con 0x7f272c154ac0 2024-04-25T10:34:43.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.044+0000 7f2721ffb640 1 -- 172.21.15.106:0/3757908397 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f272400c8e0 con 0x7f272c154ac0 2024-04-25T10:34:43.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.044+0000 7f2721ffb640 1 --2- 172.21.15.106:0/3757908397 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f27100744e0 0x7f27100769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:43.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.044+0000 7f2721ffb640 1 -- 172.21.15.106:0/3757908397 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f2724095280 con 0x7f272c154ac0 2024-04-25T10:34:43.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.044+0000 7f27313dc640 1 --2- 172.21.15.106:0/3757908397 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f27100744e0 0x7f27100769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:43.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.045+0000 7f27313dc640 1 --2- 172.21.15.106:0/3757908397 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f27100744e0 0x7f27100769f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f27280012e0 tx=0x7f2728009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:43.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.049+0000 7f2721ffb640 1 -- 172.21.15.106:0/3757908397 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f272405f3d0 con 0x7f272c154ac0 2024-04-25T10:34:43.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.223+0000 7f2733640640 1 -- 172.21.15.106:0/3757908397 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f272c15adb0 con 0x7f272c154ac0 2024-04-25T10:34:43.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.224+0000 7f2721ffb640 1 -- 172.21.15.106:0/3757908397 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f27240636d0 con 0x7f272c154ac0 2024-04-25T10:34:43.225 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-25T10:34:43.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.226+0000 7f2733640640 1 -- 172.21.15.106:0/3757908397 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f27100744e0 msgr2=0x7f27100769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:43.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.226+0000 7f2733640640 1 --2- 172.21.15.106:0/3757908397 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f27100744e0 0x7f27100769f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f27280012e0 tx=0x7f2728009040 comp rx=0 tx=0).stop 2024-04-25T10:34:43.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.226+0000 7f2733640640 1 -- 172.21.15.106:0/3757908397 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f272c154ac0 msgr2=0x7f272c14f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:43.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.226+0000 7f2733640640 1 --2- 172.21.15.106:0/3757908397 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f272c154ac0 0x7f272c14f320 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f272400e880 tx=0x7f272400ed40 comp rx=0 tx=0).stop 2024-04-25T10:34:43.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.226+0000 7f2733640640 1 -- 172.21.15.106:0/3757908397 shutdown_connections 2024-04-25T10:34:43.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.226+0000 7f2733640640 1 --2- 172.21.15.106:0/3757908397 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f27100744e0 0x7f27100769f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:43.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.226+0000 7f2733640640 1 --2- 172.21.15.106:0/3757908397 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f272c154ac0 0x7f272c14f320 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:43.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.226+0000 7f2733640640 1 --2- 172.21.15.106:0/3757908397 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f272c0cc600 0x7f272c158fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:43.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.226+0000 7f2733640640 1 --2- 172.21.15.106:0/3757908397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f272c0cb2b0 0x7f272c1f3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:43.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.226+0000 7f2733640640 1 -- 172.21.15.106:0/3757908397 >> 172.21.15.106:0/3757908397 conn(0x7f272c0727d0 msgr2=0x7f272c15bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:43.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.226+0000 7f2733640640 1 -- 172.21.15.106:0/3757908397 shutdown_connections 2024-04-25T10:34:43.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.226+0000 7f2733640640 1 -- 172.21.15.106:0/3757908397 wait complete. 2024-04-25T10:34:43.239 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:34:43.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.347+0000 7ff99dc70640 1 Processor -- start 2024-04-25T10:34:43.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.348+0000 7ff99dc70640 1 -- start start 2024-04-25T10:34:43.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.349+0000 7ff99dc70640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff9981570d0 0x7ff99806dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:43.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.349+0000 7ff99dc70640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff9980c6250 0x7ff9980c6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:43.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.349+0000 7ff99dc70640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff99815d750 con 0x7ff99806f390 2024-04-25T10:34:43.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.349+0000 7ff99dc70640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7ff99815dcf0 con 0x7ff9980c6250 2024-04-25T10:34:43.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.349+0000 7ff99dc70640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7ff99806ef00 con 0x7ff9981570d0 2024-04-25T10:34:43.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.349+0000 7ff997fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff9980c6250 0x7ff9980c6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:43.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.349+0000 7ff997fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff9980c6250 0x7ff9980c6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:37424/0 (socket says 172.21.15.106:37424) 2024-04-25T10:34:43.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.349+0000 7ff997fff640 1 -- 172.21.15.106:0/3297415798 learned_addr learned my addr 172.21.15.106:0/3297415798 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:34:43.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.349+0000 7ff996ffd640 1 --2- 172.21.15.106:0/3297415798 >> v2:172.21.15.134:3300/0 conn(0x7ff9981570d0 0x7ff99806dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:43.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.349+0000 7ff997fff640 1 -- 172.21.15.106:0/3297415798 >> v2:172.21.15.134:3300/0 conn(0x7ff9981570d0 msgr2=0x7ff99806dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:43.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.349+0000 7ff997fff640 1 --2- 172.21.15.106:0/3297415798 >> v2:172.21.15.134:3300/0 conn(0x7ff9981570d0 0x7ff99806dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:43.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.349+0000 7ff997fff640 1 -- 172.21.15.106:0/3297415798 >> v1:172.21.15.134:6789/0 conn(0x7ff99806f390 legacy=0x7ff9980ca960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:43.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.349+0000 7ff997fff640 1 -- 172.21.15.106:0/3297415798 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff99815da10 con 0x7ff9980c6250 2024-04-25T10:34:43.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.350+0000 7ff997fff640 1 --2- 172.21.15.106:0/3297415798 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9980c6250 0x7ff9980c6660 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff998072240 tx=0x7ff99002a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4a3bcaea0e53022 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:43.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.350+0000 7ff9967fc640 1 -- 172.21.15.106:0/3297415798 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff990036070 con 0x7ff9980c6250 2024-04-25T10:34:43.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.350+0000 7ff9967fc640 1 -- 172.21.15.106:0/3297415798 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff990031600 con 0x7ff9980c6250 2024-04-25T10:34:43.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.350+0000 7ff9967fc640 1 -- 172.21.15.106:0/3297415798 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9900319d0 con 0x7ff9980c6250 2024-04-25T10:34:43.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.351+0000 7ff99dc70640 1 -- 172.21.15.106:0/3297415798 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9980c6250 msgr2=0x7ff9980c6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:43.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.351+0000 7ff99dc70640 1 --2- 172.21.15.106:0/3297415798 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9980c6250 0x7ff9980c6660 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff998072240 tx=0x7ff99002a780 comp rx=0 tx=0).stop 2024-04-25T10:34:43.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.351+0000 7ff99dc70640 1 -- 172.21.15.106:0/3297415798 shutdown_connections 2024-04-25T10:34:43.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.351+0000 7ff99dc70640 1 --2- 172.21.15.106:0/3297415798 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9980c6250 0x7ff9980c6660 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:43.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.351+0000 7ff99dc70640 1 --2- 172.21.15.106:0/3297415798 >> v2:172.21.15.134:3300/0 conn(0x7ff9981570d0 0x7ff99806dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:43.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.351+0000 7ff99dc70640 1 -- 172.21.15.106:0/3297415798 >> 172.21.15.106:0/3297415798 conn(0x7ff9980cff60 msgr2=0x7ff9980d0370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:43.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.351+0000 7ff99dc70640 1 -- 172.21.15.106:0/3297415798 shutdown_connections 2024-04-25T10:34:43.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.351+0000 7ff99dc70640 1 -- 172.21.15.106:0/3297415798 wait complete. 2024-04-25T10:34:43.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.353+0000 7ff99dc70640 1 Processor -- start 2024-04-25T10:34:43.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.353+0000 7ff99dc70640 1 -- start start 2024-04-25T10:34:43.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.354+0000 7ff99dc70640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff99806f390 0x7ff9980c7390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:43.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.354+0000 7ff99dc70640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9980c6250 0x7ff99815b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:43.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.354+0000 7ff99dc70640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9981570d0 0x7ff99815e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:43.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.354+0000 7ff99dc70640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff99806eb90 con 0x7ff9981570d0 2024-04-25T10:34:43.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.354+0000 7ff99dc70640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff99806e8b0 con 0x7ff9980c6250 2024-04-25T10:34:43.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.354+0000 7ff99dc70640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff99806d840 con 0x7ff99806f390 2024-04-25T10:34:43.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.354+0000 7ff997fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9981570d0 0x7ff99815e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:43.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.354+0000 7ff997fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9981570d0 0x7ff99815e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:37434/0 (socket says 172.21.15.106:37434) 2024-04-25T10:34:43.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.354+0000 7ff997fff640 1 -- 172.21.15.106:0/792630730 learned_addr learned my addr 172.21.15.106:0/792630730 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:34:43.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.354+0000 7ff996ffd640 1 --2- 172.21.15.106:0/792630730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9980c6250 0x7ff99815b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:43.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.354+0000 7ff9977fe640 1 --2- 172.21.15.106:0/792630730 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff99806f390 0x7ff9980c7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:43.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.355+0000 7ff997fff640 1 -- 172.21.15.106:0/792630730 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff99806f390 msgr2=0x7ff9980c7390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:43.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.355+0000 7ff997fff640 1 --2- 172.21.15.106:0/792630730 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff99806f390 0x7ff9980c7390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:43.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.355+0000 7ff997fff640 1 -- 172.21.15.106:0/792630730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9980c6250 msgr2=0x7ff99815b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:43.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.355+0000 7ff997fff640 1 --2- 172.21.15.106:0/792630730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9980c6250 0x7ff99815b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:43.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.355+0000 7ff997fff640 1 -- 172.21.15.106:0/792630730 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9980c78d0 con 0x7ff9981570d0 2024-04-25T10:34:43.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.355+0000 7ff997fff640 1 --2- 172.21.15.106:0/792630730 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9981570d0 0x7ff99815e070 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff99002ad70 tx=0x7ff990004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:43.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.355+0000 7ff97ffff640 1 -- 172.21.15.106:0/792630730 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff990036050 con 0x7ff9981570d0 2024-04-25T10:34:43.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.355+0000 7ff97ffff640 1 -- 172.21.15.106:0/792630730 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff990042070 con 0x7ff9981570d0 2024-04-25T10:34:43.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.355+0000 7ff99dc70640 1 -- 172.21.15.106:0/792630730 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff99815be60 con 0x7ff9981570d0 2024-04-25T10:34:43.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.355+0000 7ff97ffff640 1 -- 172.21.15.106:0/792630730 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff990009510 con 0x7ff9981570d0 2024-04-25T10:34:43.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.355+0000 7ff99dc70640 1 -- 172.21.15.106:0/792630730 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff99815ae40 con 0x7ff9981570d0 2024-04-25T10:34:43.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.356+0000 7ff99dc70640 1 -- 172.21.15.106:0/792630730 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9980d0ce0 con 0x7ff9981570d0 2024-04-25T10:34:43.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.356+0000 7ff97ffff640 1 -- 172.21.15.106:0/792630730 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff990007bd0 con 0x7ff9981570d0 2024-04-25T10:34:43.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.357+0000 7ff97ffff640 1 --2- 172.21.15.106:0/792630730 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7ff9840744b0 0x7ff9840769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:43.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.357+0000 7ff97ffff640 1 -- 172.21.15.106:0/792630730 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7ff9900b60a0 con 0x7ff9981570d0 2024-04-25T10:34:43.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.357+0000 7ff9977fe640 1 --2- 172.21.15.106:0/792630730 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7ff9840744b0 0x7ff9840769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:43.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.357+0000 7ff9977fe640 1 --2- 172.21.15.106:0/792630730 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7ff9840744b0 0x7ff9840769c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff988006fd0 tx=0x7ff988009060 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:43.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.362+0000 7ff97ffff640 1 -- 172.21.15.106:0/792630730 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff9900801f0 con 0x7ff9981570d0 2024-04-25T10:34:43.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.535+0000 7ff99dc70640 1 -- 172.21.15.106:0/792630730 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff99806a3d0 con 0x7ff9981570d0 2024-04-25T10:34:43.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.536+0000 7ff97ffff640 1 -- 172.21.15.106:0/792630730 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7ff9900844f0 con 0x7ff9981570d0 2024-04-25T10:34:43.537 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-25T10:34:43.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.538+0000 7ff99dc70640 1 -- 172.21.15.106:0/792630730 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7ff9840744b0 msgr2=0x7ff9840769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:43.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.538+0000 7ff99dc70640 1 --2- 172.21.15.106:0/792630730 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7ff9840744b0 0x7ff9840769c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff988006fd0 tx=0x7ff988009060 comp rx=0 tx=0).stop 2024-04-25T10:34:43.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.538+0000 7ff99dc70640 1 -- 172.21.15.106:0/792630730 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9981570d0 msgr2=0x7ff99815e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:43.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.538+0000 7ff99dc70640 1 --2- 172.21.15.106:0/792630730 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9981570d0 0x7ff99815e070 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff99002ad70 tx=0x7ff990004a40 comp rx=0 tx=0).stop 2024-04-25T10:34:43.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.538+0000 7ff99dc70640 1 -- 172.21.15.106:0/792630730 shutdown_connections 2024-04-25T10:34:43.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.538+0000 7ff99dc70640 1 --2- 172.21.15.106:0/792630730 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7ff9840744b0 0x7ff9840769c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:43.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.538+0000 7ff99dc70640 1 --2- 172.21.15.106:0/792630730 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9981570d0 0x7ff99815e070 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:43.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.538+0000 7ff99dc70640 1 --2- 172.21.15.106:0/792630730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9980c6250 0x7ff99815b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:43.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.538+0000 7ff99dc70640 1 --2- 172.21.15.106:0/792630730 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff99806f390 0x7ff9980c7390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:43.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.538+0000 7ff99dc70640 1 -- 172.21.15.106:0/792630730 >> 172.21.15.106:0/792630730 conn(0x7ff9980cff60 msgr2=0x7ff9980763f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:43.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.538+0000 7ff99dc70640 1 -- 172.21.15.106:0/792630730 shutdown_connections 2024-04-25T10:34:43.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.538+0000 7ff99dc70640 1 -- 172.21.15.106:0/792630730 wait complete. 2024-04-25T10:34:43.551 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:34:43.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.658+0000 7f341a24c640 1 Processor -- start 2024-04-25T10:34:43.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.658+0000 7f341a24c640 1 -- start start 2024-04-25T10:34:43.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.659+0000 7f341a24c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f34140ca170 0x7f3414156f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:43.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.659+0000 7f341a24c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f341415d510 0x7f341406c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:43.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.659+0000 7f341a24c640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3414073980 con 0x7f341406c5c0 2024-04-25T10:34:43.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.659+0000 7f341a24c640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f3414073f00 con 0x7f341415d510 2024-04-25T10:34:43.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.659+0000 7f341a24c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f3414074990 con 0x7f34140ca170 2024-04-25T10:34:43.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.659+0000 7f3413fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f341415d510 0x7f341406c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:43.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.660+0000 7f3413fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f341415d510 0x7f341406c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55980/0 (socket says 172.21.15.106:55980) 2024-04-25T10:34:43.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.660+0000 7f3413fff640 1 -- 172.21.15.106:0/2398073606 learned_addr learned my addr 172.21.15.106:0/2398073606 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:34:43.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.660+0000 7f34137fe640 1 --2- 172.21.15.106:0/2398073606 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f34140ca170 0x7f3414156f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:43.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.660+0000 7f3413fff640 1 -- 172.21.15.106:0/2398073606 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f34140ca170 msgr2=0x7f3414156f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:43.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.660+0000 7f3413fff640 1 --2- 172.21.15.106:0/2398073606 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f34140ca170 0x7f3414156f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:43.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.660+0000 7f3413fff640 1 -- 172.21.15.106:0/2398073606 >> v1:172.21.15.134:6789/0 conn(0x7f341406c5c0 legacy=0x7f341415ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:43.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.660+0000 7f3413fff640 1 -- 172.21.15.106:0/2398073606 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3414160ea0 con 0x7f341415d510 2024-04-25T10:34:43.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.660+0000 7f34137fe640 1 --2- 172.21.15.106:0/2398073606 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f34140ca170 0x7f3414156f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:34:43.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.660+0000 7f3413fff640 1 --2- 172.21.15.106:0/2398073606 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f341415d510 0x7f341406c9d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f340c009e50 tx=0x7f340c02a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bc8f4a69d49f47e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:43.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.661+0000 7f34127fc640 1 -- 172.21.15.106:0/2398073606 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f340c036070 con 0x7f341415d510 2024-04-25T10:34:43.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.661+0000 7f34127fc640 1 -- 172.21.15.106:0/2398073606 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f340c031600 con 0x7f341415d510 2024-04-25T10:34:43.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.661+0000 7f34127fc640 1 -- 172.21.15.106:0/2398073606 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f340c0319d0 con 0x7f341415d510 2024-04-25T10:34:43.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.661+0000 7f341a24c640 1 -- 172.21.15.106:0/2398073606 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f341415d510 msgr2=0x7f341406c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:43.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.661+0000 7f341a24c640 1 --2- 172.21.15.106:0/2398073606 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f341415d510 0x7f341406c9d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f340c009e50 tx=0x7f340c02a7a0 comp rx=0 tx=0).stop 2024-04-25T10:34:43.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.661+0000 7f341a24c640 1 -- 172.21.15.106:0/2398073606 shutdown_connections 2024-04-25T10:34:43.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.661+0000 7f341a24c640 1 --2- 172.21.15.106:0/2398073606 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f341415d510 0x7f341406c9d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:43.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.661+0000 7f341a24c640 1 --2- 172.21.15.106:0/2398073606 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f34140ca170 0x7f3414156f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:43.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.661+0000 7f341a24c640 1 -- 172.21.15.106:0/2398073606 >> 172.21.15.106:0/2398073606 conn(0x7f34140cef20 msgr2=0x7f34140cf330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:43.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.661+0000 7f341a24c640 1 -- 172.21.15.106:0/2398073606 shutdown_connections 2024-04-25T10:34:43.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.661+0000 7f341a24c640 1 -- 172.21.15.106:0/2398073606 wait complete. 2024-04-25T10:34:43.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.663+0000 7f341a24c640 1 Processor -- start 2024-04-25T10:34:43.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.663+0000 7f341a24c640 1 -- start start 2024-04-25T10:34:43.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.664+0000 7f341a24c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f341406c5c0 0x7f34141e6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:43.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.664+0000 7f341a24c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34140ca170 0x7f34140cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:43.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.664+0000 7f341a24c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f341415d510 0x7f3414151a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:43.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.664+0000 7f341a24c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3414161140 con 0x7f341415d510 2024-04-25T10:34:43.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.664+0000 7f341a24c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3414073c20 con 0x7f34140ca170 2024-04-25T10:34:43.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.664+0000 7f341a24c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f34140c9cd0 con 0x7f341406c5c0 2024-04-25T10:34:43.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.665+0000 7f3413fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f341415d510 0x7f3414151a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:43.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.665+0000 7f3413fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f341415d510 0x7f3414151a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55994/0 (socket says 172.21.15.106:55994) 2024-04-25T10:34:43.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.665+0000 7f3413fff640 1 -- 172.21.15.106:0/4258578901 learned_addr learned my addr 172.21.15.106:0/4258578901 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:34:43.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.665+0000 7f34137fe640 1 --2- 172.21.15.106:0/4258578901 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f341406c5c0 0x7f34141e6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:43.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.665+0000 7f3412ffd640 1 --2- 172.21.15.106:0/4258578901 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34140ca170 0x7f34140cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:43.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.665+0000 7f3413fff640 1 -- 172.21.15.106:0/4258578901 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f341406c5c0 msgr2=0x7f34141e6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:43.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.665+0000 7f3413fff640 1 --2- 172.21.15.106:0/4258578901 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f341406c5c0 0x7f34141e6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:43.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.665+0000 7f3413fff640 1 -- 172.21.15.106:0/4258578901 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34140ca170 msgr2=0x7f34140cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:43.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.665+0000 7f3413fff640 1 --2- 172.21.15.106:0/4258578901 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34140ca170 0x7f34140cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:43.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.665+0000 7f3413fff640 1 -- 172.21.15.106:0/4258578901 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34141f3a80 con 0x7f341415d510 2024-04-25T10:34:43.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.665+0000 7f3413fff640 1 --2- 172.21.15.106:0/4258578901 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f341415d510 0x7f3414151a40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f340c009f70 tx=0x7f340c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:43.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.665+0000 7f33fbfff640 1 -- 172.21.15.106:0/4258578901 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f340c041020 con 0x7f341415d510 2024-04-25T10:34:43.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.665+0000 7f33fbfff640 1 -- 172.21.15.106:0/4258578901 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f340c007780 con 0x7f341415d510 2024-04-25T10:34:43.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.665+0000 7f341a24c640 1 -- 172.21.15.106:0/4258578901 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34141566e0 con 0x7f341415d510 2024-04-25T10:34:43.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.665+0000 7f33fbfff640 1 -- 172.21.15.106:0/4258578901 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f340c036040 con 0x7f341415d510 2024-04-25T10:34:43.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.665+0000 7f341a24c640 1 -- 172.21.15.106:0/4258578901 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34141f2fa0 con 0x7f341415d510 2024-04-25T10:34:43.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.667+0000 7f341a24c640 1 -- 172.21.15.106:0/4258578901 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f341406a3d0 con 0x7f341415d510 2024-04-25T10:34:43.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.667+0000 7f33fbfff640 1 -- 172.21.15.106:0/4258578901 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f340c044020 con 0x7f341415d510 2024-04-25T10:34:43.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.667+0000 7f33fbfff640 1 --2- 172.21.15.106:0/4258578901 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f34000744e0 0x7f34000769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:43.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.667+0000 7f33fbfff640 1 -- 172.21.15.106:0/4258578901 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f340c0b5b80 con 0x7f341415d510 2024-04-25T10:34:43.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.667+0000 7f34137fe640 1 --2- 172.21.15.106:0/4258578901 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f34000744e0 0x7f34000769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:43.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.668+0000 7f34137fe640 1 --2- 172.21.15.106:0/4258578901 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f34000744e0 0x7f34000769f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3404006fd0 tx=0x7f3404008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:43.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.672+0000 7f33fbfff640 1 -- 172.21.15.106:0/4258578901 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f340c04b070 con 0x7f341415d510 2024-04-25T10:34:43.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.859+0000 7f341a24c640 1 -- 172.21.15.106:0/4258578901 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f34141e7390 con 0x7f341415d510 2024-04-25T10:34:43.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.860+0000 7f33fbfff640 1 -- 172.21.15.106:0/4258578901 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f340c07fc50 con 0x7f341415d510 2024-04-25T10:34:43.861 INFO:teuthology.orchestra.run.smithi106.stderr:listed 0 entries 2024-04-25T10:34:43.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.862+0000 7f341a24c640 1 -- 172.21.15.106:0/4258578901 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f34000744e0 msgr2=0x7f34000769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:43.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.862+0000 7f341a24c640 1 --2- 172.21.15.106:0/4258578901 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f34000744e0 0x7f34000769f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3404006fd0 tx=0x7f3404008040 comp rx=0 tx=0).stop 2024-04-25T10:34:43.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.862+0000 7f341a24c640 1 -- 172.21.15.106:0/4258578901 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f341415d510 msgr2=0x7f3414151a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:43.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.862+0000 7f341a24c640 1 --2- 172.21.15.106:0/4258578901 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f341415d510 0x7f3414151a40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f340c009f70 tx=0x7f340c004a40 comp rx=0 tx=0).stop 2024-04-25T10:34:43.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.862+0000 7f341a24c640 1 -- 172.21.15.106:0/4258578901 shutdown_connections 2024-04-25T10:34:43.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.862+0000 7f341a24c640 1 --2- 172.21.15.106:0/4258578901 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f34000744e0 0x7f34000769f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:43.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.862+0000 7f341a24c640 1 --2- 172.21.15.106:0/4258578901 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f341415d510 0x7f3414151a40 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:43.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.862+0000 7f341a24c640 1 --2- 172.21.15.106:0/4258578901 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34140ca170 0x7f34140cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:43.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.862+0000 7f341a24c640 1 --2- 172.21.15.106:0/4258578901 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f341406c5c0 0x7f34141e6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:43.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.862+0000 7f341a24c640 1 -- 172.21.15.106:0/4258578901 >> 172.21.15.106:0/4258578901 conn(0x7f34140cef20 msgr2=0x7f34140cab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:43.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.862+0000 7f341a24c640 1 -- 172.21.15.106:0/4258578901 shutdown_connections 2024-04-25T10:34:43.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:43.862+0000 7f341a24c640 1 -- 172.21.15.106:0/4258578901 wait complete. 2024-04-25T10:34:43.875 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi106.front.sepia.ceph.com... 2024-04-25T10:34:43.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:34:43.875 DEBUG:teuthology.orchestra.run.smithi106:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T10:34:43.898 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:34:43.900 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi106.stderr:ceph-fuse[43955]: fuse finished with error 0 and tester_r 0 2024-04-25T10:34:50.000 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi106.front.sepia.ceph.com 2024-04-25T10:34:50.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:34:50.001 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:34:50.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:34:50.030 DEBUG:teuthology.orchestra.run.smithi106:> ip netns list 2024-04-25T10:34:50.088 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:34:50.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:34:50.088 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:34:50.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:34:50.157 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link delete ceph-brx 2024-04-25T10:34:50.250 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-25T10:34:50.262 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T10:34:50.262 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:34:50.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.372+0000 7f25c4fd7640 1 Processor -- start 2024-04-25T10:34:50.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.372+0000 7f25c4fd7640 1 -- start start 2024-04-25T10:34:50.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.373+0000 7f25c4fd7640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f25c0072870 0x7f25c015a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:50.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.373+0000 7f25c4fd7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f25c015d9a0 0x7f25c0153710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:50.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.373+0000 7f25c4fd7640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25c00c5690 con 0x7f25c00c5280 2024-04-25T10:34:50.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.373+0000 7f25c4fd7640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f25c00cb590 con 0x7f25c0072870 2024-04-25T10:34:50.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.373+0000 7f25c4fd7640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f25c00c7c40 con 0x7f25c015d9a0 2024-04-25T10:34:50.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.374+0000 7f25bed9d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f25c015d9a0 0x7f25c0153710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:50.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.374+0000 7f25bed9d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f25c015d9a0 0x7f25c0153710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.106:59764/0 (socket says 172.21.15.106:59764) 2024-04-25T10:34:50.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.374+0000 7f25bed9d640 1 -- 172.21.15.106:0/3027095492 learned_addr learned my addr 172.21.15.106:0/3027095492 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:34:50.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.374+0000 7f25be59c640 1 --2- 172.21.15.106:0/3027095492 >> v2:172.21.15.134:3300/0 conn(0x7f25c0072870 0x7f25c015a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:50.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.374+0000 7f25bed9d640 1 -- 172.21.15.106:0/3027095492 >> v2:172.21.15.134:3300/0 conn(0x7f25c0072870 msgr2=0x7f25c015a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:50.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.374+0000 7f25bed9d640 1 --2- 172.21.15.106:0/3027095492 >> v2:172.21.15.134:3300/0 conn(0x7f25c0072870 0x7f25c015a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:50.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.374+0000 7f25bed9d640 1 -- 172.21.15.106:0/3027095492 >> v1:172.21.15.134:6789/0 conn(0x7f25c00c5280 legacy=0x7f25c00cae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:50.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.374+0000 7f25bed9d640 1 -- 172.21.15.106:0/3027095492 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25c00c7f20 con 0x7f25c015d9a0 2024-04-25T10:34:50.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.375+0000 7f25bed9d640 1 --2- 172.21.15.106:0/3027095492 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f25c015d9a0 0x7f25c0153710 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f25b8009e50 tx=0x7f25b802a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b36416d812a3f4e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:50.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.375+0000 7f25bdd9b640 1 -- 172.21.15.106:0/3027095492 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b8036070 con 0x7f25c015d9a0 2024-04-25T10:34:50.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.375+0000 7f25bdd9b640 1 -- 172.21.15.106:0/3027095492 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25b8031600 con 0x7f25c015d9a0 2024-04-25T10:34:50.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.375+0000 7f25bdd9b640 1 -- 172.21.15.106:0/3027095492 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b80319d0 con 0x7f25c015d9a0 2024-04-25T10:34:50.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.376+0000 7f25c4fd7640 1 -- 172.21.15.106:0/3027095492 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f25c015d9a0 msgr2=0x7f25c0153710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:50.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.376+0000 7f25c4fd7640 1 --2- 172.21.15.106:0/3027095492 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f25c015d9a0 0x7f25c0153710 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f25b8009e50 tx=0x7f25b802a680 comp rx=0 tx=0).stop 2024-04-25T10:34:50.381 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.376+0000 7f25c4fd7640 1 -- 172.21.15.106:0/3027095492 shutdown_connections 2024-04-25T10:34:50.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.376+0000 7f25c4fd7640 1 --2- 172.21.15.106:0/3027095492 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f25c015d9a0 0x7f25c0153710 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:50.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.376+0000 7f25c4fd7640 1 --2- 172.21.15.106:0/3027095492 >> v2:172.21.15.134:3300/0 conn(0x7f25c0072870 0x7f25c015a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:50.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.376+0000 7f25c4fd7640 1 -- 172.21.15.106:0/3027095492 >> 172.21.15.106:0/3027095492 conn(0x7f25c006c9b0 msgr2=0x7f25c006cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:50.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.376+0000 7f25c4fd7640 1 -- 172.21.15.106:0/3027095492 shutdown_connections 2024-04-25T10:34:50.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.376+0000 7f25c4fd7640 1 -- 172.21.15.106:0/3027095492 wait complete. 2024-04-25T10:34:50.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.378+0000 7f25c4fd7640 1 Processor -- start 2024-04-25T10:34:50.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.378+0000 7f25c4fd7640 1 -- start start 2024-04-25T10:34:50.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.379+0000 7f25c4fd7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25c0072870 0x7f25c01f5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:50.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.379+0000 7f25c4fd7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25c00c5280 0x7f25c00701b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:50.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.379+0000 7f25c4fd7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f25c015d9a0 0x7f25c014e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:50.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.379+0000 7f25c4fd7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f25c0159d90 con 0x7f25c00c5280 2024-04-25T10:34:50.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.379+0000 7f25c4fd7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f25c0159ab0 con 0x7f25c0072870 2024-04-25T10:34:50.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.379+0000 7f25c4fd7640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f25c015a070 con 0x7f25c015d9a0 2024-04-25T10:34:50.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.379+0000 7f25b7fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25c00c5280 0x7f25c00701b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:50.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.379+0000 7f25b7fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25c00c5280 0x7f25c00701b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56002/0 (socket says 172.21.15.106:56002) 2024-04-25T10:34:50.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.379+0000 7f25b7fff640 1 -- 172.21.15.106:0/2429653112 learned_addr learned my addr 172.21.15.106:0/2429653112 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:34:50.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.379+0000 7f25bed9d640 1 --2- 172.21.15.106:0/2429653112 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f25c015d9a0 0x7f25c014e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:50.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.379+0000 7f25be59c640 1 --2- 172.21.15.106:0/2429653112 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25c0072870 0x7f25c01f5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:50.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.380+0000 7f25b7fff640 1 -- 172.21.15.106:0/2429653112 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f25c015d9a0 msgr2=0x7f25c014e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:50.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.380+0000 7f25b7fff640 1 --2- 172.21.15.106:0/2429653112 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f25c015d9a0 0x7f25c014e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:50.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.380+0000 7f25b7fff640 1 -- 172.21.15.106:0/2429653112 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25c0072870 msgr2=0x7f25c01f5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:50.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.380+0000 7f25b7fff640 1 --2- 172.21.15.106:0/2429653112 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25c0072870 0x7f25c01f5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:50.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.380+0000 7f25b7fff640 1 -- 172.21.15.106:0/2429653112 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25c01f45b0 con 0x7f25c00c5280 2024-04-25T10:34:50.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.380+0000 7f25bed9d640 1 --2- 172.21.15.106:0/2429653112 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f25c015d9a0 0x7f25c014e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:34:50.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.380+0000 7f25b7fff640 1 --2- 172.21.15.106:0/2429653112 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25c00c5280 0x7f25c00701b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f25a800c3f0 tx=0x7f25a800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:50.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.380+0000 7f25b6ffd640 1 -- 172.21.15.106:0/2429653112 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a80052d0 con 0x7f25c00c5280 2024-04-25T10:34:50.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.380+0000 7f25b6ffd640 1 -- 172.21.15.106:0/2429653112 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25a8005490 con 0x7f25c00c5280 2024-04-25T10:34:50.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.380+0000 7f25c4fd7640 1 -- 172.21.15.106:0/2429653112 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25c0070c90 con 0x7f25c00c5280 2024-04-25T10:34:50.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.380+0000 7f25b6ffd640 1 -- 172.21.15.106:0/2429653112 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a8007260 con 0x7f25c00c5280 2024-04-25T10:34:50.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.380+0000 7f25c4fd7640 1 -- 172.21.15.106:0/2429653112 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25c0157cf0 con 0x7f25c00c5280 2024-04-25T10:34:50.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.381+0000 7f25c4fd7640 1 -- 172.21.15.106:0/2429653112 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25c00507e0 con 0x7f25c00c5280 2024-04-25T10:34:50.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.382+0000 7f25b6ffd640 1 -- 172.21.15.106:0/2429653112 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f25a8007400 con 0x7f25c00c5280 2024-04-25T10:34:50.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.382+0000 7f25b6ffd640 1 --2- 172.21.15.106:0/2429653112 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f25a00744e0 0x7f25a00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:50.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.382+0000 7f25be59c640 1 --2- 172.21.15.106:0/2429653112 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f25a00744e0 0x7f25a00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:50.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.382+0000 7f25b6ffd640 1 -- 172.21.15.106:0/2429653112 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f25a80945d0 con 0x7f25c00c5280 2024-04-25T10:34:50.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.383+0000 7f25be59c640 1 --2- 172.21.15.106:0/2429653112 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f25a00744e0 0x7f25a00769f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f25b0009aa0 tx=0x7f25b0009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:50.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.387+0000 7f25b6ffd640 1 -- 172.21.15.106:0/2429653112 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f25c00507e0 con 0x7f25c00c5280 2024-04-25T10:34:50.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.561+0000 7f25c4fd7640 1 -- 172.21.15.106:0/2429653112 --> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f25c01f5570 con 0x7f25a00744e0 2024-04-25T10:34:50.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.567+0000 7f25b6ffd640 1 -- 172.21.15.106:0/2429653112 <== mgr.4107 v2:172.21.15.106:6832/1744210167 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252928 (secure 0 0 0) 0x7f25b0011a00 con 0x7f25a00744e0 2024-04-25T10:34:50.573 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:34:50.574 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-25T10:34:50.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.570+0000 7f25c4fd7640 1 -- 172.21.15.106:0/2429653112 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f25a00744e0 msgr2=0x7f25a00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:50.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.570+0000 7f25c4fd7640 1 --2- 172.21.15.106:0/2429653112 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f25a00744e0 0x7f25a00769f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f25b0009aa0 tx=0x7f25b0009190 comp rx=0 tx=0).stop 2024-04-25T10:34:50.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.570+0000 7f25c4fd7640 1 -- 172.21.15.106:0/2429653112 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25c00c5280 msgr2=0x7f25c00701b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:50.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.570+0000 7f25c4fd7640 1 --2- 172.21.15.106:0/2429653112 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25c00c5280 0x7f25c00701b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f25a800c3f0 tx=0x7f25a800c7f0 comp rx=0 tx=0).stop 2024-04-25T10:34:50.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.570+0000 7f25c4fd7640 1 -- 172.21.15.106:0/2429653112 shutdown_connections 2024-04-25T10:34:50.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.570+0000 7f25c4fd7640 1 --2- 172.21.15.106:0/2429653112 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f25a00744e0 0x7f25a00769f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:50.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.570+0000 7f25c4fd7640 1 --2- 172.21.15.106:0/2429653112 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f25c015d9a0 0x7f25c014e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:50.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.570+0000 7f25c4fd7640 1 --2- 172.21.15.106:0/2429653112 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25c00c5280 0x7f25c00701b0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:50.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.570+0000 7f25c4fd7640 1 --2- 172.21.15.106:0/2429653112 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25c0072870 0x7f25c01f5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:50.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.570+0000 7f25c4fd7640 1 -- 172.21.15.106:0/2429653112 >> 172.21.15.106:0/2429653112 conn(0x7f25c006c9b0 msgr2=0x7f25c015ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:50.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.570+0000 7f25c4fd7640 1 -- 172.21.15.106:0/2429653112 shutdown_connections 2024-04-25T10:34:50.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.570+0000 7f25c4fd7640 1 -- 172.21.15.106:0/2429653112 wait complete. 2024-04-25T10:34:50.586 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":150,"stamp":"2024-04-25T10:34:50.160934+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":800831,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":71,"num_read_kb":60,"num_write":9662,"num_write_kb":25167776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10097,"ondisk_log_size":10097,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":307096,"kb_used_data":3608,"kb_used_omap":12,"kb_used_meta":303411,"kb_avail":754667624,"statfs":{"total":773094113280,"available":772779646976,"internally_reserved":0,"allocated":3694592,"data_stored":2266718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":310693424},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":672,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002917"},"pg_stats":[{"pgid":"3.3f","version":"22'146","reported_seq":164,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.965117+0000","last_change":"2024-04-25T10:30:32.440983+0000","last_active":"2024-04-25T10:34:43.965117+0000","last_peered":"2024-04-25T10:34:43.965117+0000","last_clean":"2024-04-25T10:34:43.965117+0000","last_became_active":"2024-04-25T10:30:32.440552+0000","last_became_peered":"2024-04-25T10:30:32.440552+0000","last_unstale":"2024-04-25T10:34:43.965117+0000","last_undegraded":"2024-04-25T10:34:43.965117+0000","last_fullsized":"2024-04-25T10:34:43.965117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:47.963061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"22'130","reported_seq":148,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.937860+0000","last_change":"2024-04-25T10:30:32.425695+0000","last_active":"2024-04-25T10:34:43.937860+0000","last_peered":"2024-04-25T10:34:43.937860+0000","last_clean":"2024-04-25T10:34:43.937860+0000","last_became_active":"2024-04-25T10:30:32.425368+0000","last_became_peered":"2024-04-25T10:30:32.425368+0000","last_unstale":"2024-04-25T10:34:43.937860+0000","last_undegraded":"2024-04-25T10:34:43.937860+0000","last_fullsized":"2024-04-25T10:34:43.937860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:39.120522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"22'151","reported_seq":169,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.929471+0000","last_change":"2024-04-25T10:30:32.430144+0000","last_active":"2024-04-25T10:34:43.929471+0000","last_peered":"2024-04-25T10:34:43.929471+0000","last_clean":"2024-04-25T10:34:43.929471+0000","last_became_active":"2024-04-25T10:30:32.429754+0000","last_became_peered":"2024-04-25T10:30:32.429754+0000","last_unstale":"2024-04-25T10:34:43.929471+0000","last_undegraded":"2024-04-25T10:34:43.929471+0000","last_fullsized":"2024-04-25T10:34:43.929471+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:11.448907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"22'173","reported_seq":191,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.964702+0000","last_change":"2024-04-25T10:30:32.442478+0000","last_active":"2024-04-25T10:34:43.964702+0000","last_peered":"2024-04-25T10:34:43.964702+0000","last_clean":"2024-04-25T10:34:43.964702+0000","last_became_active":"2024-04-25T10:30:32.442182+0000","last_became_peered":"2024-04-25T10:30:32.442182+0000","last_unstale":"2024-04-25T10:34:43.964702+0000","last_undegraded":"2024-04-25T10:34:43.964702+0000","last_fullsized":"2024-04-25T10:34:43.964702+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":173,"log_dups_size":0,"ondisk_log_size":173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:22.185978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":450560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"22'148","reported_seq":165,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.934476+0000","last_change":"2024-04-25T10:30:32.438231+0000","last_active":"2024-04-25T10:34:43.934476+0000","last_peered":"2024-04-25T10:34:43.934476+0000","last_clean":"2024-04-25T10:34:43.934476+0000","last_became_active":"2024-04-25T10:30:32.437656+0000","last_became_peered":"2024-04-25T10:30:32.437656+0000","last_unstale":"2024-04-25T10:34:43.934476+0000","last_undegraded":"2024-04-25T10:34:43.934476+0000","last_fullsized":"2024-04-25T10:34:43.934476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:56.915999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"22'135","reported_seq":146,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.925882+0000","last_change":"2024-04-25T10:31:13.676329+0000","last_active":"2024-04-25T10:34:43.925882+0000","last_peered":"2024-04-25T10:34:43.925882+0000","last_clean":"2024-04-25T10:34:43.925882+0000","last_became_active":"2024-04-25T10:31:13.676013+0000","last_became_peered":"2024-04-25T10:31:13.676013+0000","last_unstale":"2024-04-25T10:34:43.925882+0000","last_undegraded":"2024-04-25T10:34:43.925882+0000","last_fullsized":"2024-04-25T10:34:43.925882+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:07:15.646068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"22'164","reported_seq":181,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.941596+0000","last_change":"2024-04-25T10:30:32.429453+0000","last_active":"2024-04-25T10:34:43.941596+0000","last_peered":"2024-04-25T10:34:43.941596+0000","last_clean":"2024-04-25T10:34:43.941596+0000","last_became_active":"2024-04-25T10:30:32.429131+0000","last_became_peered":"2024-04-25T10:30:32.429131+0000","last_unstale":"2024-04-25T10:34:43.941596+0000","last_undegraded":"2024-04-25T10:34:43.941596+0000","last_fullsized":"2024-04-25T10:34:43.941596+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:26.785922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159,"num_write_kb":434176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"22'186","reported_seq":203,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.963510+0000","last_change":"2024-04-25T10:30:32.438860+0000","last_active":"2024-04-25T10:34:43.963510+0000","last_peered":"2024-04-25T10:34:43.963510+0000","last_clean":"2024-04-25T10:34:43.963510+0000","last_became_active":"2024-04-25T10:30:32.438491+0000","last_became_peered":"2024-04-25T10:30:32.438491+0000","last_unstale":"2024-04-25T10:34:43.963510+0000","last_undegraded":"2024-04-25T10:34:43.963510+0000","last_fullsized":"2024-04-25T10:34:43.963510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:08.610707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":177,"num_write_kb":483328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"22'152","reported_seq":169,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.967681+0000","last_change":"2024-04-25T10:30:32.440331+0000","last_active":"2024-04-25T10:34:43.967681+0000","last_peered":"2024-04-25T10:34:43.967681+0000","last_clean":"2024-04-25T10:34:43.967681+0000","last_became_active":"2024-04-25T10:30:32.440043+0000","last_became_peered":"2024-04-25T10:30:32.440043+0000","last_unstale":"2024-04-25T10:34:43.967681+0000","last_undegraded":"2024-04-25T10:34:43.967681+0000","last_fullsized":"2024-04-25T10:34:43.967681+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:17.684685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"22'156","reported_seq":173,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.968251+0000","last_change":"2024-04-25T10:30:32.435388+0000","last_active":"2024-04-25T10:34:43.968251+0000","last_peered":"2024-04-25T10:34:43.968251+0000","last_clean":"2024-04-25T10:34:43.968251+0000","last_became_active":"2024-04-25T10:30:32.434983+0000","last_became_peered":"2024-04-25T10:30:32.434983+0000","last_unstale":"2024-04-25T10:34:43.968251+0000","last_undegraded":"2024-04-25T10:34:43.968251+0000","last_fullsized":"2024-04-25T10:34:43.968251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:39.256604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"22'189","reported_seq":211,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.953182+0000","last_change":"2024-04-25T10:31:13.640959+0000","last_active":"2024-04-25T10:34:43.953182+0000","last_peered":"2024-04-25T10:34:43.953182+0000","last_clean":"2024-04-25T10:34:43.953182+0000","last_became_active":"2024-04-25T10:31:13.640582+0000","last_became_peered":"2024-04-25T10:31:13.640582+0000","last_unstale":"2024-04-25T10:34:43.953182+0000","last_undegraded":"2024-04-25T10:34:43.953182+0000","last_fullsized":"2024-04-25T10:34:43.953182+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:30.907171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"22'142","reported_seq":159,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.955837+0000","last_change":"2024-04-25T10:30:32.436403+0000","last_active":"2024-04-25T10:34:43.955837+0000","last_peered":"2024-04-25T10:34:43.955837+0000","last_clean":"2024-04-25T10:34:43.955837+0000","last_became_active":"2024-04-25T10:30:32.435908+0000","last_became_peered":"2024-04-25T10:30:32.435908+0000","last_unstale":"2024-04-25T10:34:43.955837+0000","last_undegraded":"2024-04-25T10:34:43.955837+0000","last_fullsized":"2024-04-25T10:34:43.955837+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:03.339316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"22'151","reported_seq":168,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.951346+0000","last_change":"2024-04-25T10:30:32.432110+0000","last_active":"2024-04-25T10:34:43.951346+0000","last_peered":"2024-04-25T10:34:43.951346+0000","last_clean":"2024-04-25T10:34:43.951346+0000","last_became_active":"2024-04-25T10:30:32.431792+0000","last_became_peered":"2024-04-25T10:30:32.431792+0000","last_unstale":"2024-04-25T10:34:43.951346+0000","last_undegraded":"2024-04-25T10:34:43.951346+0000","last_fullsized":"2024-04-25T10:34:43.951346+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:34.185989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"22'149","reported_seq":166,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.940258+0000","last_change":"2024-04-25T10:30:32.433676+0000","last_active":"2024-04-25T10:34:43.940258+0000","last_peered":"2024-04-25T10:34:43.940258+0000","last_clean":"2024-04-25T10:34:43.940258+0000","last_became_active":"2024-04-25T10:30:32.433329+0000","last_became_peered":"2024-04-25T10:30:32.433329+0000","last_unstale":"2024-04-25T10:34:43.940258+0000","last_undegraded":"2024-04-25T10:34:43.940258+0000","last_fullsized":"2024-04-25T10:34:43.940258+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:59.942368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"22'151","reported_seq":168,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.962728+0000","last_change":"2024-04-25T10:30:32.423582+0000","last_active":"2024-04-25T10:34:43.962728+0000","last_peered":"2024-04-25T10:34:43.962728+0000","last_clean":"2024-04-25T10:34:43.962728+0000","last_became_active":"2024-04-25T10:30:32.422991+0000","last_became_peered":"2024-04-25T10:30:32.422991+0000","last_unstale":"2024-04-25T10:34:43.962728+0000","last_undegraded":"2024-04-25T10:34:43.962728+0000","last_fullsized":"2024-04-25T10:34:43.962728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:11:47.132928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"22'147","reported_seq":164,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.965990+0000","last_change":"2024-04-25T10:30:32.440235+0000","last_active":"2024-04-25T10:34:43.965990+0000","last_peered":"2024-04-25T10:34:43.965990+0000","last_clean":"2024-04-25T10:34:43.965990+0000","last_became_active":"2024-04-25T10:30:32.439191+0000","last_became_peered":"2024-04-25T10:30:32.439191+0000","last_unstale":"2024-04-25T10:34:43.965990+0000","last_undegraded":"2024-04-25T10:34:43.965990+0000","last_fullsized":"2024-04-25T10:34:43.965990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":147,"log_dups_size":0,"ondisk_log_size":147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:51.423086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"22'146","reported_seq":163,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.959761+0000","last_change":"2024-04-25T10:30:32.442418+0000","last_active":"2024-04-25T10:34:43.959761+0000","last_peered":"2024-04-25T10:34:43.959761+0000","last_clean":"2024-04-25T10:34:43.959761+0000","last_became_active":"2024-04-25T10:30:32.442125+0000","last_became_peered":"2024-04-25T10:30:32.442125+0000","last_unstale":"2024-04-25T10:34:43.959761+0000","last_undegraded":"2024-04-25T10:34:43.959761+0000","last_fullsized":"2024-04-25T10:34:43.959761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:14.213809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"22'157","reported_seq":174,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.925082+0000","last_change":"2024-04-25T10:30:32.434758+0000","last_active":"2024-04-25T10:34:43.925082+0000","last_peered":"2024-04-25T10:34:43.925082+0000","last_clean":"2024-04-25T10:34:43.925082+0000","last_became_active":"2024-04-25T10:30:32.434492+0000","last_became_peered":"2024-04-25T10:30:32.434492+0000","last_unstale":"2024-04-25T10:34:43.925082+0000","last_undegraded":"2024-04-25T10:34:43.925082+0000","last_fullsized":"2024-04-25T10:34:43.925082+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:41.454407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"22'172","reported_seq":189,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.948403+0000","last_change":"2024-04-25T10:30:32.440399+0000","last_active":"2024-04-25T10:34:43.948403+0000","last_peered":"2024-04-25T10:34:43.948403+0000","last_clean":"2024-04-25T10:34:43.948403+0000","last_became_active":"2024-04-25T10:30:32.438227+0000","last_became_peered":"2024-04-25T10:30:32.438227+0000","last_unstale":"2024-04-25T10:34:43.948403+0000","last_undegraded":"2024-04-25T10:34:43.948403+0000","last_fullsized":"2024-04-25T10:34:43.948403+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:49.571671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":450560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"22'161","reported_seq":178,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.963147+0000","last_change":"2024-04-25T10:30:32.431609+0000","last_active":"2024-04-25T10:34:43.963147+0000","last_peered":"2024-04-25T10:34:43.963147+0000","last_clean":"2024-04-25T10:34:43.963147+0000","last_became_active":"2024-04-25T10:30:32.431314+0000","last_became_peered":"2024-04-25T10:30:32.431314+0000","last_unstale":"2024-04-25T10:34:43.963147+0000","last_undegraded":"2024-04-25T10:34:43.963147+0000","last_fullsized":"2024-04-25T10:34:43.963147+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:33.421989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":153,"num_write_kb":417792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"22'137","reported_seq":148,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.956088+0000","last_change":"2024-04-25T10:31:13.677426+0000","last_active":"2024-04-25T10:34:43.956088+0000","last_peered":"2024-04-25T10:34:43.956088+0000","last_clean":"2024-04-25T10:34:43.956088+0000","last_became_active":"2024-04-25T10:31:13.677164+0000","last_became_peered":"2024-04-25T10:31:13.677164+0000","last_unstale":"2024-04-25T10:34:43.956088+0000","last_undegraded":"2024-04-25T10:34:43.956088+0000","last_fullsized":"2024-04-25T10:34:43.956088+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:35.850650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2a","version":"22'166","reported_seq":183,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.968866+0000","last_change":"2024-04-25T10:30:32.444618+0000","last_active":"2024-04-25T10:34:43.968866+0000","last_peered":"2024-04-25T10:34:43.968866+0000","last_clean":"2024-04-25T10:34:43.968866+0000","last_became_active":"2024-04-25T10:30:32.444273+0000","last_became_peered":"2024-04-25T10:30:32.444273+0000","last_unstale":"2024-04-25T10:34:43.968866+0000","last_undegraded":"2024-04-25T10:34:43.968866+0000","last_fullsized":"2024-04-25T10:34:43.968866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:09.295251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"22'134","reported_seq":151,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.960474+0000","last_change":"2024-04-25T10:30:32.426578+0000","last_active":"2024-04-25T10:34:43.960474+0000","last_peered":"2024-04-25T10:34:43.960474+0000","last_clean":"2024-04-25T10:34:43.960474+0000","last_became_active":"2024-04-25T10:30:32.426223+0000","last_became_peered":"2024-04-25T10:30:32.426223+0000","last_unstale":"2024-04-25T10:34:43.960474+0000","last_undegraded":"2024-04-25T10:34:43.960474+0000","last_fullsized":"2024-04-25T10:34:43.960474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:53.700585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"22'180","reported_seq":197,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.964675+0000","last_change":"2024-04-25T10:30:32.439767+0000","last_active":"2024-04-25T10:34:43.964675+0000","last_peered":"2024-04-25T10:34:43.964675+0000","last_clean":"2024-04-25T10:34:43.964675+0000","last_became_active":"2024-04-25T10:30:32.438801+0000","last_became_peered":"2024-04-25T10:30:32.438801+0000","last_unstale":"2024-04-25T10:34:43.964675+0000","last_undegraded":"2024-04-25T10:34:43.964675+0000","last_fullsized":"2024-04-25T10:34:43.964675+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:58.231512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":174,"num_write_kb":475136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"22'134","reported_seq":151,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.959780+0000","last_change":"2024-04-25T10:30:32.427424+0000","last_active":"2024-04-25T10:34:43.959780+0000","last_peered":"2024-04-25T10:34:43.959780+0000","last_clean":"2024-04-25T10:34:43.959780+0000","last_became_active":"2024-04-25T10:30:32.427081+0000","last_became_peered":"2024-04-25T10:30:32.427081+0000","last_unstale":"2024-04-25T10:34:43.959780+0000","last_undegraded":"2024-04-25T10:34:43.959780+0000","last_fullsized":"2024-04-25T10:34:43.959780+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:47.372451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"22'198","reported_seq":215,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.968855+0000","last_change":"2024-04-25T10:30:32.432472+0000","last_active":"2024-04-25T10:34:43.968855+0000","last_peered":"2024-04-25T10:34:43.968855+0000","last_clean":"2024-04-25T10:34:43.968855+0000","last_became_active":"2024-04-25T10:30:32.432121+0000","last_became_peered":"2024-04-25T10:30:32.432121+0000","last_unstale":"2024-04-25T10:34:43.968855+0000","last_undegraded":"2024-04-25T10:34:43.968855+0000","last_fullsized":"2024-04-25T10:34:43.968855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":198,"log_dups_size":0,"ondisk_log_size":198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:34.191339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"22'144","reported_seq":161,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.933640+0000","last_change":"2024-04-25T10:30:32.425108+0000","last_active":"2024-04-25T10:34:43.933640+0000","last_peered":"2024-04-25T10:34:43.933640+0000","last_clean":"2024-04-25T10:34:43.933640+0000","last_became_active":"2024-04-25T10:30:32.424794+0000","last_became_peered":"2024-04-25T10:30:32.424794+0000","last_unstale":"2024-04-25T10:34:43.933640+0000","last_undegraded":"2024-04-25T10:34:43.933640+0000","last_fullsized":"2024-04-25T10:34:43.933640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:01.282758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"22'133","reported_seq":150,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.960152+0000","last_change":"2024-04-25T10:30:32.426672+0000","last_active":"2024-04-25T10:34:43.960152+0000","last_peered":"2024-04-25T10:34:43.960152+0000","last_clean":"2024-04-25T10:34:43.960152+0000","last_became_active":"2024-04-25T10:30:32.426348+0000","last_became_peered":"2024-04-25T10:30:32.426348+0000","last_unstale":"2024-04-25T10:34:43.960152+0000","last_undegraded":"2024-04-25T10:34:43.960152+0000","last_fullsized":"2024-04-25T10:34:43.960152+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:57.645772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"22'118","reported_seq":136,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.939753+0000","last_change":"2024-04-25T10:30:32.427989+0000","last_active":"2024-04-25T10:34:43.939753+0000","last_peered":"2024-04-25T10:34:43.939753+0000","last_clean":"2024-04-25T10:34:43.939753+0000","last_became_active":"2024-04-25T10:30:32.427676+0000","last_became_peered":"2024-04-25T10:30:32.427676+0000","last_unstale":"2024-04-25T10:34:43.939753+0000","last_undegraded":"2024-04-25T10:34:43.939753+0000","last_fullsized":"2024-04-25T10:34:43.939753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:15:02.332186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":111,"num_write_kb":303104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"22'131","reported_seq":149,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.960144+0000","last_change":"2024-04-25T10:30:32.441112+0000","last_active":"2024-04-25T10:34:43.960144+0000","last_peered":"2024-04-25T10:34:43.960144+0000","last_clean":"2024-04-25T10:34:43.960144+0000","last_became_active":"2024-04-25T10:30:32.440815+0000","last_became_peered":"2024-04-25T10:30:32.440815+0000","last_unstale":"2024-04-25T10:34:43.960144+0000","last_undegraded":"2024-04-25T10:34:43.960144+0000","last_fullsized":"2024-04-25T10:34:43.960144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:02.956759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"22'135","reported_seq":153,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.938625+0000","last_change":"2024-04-25T10:30:32.430902+0000","last_active":"2024-04-25T10:34:43.938625+0000","last_peered":"2024-04-25T10:34:43.938625+0000","last_clean":"2024-04-25T10:34:43.938625+0000","last_became_active":"2024-04-25T10:30:32.430615+0000","last_became_peered":"2024-04-25T10:30:32.430615+0000","last_unstale":"2024-04-25T10:34:43.938625+0000","last_undegraded":"2024-04-25T10:34:43.938625+0000","last_fullsized":"2024-04-25T10:34:43.938625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:12.747595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"22'150","reported_seq":168,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.959087+0000","last_change":"2024-04-25T10:30:32.435379+0000","last_active":"2024-04-25T10:34:43.959087+0000","last_peered":"2024-04-25T10:34:43.959087+0000","last_clean":"2024-04-25T10:34:43.959087+0000","last_became_active":"2024-04-25T10:30:32.435055+0000","last_became_peered":"2024-04-25T10:30:32.435055+0000","last_unstale":"2024-04-25T10:34:43.959087+0000","last_undegraded":"2024-04-25T10:34:43.959087+0000","last_fullsized":"2024-04-25T10:34:43.959087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:13.623762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"22'130","reported_seq":147,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.959593+0000","last_change":"2024-04-25T10:30:32.422102+0000","last_active":"2024-04-25T10:34:43.959593+0000","last_peered":"2024-04-25T10:34:43.959593+0000","last_clean":"2024-04-25T10:34:43.959593+0000","last_became_active":"2024-04-25T10:30:32.421766+0000","last_became_peered":"2024-04-25T10:30:32.421766+0000","last_unstale":"2024-04-25T10:34:43.959593+0000","last_undegraded":"2024-04-25T10:34:43.959593+0000","last_fullsized":"2024-04-25T10:34:43.959593+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:30.300572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'7","reported_seq":26,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:33:05.134865+0000","last_change":"2024-04-25T10:30:31.418065+0000","last_active":"2024-04-25T10:33:05.134865+0000","last_peered":"2024-04-25T10:33:05.134865+0000","last_clean":"2024-04-25T10:33:05.134865+0000","last_became_active":"2024-04-25T10:30:31.417451+0000","last_became_peered":"2024-04-25T10:30:31.417451+0000","last_unstale":"2024-04-25T10:33:05.134865+0000","last_undegraded":"2024-04-25T10:33:05.134865+0000","last_fullsized":"2024-04-25T10:33:05.134865+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:17.262861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1204,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"22'137","reported_seq":154,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.941470+0000","last_change":"2024-04-25T10:30:32.426913+0000","last_active":"2024-04-25T10:34:43.941470+0000","last_peered":"2024-04-25T10:34:43.941470+0000","last_clean":"2024-04-25T10:34:43.941470+0000","last_became_active":"2024-04-25T10:30:32.426616+0000","last_became_peered":"2024-04-25T10:30:32.426616+0000","last_unstale":"2024-04-25T10:34:43.941470+0000","last_undegraded":"2024-04-25T10:34:43.941470+0000","last_fullsized":"2024-04-25T10:34:43.941470+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:50.614932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'7","reported_seq":26,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:31:13.638951+0000","last_change":"2024-04-25T10:30:31.416432+0000","last_active":"2024-04-25T10:31:13.638951+0000","last_peered":"2024-04-25T10:31:13.638951+0000","last_clean":"2024-04-25T10:31:13.638951+0000","last_became_active":"2024-04-25T10:30:31.415112+0000","last_became_peered":"2024-04-25T10:30:31.415112+0000","last_unstale":"2024-04-25T10:31:13.638951+0000","last_undegraded":"2024-04-25T10:31:13.638951+0000","last_fullsized":"2024-04-25T10:31:13.638951+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:22.674365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"20'3","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:31:13.636650+0000","last_change":"2024-04-25T10:30:31.418059+0000","last_active":"2024-04-25T10:31:13.636650+0000","last_peered":"2024-04-25T10:31:13.636650+0000","last_clean":"2024-04-25T10:31:13.636650+0000","last_became_active":"2024-04-25T10:30:31.417716+0000","last_became_peered":"2024-04-25T10:30:31.417716+0000","last_unstale":"2024-04-25T10:31:13.636650+0000","last_undegraded":"2024-04-25T10:31:13.636650+0000","last_fullsized":"2024-04-25T10:31:13.636650+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:57.752218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"22'170","reported_seq":187,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.938474+0000","last_change":"2024-04-25T10:30:32.428078+0000","last_active":"2024-04-25T10:34:43.938474+0000","last_peered":"2024-04-25T10:34:43.938474+0000","last_clean":"2024-04-25T10:34:43.938474+0000","last_became_active":"2024-04-25T10:30:32.427700+0000","last_became_peered":"2024-04-25T10:30:32.427700+0000","last_unstale":"2024-04-25T10:34:43.938474+0000","last_undegraded":"2024-04-25T10:34:43.938474+0000","last_fullsized":"2024-04-25T10:34:43.938474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:26.931791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":162,"num_write_kb":442368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"20'3","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:31:13.638118+0000","last_change":"2024-04-25T10:30:31.415030+0000","last_active":"2024-04-25T10:31:13.638118+0000","last_peered":"2024-04-25T10:31:13.638118+0000","last_clean":"2024-04-25T10:31:13.638118+0000","last_became_active":"2024-04-25T10:30:31.414542+0000","last_became_peered":"2024-04-25T10:30:31.414542+0000","last_unstale":"2024-04-25T10:31:13.638118+0000","last_undegraded":"2024-04-25T10:31:13.638118+0000","last_fullsized":"2024-04-25T10:31:13.638118+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:20.082420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"22'157","reported_seq":174,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.939532+0000","last_change":"2024-04-25T10:30:32.425207+0000","last_active":"2024-04-25T10:34:43.939532+0000","last_peered":"2024-04-25T10:34:43.939532+0000","last_clean":"2024-04-25T10:34:43.939532+0000","last_became_active":"2024-04-25T10:30:32.424849+0000","last_became_peered":"2024-04-25T10:30:32.424849+0000","last_unstale":"2024-04-25T10:34:43.939532+0000","last_undegraded":"2024-04-25T10:34:43.939532+0000","last_fullsized":"2024-04-25T10:34:43.939532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:11.533686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"22'8","reported_seq":27,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.902339+0000","last_change":"2024-04-25T10:30:31.407700+0000","last_active":"2024-04-25T10:34:43.902339+0000","last_peered":"2024-04-25T10:34:43.902339+0000","last_clean":"2024-04-25T10:34:43.902339+0000","last_became_active":"2024-04-25T10:30:31.407376+0000","last_became_peered":"2024-04-25T10:30:31.407376+0000","last_unstale":"2024-04-25T10:34:43.902339+0000","last_undegraded":"2024-04-25T10:34:43.902339+0000","last_fullsized":"2024-04-25T10:34:43.902339+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:50.112742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"22'229","reported_seq":246,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.960310+0000","last_change":"2024-04-25T10:30:32.444599+0000","last_active":"2024-04-25T10:34:43.960310+0000","last_peered":"2024-04-25T10:34:43.960310+0000","last_clean":"2024-04-25T10:34:43.960310+0000","last_became_active":"2024-04-25T10:30:32.444250+0000","last_became_peered":"2024-04-25T10:30:32.444250+0000","last_unstale":"2024-04-25T10:34:43.960310+0000","last_undegraded":"2024-04-25T10:34:43.960310+0000","last_fullsized":"2024-04-25T10:34:43.960310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":229,"log_dups_size":0,"ondisk_log_size":229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:55.711525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":219,"num_write_kb":598016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"20'4","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:31:13.638996+0000","last_change":"2024-04-25T10:30:31.405051+0000","last_active":"2024-04-25T10:31:13.638996+0000","last_peered":"2024-04-25T10:31:13.638996+0000","last_clean":"2024-04-25T10:31:13.638996+0000","last_became_active":"2024-04-25T10:30:31.404669+0000","last_became_peered":"2024-04-25T10:30:31.404669+0000","last_unstale":"2024-04-25T10:31:13.638996+0000","last_undegraded":"2024-04-25T10:31:13.638996+0000","last_fullsized":"2024-04-25T10:31:13.638996+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:45.260847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"22'141","reported_seq":158,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.953061+0000","last_change":"2024-04-25T10:30:32.435804+0000","last_active":"2024-04-25T10:34:43.953061+0000","last_peered":"2024-04-25T10:34:43.953061+0000","last_clean":"2024-04-25T10:34:43.953061+0000","last_became_active":"2024-04-25T10:30:32.435471+0000","last_became_peered":"2024-04-25T10:30:32.435471+0000","last_unstale":"2024-04-25T10:34:43.953061+0000","last_undegraded":"2024-04-25T10:34:43.953061+0000","last_fullsized":"2024-04-25T10:34:43.953061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:48.213846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"22'11","reported_seq":30,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:27.461358+0000","last_change":"2024-04-25T10:30:31.403560+0000","last_active":"2024-04-25T10:34:27.461358+0000","last_peered":"2024-04-25T10:34:27.461358+0000","last_clean":"2024-04-25T10:34:27.461358+0000","last_became_active":"2024-04-25T10:30:31.403167+0000","last_became_peered":"2024-04-25T10:30:31.403167+0000","last_unstale":"2024-04-25T10:34:27.461358+0000","last_undegraded":"2024-04-25T10:34:27.461358+0000","last_fullsized":"2024-04-25T10:34:27.461358+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:54.592832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"22'149","reported_seq":166,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.967685+0000","last_change":"2024-04-25T10:30:32.424101+0000","last_active":"2024-04-25T10:34:43.967685+0000","last_peered":"2024-04-25T10:34:43.967685+0000","last_clean":"2024-04-25T10:34:43.967685+0000","last_became_active":"2024-04-25T10:30:32.423780+0000","last_became_peered":"2024-04-25T10:30:32.423780+0000","last_unstale":"2024-04-25T10:34:43.967685+0000","last_undegraded":"2024-04-25T10:34:43.967685+0000","last_fullsized":"2024-04-25T10:34:43.967685+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:29.175348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"22'95","reported_seq":114,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.905445+0000","last_change":"2024-04-25T10:30:31.418201+0000","last_active":"2024-04-25T10:34:43.905445+0000","last_peered":"2024-04-25T10:34:43.905445+0000","last_clean":"2024-04-25T10:34:43.905445+0000","last_became_active":"2024-04-25T10:30:31.417824+0000","last_became_peered":"2024-04-25T10:30:31.417824+0000","last_unstale":"2024-04-25T10:34:43.905445+0000","last_undegraded":"2024-04-25T10:34:43.905445+0000","last_fullsized":"2024-04-25T10:34:43.905445+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:38.584225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":201790,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":96,"num_write_kb":233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"22'150","reported_seq":167,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.951616+0000","last_change":"2024-04-25T10:30:32.431139+0000","last_active":"2024-04-25T10:34:43.951616+0000","last_peered":"2024-04-25T10:34:43.951616+0000","last_clean":"2024-04-25T10:34:43.951616+0000","last_became_active":"2024-04-25T10:30:32.429622+0000","last_became_peered":"2024-04-25T10:30:32.429622+0000","last_unstale":"2024-04-25T10:34:43.951616+0000","last_undegraded":"2024-04-25T10:34:43.951616+0000","last_fullsized":"2024-04-25T10:34:43.951616+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:39.993647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'9","reported_seq":28,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:31:13.675426+0000","last_change":"2024-04-25T10:30:31.414775+0000","last_active":"2024-04-25T10:31:13.675426+0000","last_peered":"2024-04-25T10:31:13.675426+0000","last_clean":"2024-04-25T10:31:13.675426+0000","last_became_active":"2024-04-25T10:30:31.414422+0000","last_became_peered":"2024-04-25T10:30:31.414422+0000","last_unstale":"2024-04-25T10:31:13.675426+0000","last_undegraded":"2024-04-25T10:31:13.675426+0000","last_fullsized":"2024-04-25T10:31:13.675426+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:37.803505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"22'157","reported_seq":174,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.944852+0000","last_change":"2024-04-25T10:30:32.432884+0000","last_active":"2024-04-25T10:34:43.944852+0000","last_peered":"2024-04-25T10:34:43.944852+0000","last_clean":"2024-04-25T10:34:43.944852+0000","last_became_active":"2024-04-25T10:30:32.431743+0000","last_became_peered":"2024-04-25T10:30:32.431743+0000","last_unstale":"2024-04-25T10:34:43.944852+0000","last_undegraded":"2024-04-25T10:34:43.944852+0000","last_fullsized":"2024-04-25T10:34:43.944852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:15:06.737631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"20'5","reported_seq":24,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:31:13.638219+0000","last_change":"2024-04-25T10:30:31.419497+0000","last_active":"2024-04-25T10:31:13.638219+0000","last_peered":"2024-04-25T10:31:13.638219+0000","last_clean":"2024-04-25T10:31:13.638219+0000","last_became_active":"2024-04-25T10:30:31.419135+0000","last_became_peered":"2024-04-25T10:30:31.419135+0000","last_unstale":"2024-04-25T10:31:13.638219+0000","last_undegraded":"2024-04-25T10:31:13.638219+0000","last_fullsized":"2024-04-25T10:31:13.638219+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:59.403927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"22'171","reported_seq":188,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.964294+0000","last_change":"2024-04-25T10:30:32.434197+0000","last_active":"2024-04-25T10:34:43.964294+0000","last_peered":"2024-04-25T10:34:43.964294+0000","last_clean":"2024-04-25T10:34:43.964294+0000","last_became_active":"2024-04-25T10:30:32.433623+0000","last_became_peered":"2024-04-25T10:30:32.433623+0000","last_unstale":"2024-04-25T10:34:43.964294+0000","last_undegraded":"2024-04-25T10:34:43.964294+0000","last_fullsized":"2024-04-25T10:34:43.964294+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:08.027472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"22'7","reported_seq":26,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:32:55.135185+0000","last_change":"2024-04-25T10:30:31.416901+0000","last_active":"2024-04-25T10:32:55.135185+0000","last_peered":"2024-04-25T10:32:55.135185+0000","last_clean":"2024-04-25T10:32:55.135185+0000","last_became_active":"2024-04-25T10:30:31.415706+0000","last_became_peered":"2024-04-25T10:30:31.415706+0000","last_unstale":"2024-04-25T10:32:55.135185+0000","last_undegraded":"2024-04-25T10:32:55.135185+0000","last_fullsized":"2024-04-25T10:32:55.135185+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:23.243039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"22'165","reported_seq":182,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.946112+0000","last_change":"2024-04-25T10:30:32.435089+0000","last_active":"2024-04-25T10:34:43.946112+0000","last_peered":"2024-04-25T10:34:43.946112+0000","last_clean":"2024-04-25T10:34:43.946112+0000","last_became_active":"2024-04-25T10:30:32.434737+0000","last_became_peered":"2024-04-25T10:30:32.434737+0000","last_unstale":"2024-04-25T10:34:43.946112+0000","last_undegraded":"2024-04-25T10:34:43.946112+0000","last_fullsized":"2024-04-25T10:34:43.946112+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:19.738850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159,"num_write_kb":434176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"22'18","reported_seq":37,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:32:55.131953+0000","last_change":"2024-04-25T10:30:31.405163+0000","last_active":"2024-04-25T10:32:55.131953+0000","last_peered":"2024-04-25T10:32:55.131953+0000","last_clean":"2024-04-25T10:32:55.131953+0000","last_became_active":"2024-04-25T10:30:31.404775+0000","last_became_peered":"2024-04-25T10:30:31.404775+0000","last_unstale":"2024-04-25T10:32:55.131953+0000","last_undegraded":"2024-04-25T10:32:55.131953+0000","last_fullsized":"2024-04-25T10:32:55.131953+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:10:21.595653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"22'189","reported_seq":206,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.961535+0000","last_change":"2024-04-25T10:30:32.432061+0000","last_active":"2024-04-25T10:34:43.961535+0000","last_peered":"2024-04-25T10:34:43.961535+0000","last_clean":"2024-04-25T10:34:43.961535+0000","last_became_active":"2024-04-25T10:30:32.431651+0000","last_became_peered":"2024-04-25T10:30:32.431651+0000","last_unstale":"2024-04-25T10:34:43.961535+0000","last_undegraded":"2024-04-25T10:34:43.961535+0000","last_fullsized":"2024-04-25T10:34:43.961535+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:39.062490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'2","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:31:13.676443+0000","last_change":"2024-04-25T10:30:31.420258+0000","last_active":"2024-04-25T10:31:13.676443+0000","last_peered":"2024-04-25T10:31:13.676443+0000","last_clean":"2024-04-25T10:31:13.676443+0000","last_became_active":"2024-04-25T10:30:31.419902+0000","last_became_peered":"2024-04-25T10:30:31.419902+0000","last_unstale":"2024-04-25T10:31:13.676443+0000","last_undegraded":"2024-04-25T10:31:13.676443+0000","last_fullsized":"2024-04-25T10:31:13.676443+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:43.919867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"22'146","reported_seq":163,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.963108+0000","last_change":"2024-04-25T10:30:32.433568+0000","last_active":"2024-04-25T10:34:43.963108+0000","last_peered":"2024-04-25T10:34:43.963108+0000","last_clean":"2024-04-25T10:34:43.963108+0000","last_became_active":"2024-04-25T10:30:32.432572+0000","last_became_peered":"2024-04-25T10:30:32.432572+0000","last_unstale":"2024-04-25T10:34:43.963108+0000","last_undegraded":"2024-04-25T10:34:43.963108+0000","last_fullsized":"2024-04-25T10:34:43.963108+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:31.771919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'5","reported_seq":24,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:31:13.636179+0000","last_change":"2024-04-25T10:30:31.419308+0000","last_active":"2024-04-25T10:31:13.636179+0000","last_peered":"2024-04-25T10:31:13.636179+0000","last_clean":"2024-04-25T10:31:13.636179+0000","last_became_active":"2024-04-25T10:30:31.418978+0000","last_became_peered":"2024-04-25T10:30:31.418978+0000","last_unstale":"2024-04-25T10:31:13.636179+0000","last_undegraded":"2024-04-25T10:31:13.636179+0000","last_fullsized":"2024-04-25T10:31:13.636179+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:33.784804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"22'138","reported_seq":155,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.966851+0000","last_change":"2024-04-25T10:30:32.436109+0000","last_active":"2024-04-25T10:34:43.966851+0000","last_peered":"2024-04-25T10:34:43.966851+0000","last_clean":"2024-04-25T10:34:43.966851+0000","last_became_active":"2024-04-25T10:30:32.434701+0000","last_became_peered":"2024-04-25T10:30:32.434701+0000","last_unstale":"2024-04-25T10:34:43.966851+0000","last_undegraded":"2024-04-25T10:34:43.966851+0000","last_fullsized":"2024-04-25T10:34:43.966851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:19.471867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'3","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:31:13.636578+0000","last_change":"2024-04-25T10:30:31.416044+0000","last_active":"2024-04-25T10:31:13.636578+0000","last_peered":"2024-04-25T10:31:13.636578+0000","last_clean":"2024-04-25T10:31:13.636578+0000","last_became_active":"2024-04-25T10:30:31.415707+0000","last_became_peered":"2024-04-25T10:30:31.415707+0000","last_unstale":"2024-04-25T10:31:13.636578+0000","last_undegraded":"2024-04-25T10:31:13.636578+0000","last_fullsized":"2024-04-25T10:31:13.636578+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:31.424135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"22'144","reported_seq":161,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.958006+0000","last_change":"2024-04-25T10:30:32.442971+0000","last_active":"2024-04-25T10:34:43.958006+0000","last_peered":"2024-04-25T10:34:43.958006+0000","last_clean":"2024-04-25T10:34:43.958006+0000","last_became_active":"2024-04-25T10:30:32.442656+0000","last_became_peered":"2024-04-25T10:30:32.442656+0000","last_unstale":"2024-04-25T10:34:43.958006+0000","last_undegraded":"2024-04-25T10:34:43.958006+0000","last_fullsized":"2024-04-25T10:34:43.958006+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:29.951142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"22'7","reported_seq":26,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:32:50.131436+0000","last_change":"2024-04-25T10:30:31.404793+0000","last_active":"2024-04-25T10:32:50.131436+0000","last_peered":"2024-04-25T10:32:50.131436+0000","last_clean":"2024-04-25T10:32:50.131436+0000","last_became_active":"2024-04-25T10:30:31.404392+0000","last_became_peered":"2024-04-25T10:30:31.404392+0000","last_unstale":"2024-04-25T10:32:50.131436+0000","last_undegraded":"2024-04-25T10:32:50.131436+0000","last_fullsized":"2024-04-25T10:32:50.131436+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:00.965450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1752,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"22'151","reported_seq":168,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.967451+0000","last_change":"2024-04-25T10:30:32.441438+0000","last_active":"2024-04-25T10:34:43.967451+0000","last_peered":"2024-04-25T10:34:43.967451+0000","last_clean":"2024-04-25T10:34:43.967451+0000","last_became_active":"2024-04-25T10:30:32.440011+0000","last_became_peered":"2024-04-25T10:30:32.440011+0000","last_unstale":"2024-04-25T10:34:43.967451+0000","last_undegraded":"2024-04-25T10:34:43.967451+0000","last_fullsized":"2024-04-25T10:34:43.967451+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:09.127517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":116,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:32:25.132929+0000","last_change":"2024-04-25T10:30:21.429098+0000","last_active":"2024-04-25T10:32:25.132929+0000","last_peered":"2024-04-25T10:32:25.132929+0000","last_clean":"2024-04-25T10:32:25.132929+0000","last_became_active":"2024-04-25T10:30:21.427958+0000","last_became_peered":"2024-04-25T10:30:21.427958+0000","last_unstale":"2024-04-25T10:32:25.132929+0000","last_undegraded":"2024-04-25T10:32:25.132929+0000","last_fullsized":"2024-04-25T10:32:25.132929+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:20.186206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:20.186206+0000","last_clean_scrub_stamp":"2024-04-25T10:30:20.186206+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:46.046488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"20'6","reported_seq":26,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:32:40.133596+0000","last_change":"2024-04-25T10:30:31.404974+0000","last_active":"2024-04-25T10:32:40.133596+0000","last_peered":"2024-04-25T10:32:40.133596+0000","last_clean":"2024-04-25T10:32:40.133596+0000","last_became_active":"2024-04-25T10:30:31.404549+0000","last_became_peered":"2024-04-25T10:30:31.404549+0000","last_unstale":"2024-04-25T10:32:40.133596+0000","last_undegraded":"2024-04-25T10:32:40.133596+0000","last_fullsized":"2024-04-25T10:32:40.133596+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:06.191321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"22'158","reported_seq":176,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.960745+0000","last_change":"2024-04-25T10:30:32.434409+0000","last_active":"2024-04-25T10:34:43.960745+0000","last_peered":"2024-04-25T10:34:43.960745+0000","last_clean":"2024-04-25T10:34:43.960745+0000","last_became_active":"2024-04-25T10:30:32.433830+0000","last_became_peered":"2024-04-25T10:30:32.433830+0000","last_unstale":"2024-04-25T10:34:43.960745+0000","last_undegraded":"2024-04-25T10:34:43.960745+0000","last_fullsized":"2024-04-25T10:34:43.960745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:57.864627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":409600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"20'4","reported_seq":24,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:32:35.134131+0000","last_change":"2024-04-25T10:30:31.416025+0000","last_active":"2024-04-25T10:32:35.134131+0000","last_peered":"2024-04-25T10:32:35.134131+0000","last_clean":"2024-04-25T10:32:35.134131+0000","last_became_active":"2024-04-25T10:30:31.415675+0000","last_became_peered":"2024-04-25T10:30:31.415675+0000","last_unstale":"2024-04-25T10:32:35.134131+0000","last_undegraded":"2024-04-25T10:32:35.134131+0000","last_fullsized":"2024-04-25T10:32:35.134131+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:28:48.185715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"22'131","reported_seq":149,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.933599+0000","last_change":"2024-04-25T10:30:32.424630+0000","last_active":"2024-04-25T10:34:43.933599+0000","last_peered":"2024-04-25T10:34:43.933599+0000","last_clean":"2024-04-25T10:34:43.933599+0000","last_became_active":"2024-04-25T10:30:32.424303+0000","last_became_peered":"2024-04-25T10:30:32.424303+0000","last_unstale":"2024-04-25T10:34:43.933599+0000","last_undegraded":"2024-04-25T10:34:43.933599+0000","last_fullsized":"2024-04-25T10:34:43.933599+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:55.535319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"22'13","reported_seq":38,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.901417+0000","last_change":"2024-04-25T10:30:31.419145+0000","last_active":"2024-04-25T10:34:43.901417+0000","last_peered":"2024-04-25T10:34:43.901417+0000","last_clean":"2024-04-25T10:34:43.901417+0000","last_became_active":"2024-04-25T10:30:31.418817+0000","last_became_peered":"2024-04-25T10:30:31.418817+0000","last_unstale":"2024-04-25T10:34:43.901417+0000","last_undegraded":"2024-04-25T10:34:43.901417+0000","last_fullsized":"2024-04-25T10:34:43.901417+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:01.608205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":12,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"22'131","reported_seq":149,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.929954+0000","last_change":"2024-04-25T10:30:32.419911+0000","last_active":"2024-04-25T10:34:43.929954+0000","last_peered":"2024-04-25T10:34:43.929954+0000","last_clean":"2024-04-25T10:34:43.929954+0000","last_became_active":"2024-04-25T10:30:32.419486+0000","last_became_peered":"2024-04-25T10:30:32.419486+0000","last_unstale":"2024-04-25T10:34:43.929954+0000","last_undegraded":"2024-04-25T10:34:43.929954+0000","last_fullsized":"2024-04-25T10:34:43.929954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:54.967873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"20'5","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:32:40.133576+0000","last_change":"2024-04-25T10:30:31.405646+0000","last_active":"2024-04-25T10:32:40.133576+0000","last_peered":"2024-04-25T10:32:40.133576+0000","last_clean":"2024-04-25T10:32:40.133576+0000","last_became_active":"2024-04-25T10:30:31.405238+0000","last_became_peered":"2024-04-25T10:30:31.405238+0000","last_unstale":"2024-04-25T10:32:40.133576+0000","last_undegraded":"2024-04-25T10:32:40.133576+0000","last_fullsized":"2024-04-25T10:32:40.133576+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:13.159787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"22'145","reported_seq":163,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.936816+0000","last_change":"2024-04-25T10:30:32.426130+0000","last_active":"2024-04-25T10:34:43.936816+0000","last_peered":"2024-04-25T10:34:43.936816+0000","last_clean":"2024-04-25T10:34:43.936816+0000","last_became_active":"2024-04-25T10:30:32.425783+0000","last_became_peered":"2024-04-25T10:30:32.425783+0000","last_unstale":"2024-04-25T10:34:43.936816+0000","last_undegraded":"2024-04-25T10:34:43.936816+0000","last_fullsized":"2024-04-25T10:34:43.936816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:30.844640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"20'1","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:32:40.131788+0000","last_change":"2024-04-25T10:30:31.412736+0000","last_active":"2024-04-25T10:32:40.131788+0000","last_peered":"2024-04-25T10:32:40.131788+0000","last_clean":"2024-04-25T10:32:40.131788+0000","last_became_active":"2024-04-25T10:30:31.412413+0000","last_became_peered":"2024-04-25T10:30:31.412413+0000","last_unstale":"2024-04-25T10:32:40.131788+0000","last_undegraded":"2024-04-25T10:32:40.131788+0000","last_fullsized":"2024-04-25T10:32:40.131788+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:19:56.236327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"22'143","reported_seq":166,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.951783+0000","last_change":"2024-04-25T10:31:13.640815+0000","last_active":"2024-04-25T10:34:43.951783+0000","last_peered":"2024-04-25T10:34:43.951783+0000","last_clean":"2024-04-25T10:34:43.951783+0000","last_became_active":"2024-04-25T10:31:13.640368+0000","last_became_peered":"2024-04-25T10:31:13.640368+0000","last_unstale":"2024-04-25T10:34:43.951783+0000","last_undegraded":"2024-04-25T10:34:43.951783+0000","last_fullsized":"2024-04-25T10:34:43.951783+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:15.845263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"20'4","reported_seq":24,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:32:40.134663+0000","last_change":"2024-04-25T10:30:31.417958+0000","last_active":"2024-04-25T10:32:40.134663+0000","last_peered":"2024-04-25T10:32:40.134663+0000","last_clean":"2024-04-25T10:32:40.134663+0000","last_became_active":"2024-04-25T10:30:31.417541+0000","last_became_peered":"2024-04-25T10:30:31.417541+0000","last_unstale":"2024-04-25T10:32:40.134663+0000","last_undegraded":"2024-04-25T10:32:40.134663+0000","last_fullsized":"2024-04-25T10:32:40.134663+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:35.373351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"22'144","reported_seq":162,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.937182+0000","last_change":"2024-04-25T10:30:32.420451+0000","last_active":"2024-04-25T10:34:43.937182+0000","last_peered":"2024-04-25T10:34:43.937182+0000","last_clean":"2024-04-25T10:34:43.937182+0000","last_became_active":"2024-04-25T10:30:32.420109+0000","last_became_peered":"2024-04-25T10:30:32.420109+0000","last_unstale":"2024-04-25T10:34:43.937182+0000","last_undegraded":"2024-04-25T10:34:43.937182+0000","last_fullsized":"2024-04-25T10:34:43.937182+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:22.067828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'23","reported_seq":42,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:28.491838+0000","last_change":"2024-04-25T10:30:31.406005+0000","last_active":"2024-04-25T10:34:28.491838+0000","last_peered":"2024-04-25T10:34:28.491838+0000","last_clean":"2024-04-25T10:34:28.491838+0000","last_became_active":"2024-04-25T10:30:31.404739+0000","last_became_peered":"2024-04-25T10:30:31.404739+0000","last_unstale":"2024-04-25T10:34:28.491838+0000","last_undegraded":"2024-04-25T10:34:28.491838+0000","last_fullsized":"2024-04-25T10:34:28.491838+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:31.771123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"22'143","reported_seq":160,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.955595+0000","last_change":"2024-04-25T10:30:32.437513+0000","last_active":"2024-04-25T10:34:43.955595+0000","last_peered":"2024-04-25T10:34:43.955595+0000","last_clean":"2024-04-25T10:34:43.955595+0000","last_became_active":"2024-04-25T10:30:32.437171+0000","last_became_peered":"2024-04-25T10:30:32.437171+0000","last_unstale":"2024-04-25T10:34:43.955595+0000","last_undegraded":"2024-04-25T10:34:43.955595+0000","last_fullsized":"2024-04-25T10:34:43.955595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:50.924843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"22'8","reported_seq":27,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:32:45.134201+0000","last_change":"2024-04-25T10:30:31.413140+0000","last_active":"2024-04-25T10:32:45.134201+0000","last_peered":"2024-04-25T10:32:45.134201+0000","last_clean":"2024-04-25T10:32:45.134201+0000","last_became_active":"2024-04-25T10:30:31.412821+0000","last_became_peered":"2024-04-25T10:30:31.412821+0000","last_unstale":"2024-04-25T10:32:45.134201+0000","last_undegraded":"2024-04-25T10:32:45.134201+0000","last_fullsized":"2024-04-25T10:32:45.134201+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:37.177254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"22'163","reported_seq":180,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.936828+0000","last_change":"2024-04-25T10:30:32.434333+0000","last_active":"2024-04-25T10:34:43.936828+0000","last_peered":"2024-04-25T10:34:43.936828+0000","last_clean":"2024-04-25T10:34:43.936828+0000","last_became_active":"2024-04-25T10:30:32.434000+0000","last_became_peered":"2024-04-25T10:30:32.434000+0000","last_unstale":"2024-04-25T10:34:43.936828+0000","last_undegraded":"2024-04-25T10:34:43.936828+0000","last_fullsized":"2024-04-25T10:34:43.936828+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:54.057457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"20'2","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:31:13.675909+0000","last_change":"2024-04-25T10:30:31.419475+0000","last_active":"2024-04-25T10:31:13.675909+0000","last_peered":"2024-04-25T10:31:13.675909+0000","last_clean":"2024-04-25T10:31:13.675909+0000","last_became_active":"2024-04-25T10:30:31.417605+0000","last_became_peered":"2024-04-25T10:30:31.417605+0000","last_unstale":"2024-04-25T10:31:13.675909+0000","last_undegraded":"2024-04-25T10:31:13.675909+0000","last_fullsized":"2024-04-25T10:31:13.675909+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:00.948269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"22'134","reported_seq":151,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.911130+0000","last_change":"2024-04-25T10:30:32.419693+0000","last_active":"2024-04-25T10:34:43.911130+0000","last_peered":"2024-04-25T10:34:43.911130+0000","last_clean":"2024-04-25T10:34:43.911130+0000","last_became_active":"2024-04-25T10:30:32.419370+0000","last_became_peered":"2024-04-25T10:30:32.419370+0000","last_unstale":"2024-04-25T10:34:43.911130+0000","last_undegraded":"2024-04-25T10:34:43.911130+0000","last_fullsized":"2024-04-25T10:34:43.911130+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:31.199065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"20'7","reported_seq":26,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:31:13.635773+0000","last_change":"2024-04-25T10:30:31.410767+0000","last_active":"2024-04-25T10:31:13.635773+0000","last_peered":"2024-04-25T10:31:13.635773+0000","last_clean":"2024-04-25T10:31:13.635773+0000","last_became_active":"2024-04-25T10:30:31.410404+0000","last_became_peered":"2024-04-25T10:30:31.410404+0000","last_unstale":"2024-04-25T10:31:13.635773+0000","last_undegraded":"2024-04-25T10:31:13.635773+0000","last_fullsized":"2024-04-25T10:31:13.635773+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:37.431532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"22'153","reported_seq":170,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.938119+0000","last_change":"2024-04-25T10:30:32.433687+0000","last_active":"2024-04-25T10:34:43.938119+0000","last_peered":"2024-04-25T10:34:43.938119+0000","last_clean":"2024-04-25T10:34:43.938119+0000","last_became_active":"2024-04-25T10:30:32.433389+0000","last_became_peered":"2024-04-25T10:30:32.433389+0000","last_unstale":"2024-04-25T10:34:43.938119+0000","last_undegraded":"2024-04-25T10:34:43.938119+0000","last_fullsized":"2024-04-25T10:34:43.938119+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:39.155317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"20'3","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:31:13.636188+0000","last_change":"2024-04-25T10:30:31.419118+0000","last_active":"2024-04-25T10:31:13.636188+0000","last_peered":"2024-04-25T10:31:13.636188+0000","last_clean":"2024-04-25T10:31:13.636188+0000","last_became_active":"2024-04-25T10:30:31.418822+0000","last_became_peered":"2024-04-25T10:30:31.418822+0000","last_unstale":"2024-04-25T10:31:13.636188+0000","last_undegraded":"2024-04-25T10:31:13.636188+0000","last_fullsized":"2024-04-25T10:31:13.636188+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:22.755042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"22'143","reported_seq":160,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.938567+0000","last_change":"2024-04-25T10:30:32.427894+0000","last_active":"2024-04-25T10:34:43.938567+0000","last_peered":"2024-04-25T10:34:43.938567+0000","last_clean":"2024-04-25T10:34:43.938567+0000","last_became_active":"2024-04-25T10:30:32.427504+0000","last_became_peered":"2024-04-25T10:30:32.427504+0000","last_unstale":"2024-04-25T10:34:43.938567+0000","last_undegraded":"2024-04-25T10:34:43.938567+0000","last_fullsized":"2024-04-25T10:34:43.938567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:53.877484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'5","reported_seq":24,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:32:50.135474+0000","last_change":"2024-04-25T10:30:31.417115+0000","last_active":"2024-04-25T10:32:50.135474+0000","last_peered":"2024-04-25T10:32:50.135474+0000","last_clean":"2024-04-25T10:32:50.135474+0000","last_became_active":"2024-04-25T10:30:31.416766+0000","last_became_peered":"2024-04-25T10:30:31.416766+0000","last_unstale":"2024-04-25T10:32:50.135474+0000","last_undegraded":"2024-04-25T10:32:50.135474+0000","last_fullsized":"2024-04-25T10:32:50.135474+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:06.021270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"22'128","reported_seq":145,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.962450+0000","last_change":"2024-04-25T10:30:32.431674+0000","last_active":"2024-04-25T10:34:43.962450+0000","last_peered":"2024-04-25T10:34:43.962450+0000","last_clean":"2024-04-25T10:34:43.962450+0000","last_became_active":"2024-04-25T10:30:32.431395+0000","last_became_peered":"2024-04-25T10:30:32.431395+0000","last_unstale":"2024-04-25T10:34:43.962450+0000","last_undegraded":"2024-04-25T10:34:43.962450+0000","last_fullsized":"2024-04-25T10:34:43.962450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:49.726323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"22'183","reported_seq":200,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.949651+0000","last_change":"2024-04-25T10:30:32.437700+0000","last_active":"2024-04-25T10:34:43.949651+0000","last_peered":"2024-04-25T10:34:43.949651+0000","last_clean":"2024-04-25T10:34:43.949651+0000","last_became_active":"2024-04-25T10:30:32.437242+0000","last_became_peered":"2024-04-25T10:30:32.437242+0000","last_unstale":"2024-04-25T10:34:43.949651+0000","last_undegraded":"2024-04-25T10:34:43.949651+0000","last_fullsized":"2024-04-25T10:34:43.949651+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:06.499414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":168,"num_write_kb":458752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'5","reported_seq":24,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:31:13.675847+0000","last_change":"2024-04-25T10:30:31.419678+0000","last_active":"2024-04-25T10:31:13.675847+0000","last_peered":"2024-04-25T10:31:13.675847+0000","last_clean":"2024-04-25T10:31:13.675847+0000","last_became_active":"2024-04-25T10:30:31.419269+0000","last_became_peered":"2024-04-25T10:30:31.419269+0000","last_unstale":"2024-04-25T10:31:13.675847+0000","last_undegraded":"2024-04-25T10:31:13.675847+0000","last_fullsized":"2024-04-25T10:31:13.675847+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:33.034260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"22'141","reported_seq":158,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.966109+0000","last_change":"2024-04-25T10:30:32.434410+0000","last_active":"2024-04-25T10:34:43.966109+0000","last_peered":"2024-04-25T10:34:43.966109+0000","last_clean":"2024-04-25T10:34:43.966109+0000","last_became_active":"2024-04-25T10:30:32.434093+0000","last_became_peered":"2024-04-25T10:30:32.434093+0000","last_unstale":"2024-04-25T10:34:43.966109+0000","last_undegraded":"2024-04-25T10:34:43.966109+0000","last_fullsized":"2024-04-25T10:34:43.966109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:46.964704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'13","reported_seq":32,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:33:05.136991+0000","last_change":"2024-04-25T10:30:31.418360+0000","last_active":"2024-04-25T10:33:05.136991+0000","last_peered":"2024-04-25T10:33:05.136991+0000","last_clean":"2024-04-25T10:33:05.136991+0000","last_became_active":"2024-04-25T10:30:31.418005+0000","last_became_peered":"2024-04-25T10:30:31.418005+0000","last_unstale":"2024-04-25T10:33:05.136991+0000","last_undegraded":"2024-04-25T10:33:05.136991+0000","last_fullsized":"2024-04-25T10:33:05.136991+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:15.556859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"22'135","reported_seq":153,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.953049+0000","last_change":"2024-04-25T10:30:32.439515+0000","last_active":"2024-04-25T10:34:43.953049+0000","last_peered":"2024-04-25T10:34:43.953049+0000","last_clean":"2024-04-25T10:34:43.953049+0000","last_became_active":"2024-04-25T10:30:32.439101+0000","last_became_peered":"2024-04-25T10:30:32.439101+0000","last_unstale":"2024-04-25T10:34:43.953049+0000","last_undegraded":"2024-04-25T10:34:43.953049+0000","last_fullsized":"2024-04-25T10:34:43.953049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:33.797903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'2","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:32:35.131979+0000","last_change":"2024-04-25T10:30:31.418070+0000","last_active":"2024-04-25T10:32:35.131979+0000","last_peered":"2024-04-25T10:32:35.131979+0000","last_clean":"2024-04-25T10:32:35.131979+0000","last_became_active":"2024-04-25T10:30:31.417737+0000","last_became_peered":"2024-04-25T10:30:31.417737+0000","last_unstale":"2024-04-25T10:32:35.131979+0000","last_undegraded":"2024-04-25T10:32:35.131979+0000","last_fullsized":"2024-04-25T10:32:35.131979+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:05.664855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"22'152","reported_seq":170,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.960186+0000","last_change":"2024-04-25T10:30:32.435258+0000","last_active":"2024-04-25T10:34:43.960186+0000","last_peered":"2024-04-25T10:34:43.960186+0000","last_clean":"2024-04-25T10:34:43.960186+0000","last_became_active":"2024-04-25T10:30:32.434930+0000","last_became_peered":"2024-04-25T10:30:32.434930+0000","last_unstale":"2024-04-25T10:34:43.960186+0000","last_undegraded":"2024-04-25T10:34:43.960186+0000","last_fullsized":"2024-04-25T10:34:43.960186+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:22.727778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'13","reported_seq":33,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:32:40.130599+0000","last_change":"2024-04-25T10:30:31.415611+0000","last_active":"2024-04-25T10:32:40.130599+0000","last_peered":"2024-04-25T10:32:40.130599+0000","last_clean":"2024-04-25T10:32:40.130599+0000","last_became_active":"2024-04-25T10:30:31.415071+0000","last_became_peered":"2024-04-25T10:30:31.415071+0000","last_unstale":"2024-04-25T10:32:40.130599+0000","last_undegraded":"2024-04-25T10:32:40.130599+0000","last_fullsized":"2024-04-25T10:32:40.130599+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:30:54.077864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9216,"num_write_kb":25165824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9717,"ondisk_log_size":9717,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":210463,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":345,"num_write_kb":324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":671744,"data_stored":420926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":308,"ondisk_log_size":308,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574906,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40336,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331504,"statfs":{"total":96636764160,"available":96595460096,"internally_reserved":0,"allocated":860160,"data_stored":676182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574906,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35784,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336056,"statfs":{"total":96636764160,"available":96600121344,"internally_reserved":0,"allocated":458752,"data_stored":286294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574907,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39768,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94332072,"statfs":{"total":96636764160,"available":96596041728,"internally_reserved":0,"allocated":278528,"data_stored":87646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574907,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39744,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94332096,"statfs":{"total":96636764160,"available":96596066304,"internally_reserved":0,"allocated":253952,"data_stored":85044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574906,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39952,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331888,"statfs":{"total":96636764160,"available":96595853312,"internally_reserved":0,"allocated":466944,"data_stored":286833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574906,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35596,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336244,"statfs":{"total":96636764160,"available":96600313856,"internally_reserved":0,"allocated":266240,"data_stored":85681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574906,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35588,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336252,"statfs":{"total":96636764160,"available":96600322048,"internally_reserved":0,"allocated":258048,"data_stored":84664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574906,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40328,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331512,"statfs":{"total":96636764160,"available":96595468288,"internally_reserved":0,"allocated":851968,"data_stored":674374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":237568,"data_stored":203701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":229376,"data_stored":203162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:34:50.591 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:34:50.736 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.734+0000 7fa69e900640 1 Processor -- start 2024-04-25T10:34:50.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.734+0000 7fa69e900640 1 -- start start 2024-04-25T10:34:50.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.735+0000 7fa69e900640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fa698156f70 0x7fa69815ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:50.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.736+0000 7fa69e900640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa69815d340 0x7fa698167680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:50.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.736+0000 7fa69e900640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa698073980 con 0x7fa6980ca170 2024-04-25T10:34:50.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.736+0000 7fa69e900640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fa698073f00 con 0x7fa698156f70 2024-04-25T10:34:50.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.736+0000 7fa69e900640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa698074990 con 0x7fa69815d340 2024-04-25T10:34:50.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.736+0000 7fa6977fe640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fa698156f70 0x7fa69815ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:50.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.736+0000 7fa69ce9d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa69815d340 0x7fa698167680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:50.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.736+0000 7fa697fff640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fa6980ca170 0x7fa69806c5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.106:48586/0 (socket says 172.21.15.106:48586) 2024-04-25T10:34:50.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.736+0000 7fa697fff640 1 -- 172.21.15.106:0/540531070 learned_addr learned my addr 172.21.15.106:0/540531070 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:34:50.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.736+0000 7fa69ce9d640 1 -- 172.21.15.106:0/540531070 >> v2:172.21.15.134:3300/0 conn(0x7fa698156f70 msgr2=0x7fa69815ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:50.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.736+0000 7fa69ce9d640 1 --2- 172.21.15.106:0/540531070 >> v2:172.21.15.134:3300/0 conn(0x7fa698156f70 0x7fa69815ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:50.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.736+0000 7fa69ce9d640 1 -- 172.21.15.106:0/540531070 >> v1:172.21.15.134:6789/0 conn(0x7fa6980ca170 legacy=0x7fa69806c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:50.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.736+0000 7fa69ce9d640 1 -- 172.21.15.106:0/540531070 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa698160ea0 con 0x7fa69815d340 2024-04-25T10:34:50.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.737+0000 7fa69ce9d640 1 --2- 172.21.15.106:0/540531070 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa69815d340 0x7fa698167680 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa698074b10 tx=0x7fa69002a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=749c27819d5c6fca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:50.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.737+0000 7fa696ffd640 1 -- 172.21.15.106:0/540531070 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa690007740 con 0x7fa69815d340 2024-04-25T10:34:50.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.737+0000 7fa696ffd640 1 -- 172.21.15.106:0/540531070 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6900078e0 con 0x7fa69815d340 2024-04-25T10:34:50.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.737+0000 7fa696ffd640 1 -- 172.21.15.106:0/540531070 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa690007cb0 con 0x7fa69815d340 2024-04-25T10:34:50.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.738+0000 7fa69e900640 1 -- 172.21.15.106:0/540531070 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa69815d340 msgr2=0x7fa698167680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:50.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.738+0000 7fa69e900640 1 --2- 172.21.15.106:0/540531070 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa69815d340 0x7fa698167680 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa698074b10 tx=0x7fa69002a6d0 comp rx=0 tx=0).stop 2024-04-25T10:34:50.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.738+0000 7fa69e900640 1 -- 172.21.15.106:0/540531070 shutdown_connections 2024-04-25T10:34:50.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.738+0000 7fa69e900640 1 --2- 172.21.15.106:0/540531070 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa69815d340 0x7fa698167680 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:50.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.738+0000 7fa69e900640 1 --2- 172.21.15.106:0/540531070 >> v2:172.21.15.134:3300/0 conn(0x7fa698156f70 0x7fa69815ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:50.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.738+0000 7fa69e900640 1 -- 172.21.15.106:0/540531070 >> 172.21.15.106:0/540531070 conn(0x7fa6980cef20 msgr2=0x7fa6980cf330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:50.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.738+0000 7fa69e900640 1 -- 172.21.15.106:0/540531070 shutdown_connections 2024-04-25T10:34:50.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.738+0000 7fa69e900640 1 -- 172.21.15.106:0/540531070 wait complete. 2024-04-25T10:34:50.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.740+0000 7fa69e900640 1 Processor -- start 2024-04-25T10:34:50.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.740+0000 7fa69e900640 1 -- start start 2024-04-25T10:34:50.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.741+0000 7fa69e900640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa6980ca170 0x7fa6981e6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:50.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.741+0000 7fa69e900640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa698156f70 0x7fa6981f3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:50.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.741+0000 7fa69e900640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa69815d340 0x7fa6981577d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:50.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.741+0000 7fa69e900640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa698073c20 con 0x7fa6980ca170 2024-04-25T10:34:50.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.741+0000 7fa69e900640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa698074640 con 0x7fa698156f70 2024-04-25T10:34:50.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.741+0000 7fa69e900640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa698161140 con 0x7fa69815d340 2024-04-25T10:34:50.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.741+0000 7fa697fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa6980ca170 0x7fa6981e6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:50.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.741+0000 7fa697fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa6980ca170 0x7fa6981e6dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56004/0 (socket says 172.21.15.106:56004) 2024-04-25T10:34:50.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.741+0000 7fa697fff640 1 -- 172.21.15.106:0/3180223301 learned_addr learned my addr 172.21.15.106:0/3180223301 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:34:50.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.741+0000 7fa69ce9d640 1 --2- 172.21.15.106:0/3180223301 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa69815d340 0x7fa6981577d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:50.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.741+0000 7fa6977fe640 1 --2- 172.21.15.106:0/3180223301 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa698156f70 0x7fa6981f3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:50.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.742+0000 7fa697fff640 1 -- 172.21.15.106:0/3180223301 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa69815d340 msgr2=0x7fa6981577d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:50.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.742+0000 7fa697fff640 1 --2- 172.21.15.106:0/3180223301 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa69815d340 0x7fa6981577d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:50.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.742+0000 7fa697fff640 1 -- 172.21.15.106:0/3180223301 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa698156f70 msgr2=0x7fa6981f3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:50.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.742+0000 7fa697fff640 1 --2- 172.21.15.106:0/3180223301 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa698156f70 0x7fa6981f3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:50.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.742+0000 7fa697fff640 1 -- 172.21.15.106:0/3180223301 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6981f3bc0 con 0x7fa6980ca170 2024-04-25T10:34:50.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.742+0000 7fa697fff640 1 --2- 172.21.15.106:0/3180223301 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa6980ca170 0x7fa6981e6dc0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa68800c3f0 tx=0x7fa68800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:50.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.742+0000 7fa694ff9640 1 -- 172.21.15.106:0/3180223301 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa688017070 con 0x7fa6980ca170 2024-04-25T10:34:50.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.742+0000 7fa694ff9640 1 -- 172.21.15.106:0/3180223301 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa688005ce0 con 0x7fa6980ca170 2024-04-25T10:34:50.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.742+0000 7fa69e900640 1 -- 172.21.15.106:0/3180223301 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa69815cef0 con 0x7fa6980ca170 2024-04-25T10:34:50.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.742+0000 7fa694ff9640 1 -- 172.21.15.106:0/3180223301 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6880072f0 con 0x7fa6980ca170 2024-04-25T10:34:50.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.742+0000 7fa69e900640 1 -- 172.21.15.106:0/3180223301 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6981e9f70 con 0x7fa6980ca170 2024-04-25T10:34:50.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.743+0000 7fa69e900640 1 -- 172.21.15.106:0/3180223301 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6981f3d90 con 0x7fa6980ca170 2024-04-25T10:34:50.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.744+0000 7fa694ff9640 1 -- 172.21.15.106:0/3180223301 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa68801a070 con 0x7fa6980ca170 2024-04-25T10:34:50.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.744+0000 7fa694ff9640 1 --2- 172.21.15.106:0/3180223301 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fa6840744e0 0x7fa6840769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:50.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.744+0000 7fa694ff9640 1 -- 172.21.15.106:0/3180223301 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fa688094480 con 0x7fa6980ca170 2024-04-25T10:34:50.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.744+0000 7fa6977fe640 1 --2- 172.21.15.106:0/3180223301 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fa6840744e0 0x7fa6840769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:50.750 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.745+0000 7fa6977fe640 1 --2- 172.21.15.106:0/3180223301 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fa6840744e0 0x7fa6840769f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa6981f0a60 tx=0x7fa68c009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:50.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.749+0000 7fa694ff9640 1 -- 172.21.15.106:0/3180223301 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa68805e550 con 0x7fa6980ca170 2024-04-25T10:34:50.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.918+0000 7fa69e900640 1 -- 172.21.15.106:0/3180223301 --> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa6981e9a40 con 0x7fa6840744e0 2024-04-25T10:34:50.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.924+0000 7fa694ff9640 1 -- 172.21.15.106:0/3180223301 <== mgr.4107 v2:172.21.15.106:6832/1744210167 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252928 (secure 0 0 0) 0x7fa6981e9a40 con 0x7fa6840744e0 2024-04-25T10:34:50.929 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:34:50.930 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-25T10:34:50.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.927+0000 7fa69e900640 1 -- 172.21.15.106:0/3180223301 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fa6840744e0 msgr2=0x7fa6840769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:50.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.927+0000 7fa69e900640 1 --2- 172.21.15.106:0/3180223301 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fa6840744e0 0x7fa6840769f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa6981f0a60 tx=0x7fa68c009190 comp rx=0 tx=0).stop 2024-04-25T10:34:50.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.927+0000 7fa69e900640 1 -- 172.21.15.106:0/3180223301 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa6980ca170 msgr2=0x7fa6981e6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:50.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.927+0000 7fa69e900640 1 --2- 172.21.15.106:0/3180223301 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa6980ca170 0x7fa6981e6dc0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa68800c3f0 tx=0x7fa68800c8b0 comp rx=0 tx=0).stop 2024-04-25T10:34:50.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.927+0000 7fa69e900640 1 -- 172.21.15.106:0/3180223301 shutdown_connections 2024-04-25T10:34:50.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.927+0000 7fa69e900640 1 --2- 172.21.15.106:0/3180223301 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fa6840744e0 0x7fa6840769f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:50.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.927+0000 7fa69e900640 1 --2- 172.21.15.106:0/3180223301 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa69815d340 0x7fa6981577d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:50.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.927+0000 7fa69e900640 1 --2- 172.21.15.106:0/3180223301 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa698156f70 0x7fa6981f3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:50.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.927+0000 7fa69e900640 1 --2- 172.21.15.106:0/3180223301 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa6980ca170 0x7fa6981e6dc0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:50.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.927+0000 7fa69e900640 1 -- 172.21.15.106:0/3180223301 >> 172.21.15.106:0/3180223301 conn(0x7fa6980cef20 msgr2=0x7fa69815fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:50.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.928+0000 7fa69e900640 1 -- 172.21.15.106:0/3180223301 shutdown_connections 2024-04-25T10:34:50.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:50.928+0000 7fa69e900640 1 -- 172.21.15.106:0/3180223301 wait complete. 2024-04-25T10:34:50.941 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":150,"stamp":"2024-04-25T10:34:50.160934+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":800831,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":71,"num_read_kb":60,"num_write":9662,"num_write_kb":25167776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10097,"ondisk_log_size":10097,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":307096,"kb_used_data":3608,"kb_used_omap":12,"kb_used_meta":303411,"kb_avail":754667624,"statfs":{"total":773094113280,"available":772779646976,"internally_reserved":0,"allocated":3694592,"data_stored":2266718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":310693424},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":672,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002917"},"pg_stats":[{"pgid":"3.3f","version":"22'146","reported_seq":164,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.965117+0000","last_change":"2024-04-25T10:30:32.440983+0000","last_active":"2024-04-25T10:34:43.965117+0000","last_peered":"2024-04-25T10:34:43.965117+0000","last_clean":"2024-04-25T10:34:43.965117+0000","last_became_active":"2024-04-25T10:30:32.440552+0000","last_became_peered":"2024-04-25T10:30:32.440552+0000","last_unstale":"2024-04-25T10:34:43.965117+0000","last_undegraded":"2024-04-25T10:34:43.965117+0000","last_fullsized":"2024-04-25T10:34:43.965117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:47.963061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"22'130","reported_seq":148,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.937860+0000","last_change":"2024-04-25T10:30:32.425695+0000","last_active":"2024-04-25T10:34:43.937860+0000","last_peered":"2024-04-25T10:34:43.937860+0000","last_clean":"2024-04-25T10:34:43.937860+0000","last_became_active":"2024-04-25T10:30:32.425368+0000","last_became_peered":"2024-04-25T10:30:32.425368+0000","last_unstale":"2024-04-25T10:34:43.937860+0000","last_undegraded":"2024-04-25T10:34:43.937860+0000","last_fullsized":"2024-04-25T10:34:43.937860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:39.120522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"22'151","reported_seq":169,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.929471+0000","last_change":"2024-04-25T10:30:32.430144+0000","last_active":"2024-04-25T10:34:43.929471+0000","last_peered":"2024-04-25T10:34:43.929471+0000","last_clean":"2024-04-25T10:34:43.929471+0000","last_became_active":"2024-04-25T10:30:32.429754+0000","last_became_peered":"2024-04-25T10:30:32.429754+0000","last_unstale":"2024-04-25T10:34:43.929471+0000","last_undegraded":"2024-04-25T10:34:43.929471+0000","last_fullsized":"2024-04-25T10:34:43.929471+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:11.448907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"22'173","reported_seq":191,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.964702+0000","last_change":"2024-04-25T10:30:32.442478+0000","last_active":"2024-04-25T10:34:43.964702+0000","last_peered":"2024-04-25T10:34:43.964702+0000","last_clean":"2024-04-25T10:34:43.964702+0000","last_became_active":"2024-04-25T10:30:32.442182+0000","last_became_peered":"2024-04-25T10:30:32.442182+0000","last_unstale":"2024-04-25T10:34:43.964702+0000","last_undegraded":"2024-04-25T10:34:43.964702+0000","last_fullsized":"2024-04-25T10:34:43.964702+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":173,"log_dups_size":0,"ondisk_log_size":173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:22.185978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":450560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"22'148","reported_seq":165,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.934476+0000","last_change":"2024-04-25T10:30:32.438231+0000","last_active":"2024-04-25T10:34:43.934476+0000","last_peered":"2024-04-25T10:34:43.934476+0000","last_clean":"2024-04-25T10:34:43.934476+0000","last_became_active":"2024-04-25T10:30:32.437656+0000","last_became_peered":"2024-04-25T10:30:32.437656+0000","last_unstale":"2024-04-25T10:34:43.934476+0000","last_undegraded":"2024-04-25T10:34:43.934476+0000","last_fullsized":"2024-04-25T10:34:43.934476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:56.915999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"22'135","reported_seq":146,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.925882+0000","last_change":"2024-04-25T10:31:13.676329+0000","last_active":"2024-04-25T10:34:43.925882+0000","last_peered":"2024-04-25T10:34:43.925882+0000","last_clean":"2024-04-25T10:34:43.925882+0000","last_became_active":"2024-04-25T10:31:13.676013+0000","last_became_peered":"2024-04-25T10:31:13.676013+0000","last_unstale":"2024-04-25T10:34:43.925882+0000","last_undegraded":"2024-04-25T10:34:43.925882+0000","last_fullsized":"2024-04-25T10:34:43.925882+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:07:15.646068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"22'164","reported_seq":181,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.941596+0000","last_change":"2024-04-25T10:30:32.429453+0000","last_active":"2024-04-25T10:34:43.941596+0000","last_peered":"2024-04-25T10:34:43.941596+0000","last_clean":"2024-04-25T10:34:43.941596+0000","last_became_active":"2024-04-25T10:30:32.429131+0000","last_became_peered":"2024-04-25T10:30:32.429131+0000","last_unstale":"2024-04-25T10:34:43.941596+0000","last_undegraded":"2024-04-25T10:34:43.941596+0000","last_fullsized":"2024-04-25T10:34:43.941596+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:26.785922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159,"num_write_kb":434176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"22'186","reported_seq":203,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.963510+0000","last_change":"2024-04-25T10:30:32.438860+0000","last_active":"2024-04-25T10:34:43.963510+0000","last_peered":"2024-04-25T10:34:43.963510+0000","last_clean":"2024-04-25T10:34:43.963510+0000","last_became_active":"2024-04-25T10:30:32.438491+0000","last_became_peered":"2024-04-25T10:30:32.438491+0000","last_unstale":"2024-04-25T10:34:43.963510+0000","last_undegraded":"2024-04-25T10:34:43.963510+0000","last_fullsized":"2024-04-25T10:34:43.963510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:08.610707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":177,"num_write_kb":483328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"22'152","reported_seq":169,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.967681+0000","last_change":"2024-04-25T10:30:32.440331+0000","last_active":"2024-04-25T10:34:43.967681+0000","last_peered":"2024-04-25T10:34:43.967681+0000","last_clean":"2024-04-25T10:34:43.967681+0000","last_became_active":"2024-04-25T10:30:32.440043+0000","last_became_peered":"2024-04-25T10:30:32.440043+0000","last_unstale":"2024-04-25T10:34:43.967681+0000","last_undegraded":"2024-04-25T10:34:43.967681+0000","last_fullsized":"2024-04-25T10:34:43.967681+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:17.684685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"22'156","reported_seq":173,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.968251+0000","last_change":"2024-04-25T10:30:32.435388+0000","last_active":"2024-04-25T10:34:43.968251+0000","last_peered":"2024-04-25T10:34:43.968251+0000","last_clean":"2024-04-25T10:34:43.968251+0000","last_became_active":"2024-04-25T10:30:32.434983+0000","last_became_peered":"2024-04-25T10:30:32.434983+0000","last_unstale":"2024-04-25T10:34:43.968251+0000","last_undegraded":"2024-04-25T10:34:43.968251+0000","last_fullsized":"2024-04-25T10:34:43.968251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:39.256604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"22'189","reported_seq":211,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.953182+0000","last_change":"2024-04-25T10:31:13.640959+0000","last_active":"2024-04-25T10:34:43.953182+0000","last_peered":"2024-04-25T10:34:43.953182+0000","last_clean":"2024-04-25T10:34:43.953182+0000","last_became_active":"2024-04-25T10:31:13.640582+0000","last_became_peered":"2024-04-25T10:31:13.640582+0000","last_unstale":"2024-04-25T10:34:43.953182+0000","last_undegraded":"2024-04-25T10:34:43.953182+0000","last_fullsized":"2024-04-25T10:34:43.953182+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:30.907171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"22'142","reported_seq":159,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.955837+0000","last_change":"2024-04-25T10:30:32.436403+0000","last_active":"2024-04-25T10:34:43.955837+0000","last_peered":"2024-04-25T10:34:43.955837+0000","last_clean":"2024-04-25T10:34:43.955837+0000","last_became_active":"2024-04-25T10:30:32.435908+0000","last_became_peered":"2024-04-25T10:30:32.435908+0000","last_unstale":"2024-04-25T10:34:43.955837+0000","last_undegraded":"2024-04-25T10:34:43.955837+0000","last_fullsized":"2024-04-25T10:34:43.955837+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:03.339316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"22'151","reported_seq":168,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.951346+0000","last_change":"2024-04-25T10:30:32.432110+0000","last_active":"2024-04-25T10:34:43.951346+0000","last_peered":"2024-04-25T10:34:43.951346+0000","last_clean":"2024-04-25T10:34:43.951346+0000","last_became_active":"2024-04-25T10:30:32.431792+0000","last_became_peered":"2024-04-25T10:30:32.431792+0000","last_unstale":"2024-04-25T10:34:43.951346+0000","last_undegraded":"2024-04-25T10:34:43.951346+0000","last_fullsized":"2024-04-25T10:34:43.951346+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:34.185989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"22'149","reported_seq":166,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.940258+0000","last_change":"2024-04-25T10:30:32.433676+0000","last_active":"2024-04-25T10:34:43.940258+0000","last_peered":"2024-04-25T10:34:43.940258+0000","last_clean":"2024-04-25T10:34:43.940258+0000","last_became_active":"2024-04-25T10:30:32.433329+0000","last_became_peered":"2024-04-25T10:30:32.433329+0000","last_unstale":"2024-04-25T10:34:43.940258+0000","last_undegraded":"2024-04-25T10:34:43.940258+0000","last_fullsized":"2024-04-25T10:34:43.940258+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:59.942368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"22'151","reported_seq":168,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.962728+0000","last_change":"2024-04-25T10:30:32.423582+0000","last_active":"2024-04-25T10:34:43.962728+0000","last_peered":"2024-04-25T10:34:43.962728+0000","last_clean":"2024-04-25T10:34:43.962728+0000","last_became_active":"2024-04-25T10:30:32.422991+0000","last_became_peered":"2024-04-25T10:30:32.422991+0000","last_unstale":"2024-04-25T10:34:43.962728+0000","last_undegraded":"2024-04-25T10:34:43.962728+0000","last_fullsized":"2024-04-25T10:34:43.962728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:11:47.132928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"22'147","reported_seq":164,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.965990+0000","last_change":"2024-04-25T10:30:32.440235+0000","last_active":"2024-04-25T10:34:43.965990+0000","last_peered":"2024-04-25T10:34:43.965990+0000","last_clean":"2024-04-25T10:34:43.965990+0000","last_became_active":"2024-04-25T10:30:32.439191+0000","last_became_peered":"2024-04-25T10:30:32.439191+0000","last_unstale":"2024-04-25T10:34:43.965990+0000","last_undegraded":"2024-04-25T10:34:43.965990+0000","last_fullsized":"2024-04-25T10:34:43.965990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":147,"log_dups_size":0,"ondisk_log_size":147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:51.423086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"22'146","reported_seq":163,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.959761+0000","last_change":"2024-04-25T10:30:32.442418+0000","last_active":"2024-04-25T10:34:43.959761+0000","last_peered":"2024-04-25T10:34:43.959761+0000","last_clean":"2024-04-25T10:34:43.959761+0000","last_became_active":"2024-04-25T10:30:32.442125+0000","last_became_peered":"2024-04-25T10:30:32.442125+0000","last_unstale":"2024-04-25T10:34:43.959761+0000","last_undegraded":"2024-04-25T10:34:43.959761+0000","last_fullsized":"2024-04-25T10:34:43.959761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:14.213809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"22'157","reported_seq":174,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.925082+0000","last_change":"2024-04-25T10:30:32.434758+0000","last_active":"2024-04-25T10:34:43.925082+0000","last_peered":"2024-04-25T10:34:43.925082+0000","last_clean":"2024-04-25T10:34:43.925082+0000","last_became_active":"2024-04-25T10:30:32.434492+0000","last_became_peered":"2024-04-25T10:30:32.434492+0000","last_unstale":"2024-04-25T10:34:43.925082+0000","last_undegraded":"2024-04-25T10:34:43.925082+0000","last_fullsized":"2024-04-25T10:34:43.925082+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:41.454407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"22'172","reported_seq":189,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.948403+0000","last_change":"2024-04-25T10:30:32.440399+0000","last_active":"2024-04-25T10:34:43.948403+0000","last_peered":"2024-04-25T10:34:43.948403+0000","last_clean":"2024-04-25T10:34:43.948403+0000","last_became_active":"2024-04-25T10:30:32.438227+0000","last_became_peered":"2024-04-25T10:30:32.438227+0000","last_unstale":"2024-04-25T10:34:43.948403+0000","last_undegraded":"2024-04-25T10:34:43.948403+0000","last_fullsized":"2024-04-25T10:34:43.948403+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:49.571671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":450560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"22'161","reported_seq":178,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.963147+0000","last_change":"2024-04-25T10:30:32.431609+0000","last_active":"2024-04-25T10:34:43.963147+0000","last_peered":"2024-04-25T10:34:43.963147+0000","last_clean":"2024-04-25T10:34:43.963147+0000","last_became_active":"2024-04-25T10:30:32.431314+0000","last_became_peered":"2024-04-25T10:30:32.431314+0000","last_unstale":"2024-04-25T10:34:43.963147+0000","last_undegraded":"2024-04-25T10:34:43.963147+0000","last_fullsized":"2024-04-25T10:34:43.963147+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:33.421989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":153,"num_write_kb":417792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"22'137","reported_seq":148,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.956088+0000","last_change":"2024-04-25T10:31:13.677426+0000","last_active":"2024-04-25T10:34:43.956088+0000","last_peered":"2024-04-25T10:34:43.956088+0000","last_clean":"2024-04-25T10:34:43.956088+0000","last_became_active":"2024-04-25T10:31:13.677164+0000","last_became_peered":"2024-04-25T10:31:13.677164+0000","last_unstale":"2024-04-25T10:34:43.956088+0000","last_undegraded":"2024-04-25T10:34:43.956088+0000","last_fullsized":"2024-04-25T10:34:43.956088+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:35.850650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2a","version":"22'166","reported_seq":183,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.968866+0000","last_change":"2024-04-25T10:30:32.444618+0000","last_active":"2024-04-25T10:34:43.968866+0000","last_peered":"2024-04-25T10:34:43.968866+0000","last_clean":"2024-04-25T10:34:43.968866+0000","last_became_active":"2024-04-25T10:30:32.444273+0000","last_became_peered":"2024-04-25T10:30:32.444273+0000","last_unstale":"2024-04-25T10:34:43.968866+0000","last_undegraded":"2024-04-25T10:34:43.968866+0000","last_fullsized":"2024-04-25T10:34:43.968866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:09.295251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"22'134","reported_seq":151,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.960474+0000","last_change":"2024-04-25T10:30:32.426578+0000","last_active":"2024-04-25T10:34:43.960474+0000","last_peered":"2024-04-25T10:34:43.960474+0000","last_clean":"2024-04-25T10:34:43.960474+0000","last_became_active":"2024-04-25T10:30:32.426223+0000","last_became_peered":"2024-04-25T10:30:32.426223+0000","last_unstale":"2024-04-25T10:34:43.960474+0000","last_undegraded":"2024-04-25T10:34:43.960474+0000","last_fullsized":"2024-04-25T10:34:43.960474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:53.700585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"22'180","reported_seq":197,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.964675+0000","last_change":"2024-04-25T10:30:32.439767+0000","last_active":"2024-04-25T10:34:43.964675+0000","last_peered":"2024-04-25T10:34:43.964675+0000","last_clean":"2024-04-25T10:34:43.964675+0000","last_became_active":"2024-04-25T10:30:32.438801+0000","last_became_peered":"2024-04-25T10:30:32.438801+0000","last_unstale":"2024-04-25T10:34:43.964675+0000","last_undegraded":"2024-04-25T10:34:43.964675+0000","last_fullsized":"2024-04-25T10:34:43.964675+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:58.231512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":174,"num_write_kb":475136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"22'134","reported_seq":151,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.959780+0000","last_change":"2024-04-25T10:30:32.427424+0000","last_active":"2024-04-25T10:34:43.959780+0000","last_peered":"2024-04-25T10:34:43.959780+0000","last_clean":"2024-04-25T10:34:43.959780+0000","last_became_active":"2024-04-25T10:30:32.427081+0000","last_became_peered":"2024-04-25T10:30:32.427081+0000","last_unstale":"2024-04-25T10:34:43.959780+0000","last_undegraded":"2024-04-25T10:34:43.959780+0000","last_fullsized":"2024-04-25T10:34:43.959780+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:47.372451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"22'198","reported_seq":215,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.968855+0000","last_change":"2024-04-25T10:30:32.432472+0000","last_active":"2024-04-25T10:34:43.968855+0000","last_peered":"2024-04-25T10:34:43.968855+0000","last_clean":"2024-04-25T10:34:43.968855+0000","last_became_active":"2024-04-25T10:30:32.432121+0000","last_became_peered":"2024-04-25T10:30:32.432121+0000","last_unstale":"2024-04-25T10:34:43.968855+0000","last_undegraded":"2024-04-25T10:34:43.968855+0000","last_fullsized":"2024-04-25T10:34:43.968855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":198,"log_dups_size":0,"ondisk_log_size":198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:34.191339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"22'144","reported_seq":161,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.933640+0000","last_change":"2024-04-25T10:30:32.425108+0000","last_active":"2024-04-25T10:34:43.933640+0000","last_peered":"2024-04-25T10:34:43.933640+0000","last_clean":"2024-04-25T10:34:43.933640+0000","last_became_active":"2024-04-25T10:30:32.424794+0000","last_became_peered":"2024-04-25T10:30:32.424794+0000","last_unstale":"2024-04-25T10:34:43.933640+0000","last_undegraded":"2024-04-25T10:34:43.933640+0000","last_fullsized":"2024-04-25T10:34:43.933640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:01.282758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"22'133","reported_seq":150,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.960152+0000","last_change":"2024-04-25T10:30:32.426672+0000","last_active":"2024-04-25T10:34:43.960152+0000","last_peered":"2024-04-25T10:34:43.960152+0000","last_clean":"2024-04-25T10:34:43.960152+0000","last_became_active":"2024-04-25T10:30:32.426348+0000","last_became_peered":"2024-04-25T10:30:32.426348+0000","last_unstale":"2024-04-25T10:34:43.960152+0000","last_undegraded":"2024-04-25T10:34:43.960152+0000","last_fullsized":"2024-04-25T10:34:43.960152+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:57.645772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"22'118","reported_seq":136,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.939753+0000","last_change":"2024-04-25T10:30:32.427989+0000","last_active":"2024-04-25T10:34:43.939753+0000","last_peered":"2024-04-25T10:34:43.939753+0000","last_clean":"2024-04-25T10:34:43.939753+0000","last_became_active":"2024-04-25T10:30:32.427676+0000","last_became_peered":"2024-04-25T10:30:32.427676+0000","last_unstale":"2024-04-25T10:34:43.939753+0000","last_undegraded":"2024-04-25T10:34:43.939753+0000","last_fullsized":"2024-04-25T10:34:43.939753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:15:02.332186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":111,"num_write_kb":303104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"22'131","reported_seq":149,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.960144+0000","last_change":"2024-04-25T10:30:32.441112+0000","last_active":"2024-04-25T10:34:43.960144+0000","last_peered":"2024-04-25T10:34:43.960144+0000","last_clean":"2024-04-25T10:34:43.960144+0000","last_became_active":"2024-04-25T10:30:32.440815+0000","last_became_peered":"2024-04-25T10:30:32.440815+0000","last_unstale":"2024-04-25T10:34:43.960144+0000","last_undegraded":"2024-04-25T10:34:43.960144+0000","last_fullsized":"2024-04-25T10:34:43.960144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:02.956759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"22'135","reported_seq":153,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.938625+0000","last_change":"2024-04-25T10:30:32.430902+0000","last_active":"2024-04-25T10:34:43.938625+0000","last_peered":"2024-04-25T10:34:43.938625+0000","last_clean":"2024-04-25T10:34:43.938625+0000","last_became_active":"2024-04-25T10:30:32.430615+0000","last_became_peered":"2024-04-25T10:30:32.430615+0000","last_unstale":"2024-04-25T10:34:43.938625+0000","last_undegraded":"2024-04-25T10:34:43.938625+0000","last_fullsized":"2024-04-25T10:34:43.938625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:12.747595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"22'150","reported_seq":168,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.959087+0000","last_change":"2024-04-25T10:30:32.435379+0000","last_active":"2024-04-25T10:34:43.959087+0000","last_peered":"2024-04-25T10:34:43.959087+0000","last_clean":"2024-04-25T10:34:43.959087+0000","last_became_active":"2024-04-25T10:30:32.435055+0000","last_became_peered":"2024-04-25T10:30:32.435055+0000","last_unstale":"2024-04-25T10:34:43.959087+0000","last_undegraded":"2024-04-25T10:34:43.959087+0000","last_fullsized":"2024-04-25T10:34:43.959087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:13.623762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"22'130","reported_seq":147,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.959593+0000","last_change":"2024-04-25T10:30:32.422102+0000","last_active":"2024-04-25T10:34:43.959593+0000","last_peered":"2024-04-25T10:34:43.959593+0000","last_clean":"2024-04-25T10:34:43.959593+0000","last_became_active":"2024-04-25T10:30:32.421766+0000","last_became_peered":"2024-04-25T10:30:32.421766+0000","last_unstale":"2024-04-25T10:34:43.959593+0000","last_undegraded":"2024-04-25T10:34:43.959593+0000","last_fullsized":"2024-04-25T10:34:43.959593+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:30.300572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'7","reported_seq":26,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:33:05.134865+0000","last_change":"2024-04-25T10:30:31.418065+0000","last_active":"2024-04-25T10:33:05.134865+0000","last_peered":"2024-04-25T10:33:05.134865+0000","last_clean":"2024-04-25T10:33:05.134865+0000","last_became_active":"2024-04-25T10:30:31.417451+0000","last_became_peered":"2024-04-25T10:30:31.417451+0000","last_unstale":"2024-04-25T10:33:05.134865+0000","last_undegraded":"2024-04-25T10:33:05.134865+0000","last_fullsized":"2024-04-25T10:33:05.134865+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:17.262861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1204,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"22'137","reported_seq":154,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.941470+0000","last_change":"2024-04-25T10:30:32.426913+0000","last_active":"2024-04-25T10:34:43.941470+0000","last_peered":"2024-04-25T10:34:43.941470+0000","last_clean":"2024-04-25T10:34:43.941470+0000","last_became_active":"2024-04-25T10:30:32.426616+0000","last_became_peered":"2024-04-25T10:30:32.426616+0000","last_unstale":"2024-04-25T10:34:43.941470+0000","last_undegraded":"2024-04-25T10:34:43.941470+0000","last_fullsized":"2024-04-25T10:34:43.941470+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:50.614932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'7","reported_seq":26,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:31:13.638951+0000","last_change":"2024-04-25T10:30:31.416432+0000","last_active":"2024-04-25T10:31:13.638951+0000","last_peered":"2024-04-25T10:31:13.638951+0000","last_clean":"2024-04-25T10:31:13.638951+0000","last_became_active":"2024-04-25T10:30:31.415112+0000","last_became_peered":"2024-04-25T10:30:31.415112+0000","last_unstale":"2024-04-25T10:31:13.638951+0000","last_undegraded":"2024-04-25T10:31:13.638951+0000","last_fullsized":"2024-04-25T10:31:13.638951+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:22.674365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"20'3","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:31:13.636650+0000","last_change":"2024-04-25T10:30:31.418059+0000","last_active":"2024-04-25T10:31:13.636650+0000","last_peered":"2024-04-25T10:31:13.636650+0000","last_clean":"2024-04-25T10:31:13.636650+0000","last_became_active":"2024-04-25T10:30:31.417716+0000","last_became_peered":"2024-04-25T10:30:31.417716+0000","last_unstale":"2024-04-25T10:31:13.636650+0000","last_undegraded":"2024-04-25T10:31:13.636650+0000","last_fullsized":"2024-04-25T10:31:13.636650+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:57.752218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"22'170","reported_seq":187,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.938474+0000","last_change":"2024-04-25T10:30:32.428078+0000","last_active":"2024-04-25T10:34:43.938474+0000","last_peered":"2024-04-25T10:34:43.938474+0000","last_clean":"2024-04-25T10:34:43.938474+0000","last_became_active":"2024-04-25T10:30:32.427700+0000","last_became_peered":"2024-04-25T10:30:32.427700+0000","last_unstale":"2024-04-25T10:34:43.938474+0000","last_undegraded":"2024-04-25T10:34:43.938474+0000","last_fullsized":"2024-04-25T10:34:43.938474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:26.931791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":162,"num_write_kb":442368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"20'3","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:31:13.638118+0000","last_change":"2024-04-25T10:30:31.415030+0000","last_active":"2024-04-25T10:31:13.638118+0000","last_peered":"2024-04-25T10:31:13.638118+0000","last_clean":"2024-04-25T10:31:13.638118+0000","last_became_active":"2024-04-25T10:30:31.414542+0000","last_became_peered":"2024-04-25T10:30:31.414542+0000","last_unstale":"2024-04-25T10:31:13.638118+0000","last_undegraded":"2024-04-25T10:31:13.638118+0000","last_fullsized":"2024-04-25T10:31:13.638118+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:20.082420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"22'157","reported_seq":174,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.939532+0000","last_change":"2024-04-25T10:30:32.425207+0000","last_active":"2024-04-25T10:34:43.939532+0000","last_peered":"2024-04-25T10:34:43.939532+0000","last_clean":"2024-04-25T10:34:43.939532+0000","last_became_active":"2024-04-25T10:30:32.424849+0000","last_became_peered":"2024-04-25T10:30:32.424849+0000","last_unstale":"2024-04-25T10:34:43.939532+0000","last_undegraded":"2024-04-25T10:34:43.939532+0000","last_fullsized":"2024-04-25T10:34:43.939532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:11.533686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"22'8","reported_seq":27,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.902339+0000","last_change":"2024-04-25T10:30:31.407700+0000","last_active":"2024-04-25T10:34:43.902339+0000","last_peered":"2024-04-25T10:34:43.902339+0000","last_clean":"2024-04-25T10:34:43.902339+0000","last_became_active":"2024-04-25T10:30:31.407376+0000","last_became_peered":"2024-04-25T10:30:31.407376+0000","last_unstale":"2024-04-25T10:34:43.902339+0000","last_undegraded":"2024-04-25T10:34:43.902339+0000","last_fullsized":"2024-04-25T10:34:43.902339+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:50.112742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"22'229","reported_seq":246,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.960310+0000","last_change":"2024-04-25T10:30:32.444599+0000","last_active":"2024-04-25T10:34:43.960310+0000","last_peered":"2024-04-25T10:34:43.960310+0000","last_clean":"2024-04-25T10:34:43.960310+0000","last_became_active":"2024-04-25T10:30:32.444250+0000","last_became_peered":"2024-04-25T10:30:32.444250+0000","last_unstale":"2024-04-25T10:34:43.960310+0000","last_undegraded":"2024-04-25T10:34:43.960310+0000","last_fullsized":"2024-04-25T10:34:43.960310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":229,"log_dups_size":0,"ondisk_log_size":229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:55.711525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":219,"num_write_kb":598016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"20'4","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:31:13.638996+0000","last_change":"2024-04-25T10:30:31.405051+0000","last_active":"2024-04-25T10:31:13.638996+0000","last_peered":"2024-04-25T10:31:13.638996+0000","last_clean":"2024-04-25T10:31:13.638996+0000","last_became_active":"2024-04-25T10:30:31.404669+0000","last_became_peered":"2024-04-25T10:30:31.404669+0000","last_unstale":"2024-04-25T10:31:13.638996+0000","last_undegraded":"2024-04-25T10:31:13.638996+0000","last_fullsized":"2024-04-25T10:31:13.638996+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:45.260847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"22'141","reported_seq":158,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.953061+0000","last_change":"2024-04-25T10:30:32.435804+0000","last_active":"2024-04-25T10:34:43.953061+0000","last_peered":"2024-04-25T10:34:43.953061+0000","last_clean":"2024-04-25T10:34:43.953061+0000","last_became_active":"2024-04-25T10:30:32.435471+0000","last_became_peered":"2024-04-25T10:30:32.435471+0000","last_unstale":"2024-04-25T10:34:43.953061+0000","last_undegraded":"2024-04-25T10:34:43.953061+0000","last_fullsized":"2024-04-25T10:34:43.953061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:48.213846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"22'11","reported_seq":30,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:27.461358+0000","last_change":"2024-04-25T10:30:31.403560+0000","last_active":"2024-04-25T10:34:27.461358+0000","last_peered":"2024-04-25T10:34:27.461358+0000","last_clean":"2024-04-25T10:34:27.461358+0000","last_became_active":"2024-04-25T10:30:31.403167+0000","last_became_peered":"2024-04-25T10:30:31.403167+0000","last_unstale":"2024-04-25T10:34:27.461358+0000","last_undegraded":"2024-04-25T10:34:27.461358+0000","last_fullsized":"2024-04-25T10:34:27.461358+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:54.592832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"22'149","reported_seq":166,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.967685+0000","last_change":"2024-04-25T10:30:32.424101+0000","last_active":"2024-04-25T10:34:43.967685+0000","last_peered":"2024-04-25T10:34:43.967685+0000","last_clean":"2024-04-25T10:34:43.967685+0000","last_became_active":"2024-04-25T10:30:32.423780+0000","last_became_peered":"2024-04-25T10:30:32.423780+0000","last_unstale":"2024-04-25T10:34:43.967685+0000","last_undegraded":"2024-04-25T10:34:43.967685+0000","last_fullsized":"2024-04-25T10:34:43.967685+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:29.175348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"22'95","reported_seq":114,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.905445+0000","last_change":"2024-04-25T10:30:31.418201+0000","last_active":"2024-04-25T10:34:43.905445+0000","last_peered":"2024-04-25T10:34:43.905445+0000","last_clean":"2024-04-25T10:34:43.905445+0000","last_became_active":"2024-04-25T10:30:31.417824+0000","last_became_peered":"2024-04-25T10:30:31.417824+0000","last_unstale":"2024-04-25T10:34:43.905445+0000","last_undegraded":"2024-04-25T10:34:43.905445+0000","last_fullsized":"2024-04-25T10:34:43.905445+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:38.584225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":201790,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":96,"num_write_kb":233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"22'150","reported_seq":167,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.951616+0000","last_change":"2024-04-25T10:30:32.431139+0000","last_active":"2024-04-25T10:34:43.951616+0000","last_peered":"2024-04-25T10:34:43.951616+0000","last_clean":"2024-04-25T10:34:43.951616+0000","last_became_active":"2024-04-25T10:30:32.429622+0000","last_became_peered":"2024-04-25T10:30:32.429622+0000","last_unstale":"2024-04-25T10:34:43.951616+0000","last_undegraded":"2024-04-25T10:34:43.951616+0000","last_fullsized":"2024-04-25T10:34:43.951616+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:39.993647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'9","reported_seq":28,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:31:13.675426+0000","last_change":"2024-04-25T10:30:31.414775+0000","last_active":"2024-04-25T10:31:13.675426+0000","last_peered":"2024-04-25T10:31:13.675426+0000","last_clean":"2024-04-25T10:31:13.675426+0000","last_became_active":"2024-04-25T10:30:31.414422+0000","last_became_peered":"2024-04-25T10:30:31.414422+0000","last_unstale":"2024-04-25T10:31:13.675426+0000","last_undegraded":"2024-04-25T10:31:13.675426+0000","last_fullsized":"2024-04-25T10:31:13.675426+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:37.803505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"22'157","reported_seq":174,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.944852+0000","last_change":"2024-04-25T10:30:32.432884+0000","last_active":"2024-04-25T10:34:43.944852+0000","last_peered":"2024-04-25T10:34:43.944852+0000","last_clean":"2024-04-25T10:34:43.944852+0000","last_became_active":"2024-04-25T10:30:32.431743+0000","last_became_peered":"2024-04-25T10:30:32.431743+0000","last_unstale":"2024-04-25T10:34:43.944852+0000","last_undegraded":"2024-04-25T10:34:43.944852+0000","last_fullsized":"2024-04-25T10:34:43.944852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:15:06.737631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"20'5","reported_seq":24,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:31:13.638219+0000","last_change":"2024-04-25T10:30:31.419497+0000","last_active":"2024-04-25T10:31:13.638219+0000","last_peered":"2024-04-25T10:31:13.638219+0000","last_clean":"2024-04-25T10:31:13.638219+0000","last_became_active":"2024-04-25T10:30:31.419135+0000","last_became_peered":"2024-04-25T10:30:31.419135+0000","last_unstale":"2024-04-25T10:31:13.638219+0000","last_undegraded":"2024-04-25T10:31:13.638219+0000","last_fullsized":"2024-04-25T10:31:13.638219+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:59.403927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"22'171","reported_seq":188,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.964294+0000","last_change":"2024-04-25T10:30:32.434197+0000","last_active":"2024-04-25T10:34:43.964294+0000","last_peered":"2024-04-25T10:34:43.964294+0000","last_clean":"2024-04-25T10:34:43.964294+0000","last_became_active":"2024-04-25T10:30:32.433623+0000","last_became_peered":"2024-04-25T10:30:32.433623+0000","last_unstale":"2024-04-25T10:34:43.964294+0000","last_undegraded":"2024-04-25T10:34:43.964294+0000","last_fullsized":"2024-04-25T10:34:43.964294+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:08.027472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"22'7","reported_seq":26,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:32:55.135185+0000","last_change":"2024-04-25T10:30:31.416901+0000","last_active":"2024-04-25T10:32:55.135185+0000","last_peered":"2024-04-25T10:32:55.135185+0000","last_clean":"2024-04-25T10:32:55.135185+0000","last_became_active":"2024-04-25T10:30:31.415706+0000","last_became_peered":"2024-04-25T10:30:31.415706+0000","last_unstale":"2024-04-25T10:32:55.135185+0000","last_undegraded":"2024-04-25T10:32:55.135185+0000","last_fullsized":"2024-04-25T10:32:55.135185+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:23.243039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"22'165","reported_seq":182,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.946112+0000","last_change":"2024-04-25T10:30:32.435089+0000","last_active":"2024-04-25T10:34:43.946112+0000","last_peered":"2024-04-25T10:34:43.946112+0000","last_clean":"2024-04-25T10:34:43.946112+0000","last_became_active":"2024-04-25T10:30:32.434737+0000","last_became_peered":"2024-04-25T10:30:32.434737+0000","last_unstale":"2024-04-25T10:34:43.946112+0000","last_undegraded":"2024-04-25T10:34:43.946112+0000","last_fullsized":"2024-04-25T10:34:43.946112+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:19.738850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159,"num_write_kb":434176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"22'18","reported_seq":37,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:32:55.131953+0000","last_change":"2024-04-25T10:30:31.405163+0000","last_active":"2024-04-25T10:32:55.131953+0000","last_peered":"2024-04-25T10:32:55.131953+0000","last_clean":"2024-04-25T10:32:55.131953+0000","last_became_active":"2024-04-25T10:30:31.404775+0000","last_became_peered":"2024-04-25T10:30:31.404775+0000","last_unstale":"2024-04-25T10:32:55.131953+0000","last_undegraded":"2024-04-25T10:32:55.131953+0000","last_fullsized":"2024-04-25T10:32:55.131953+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:10:21.595653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"22'189","reported_seq":206,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.961535+0000","last_change":"2024-04-25T10:30:32.432061+0000","last_active":"2024-04-25T10:34:43.961535+0000","last_peered":"2024-04-25T10:34:43.961535+0000","last_clean":"2024-04-25T10:34:43.961535+0000","last_became_active":"2024-04-25T10:30:32.431651+0000","last_became_peered":"2024-04-25T10:30:32.431651+0000","last_unstale":"2024-04-25T10:34:43.961535+0000","last_undegraded":"2024-04-25T10:34:43.961535+0000","last_fullsized":"2024-04-25T10:34:43.961535+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:39.062490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'2","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:31:13.676443+0000","last_change":"2024-04-25T10:30:31.420258+0000","last_active":"2024-04-25T10:31:13.676443+0000","last_peered":"2024-04-25T10:31:13.676443+0000","last_clean":"2024-04-25T10:31:13.676443+0000","last_became_active":"2024-04-25T10:30:31.419902+0000","last_became_peered":"2024-04-25T10:30:31.419902+0000","last_unstale":"2024-04-25T10:31:13.676443+0000","last_undegraded":"2024-04-25T10:31:13.676443+0000","last_fullsized":"2024-04-25T10:31:13.676443+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:43.919867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"22'146","reported_seq":163,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.963108+0000","last_change":"2024-04-25T10:30:32.433568+0000","last_active":"2024-04-25T10:34:43.963108+0000","last_peered":"2024-04-25T10:34:43.963108+0000","last_clean":"2024-04-25T10:34:43.963108+0000","last_became_active":"2024-04-25T10:30:32.432572+0000","last_became_peered":"2024-04-25T10:30:32.432572+0000","last_unstale":"2024-04-25T10:34:43.963108+0000","last_undegraded":"2024-04-25T10:34:43.963108+0000","last_fullsized":"2024-04-25T10:34:43.963108+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:31.771919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'5","reported_seq":24,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:31:13.636179+0000","last_change":"2024-04-25T10:30:31.419308+0000","last_active":"2024-04-25T10:31:13.636179+0000","last_peered":"2024-04-25T10:31:13.636179+0000","last_clean":"2024-04-25T10:31:13.636179+0000","last_became_active":"2024-04-25T10:30:31.418978+0000","last_became_peered":"2024-04-25T10:30:31.418978+0000","last_unstale":"2024-04-25T10:31:13.636179+0000","last_undegraded":"2024-04-25T10:31:13.636179+0000","last_fullsized":"2024-04-25T10:31:13.636179+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:33.784804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"22'138","reported_seq":155,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.966851+0000","last_change":"2024-04-25T10:30:32.436109+0000","last_active":"2024-04-25T10:34:43.966851+0000","last_peered":"2024-04-25T10:34:43.966851+0000","last_clean":"2024-04-25T10:34:43.966851+0000","last_became_active":"2024-04-25T10:30:32.434701+0000","last_became_peered":"2024-04-25T10:30:32.434701+0000","last_unstale":"2024-04-25T10:34:43.966851+0000","last_undegraded":"2024-04-25T10:34:43.966851+0000","last_fullsized":"2024-04-25T10:34:43.966851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:19.471867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'3","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:31:13.636578+0000","last_change":"2024-04-25T10:30:31.416044+0000","last_active":"2024-04-25T10:31:13.636578+0000","last_peered":"2024-04-25T10:31:13.636578+0000","last_clean":"2024-04-25T10:31:13.636578+0000","last_became_active":"2024-04-25T10:30:31.415707+0000","last_became_peered":"2024-04-25T10:30:31.415707+0000","last_unstale":"2024-04-25T10:31:13.636578+0000","last_undegraded":"2024-04-25T10:31:13.636578+0000","last_fullsized":"2024-04-25T10:31:13.636578+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:31.424135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"22'144","reported_seq":161,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.958006+0000","last_change":"2024-04-25T10:30:32.442971+0000","last_active":"2024-04-25T10:34:43.958006+0000","last_peered":"2024-04-25T10:34:43.958006+0000","last_clean":"2024-04-25T10:34:43.958006+0000","last_became_active":"2024-04-25T10:30:32.442656+0000","last_became_peered":"2024-04-25T10:30:32.442656+0000","last_unstale":"2024-04-25T10:34:43.958006+0000","last_undegraded":"2024-04-25T10:34:43.958006+0000","last_fullsized":"2024-04-25T10:34:43.958006+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:29.951142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"22'7","reported_seq":26,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:32:50.131436+0000","last_change":"2024-04-25T10:30:31.404793+0000","last_active":"2024-04-25T10:32:50.131436+0000","last_peered":"2024-04-25T10:32:50.131436+0000","last_clean":"2024-04-25T10:32:50.131436+0000","last_became_active":"2024-04-25T10:30:31.404392+0000","last_became_peered":"2024-04-25T10:30:31.404392+0000","last_unstale":"2024-04-25T10:32:50.131436+0000","last_undegraded":"2024-04-25T10:32:50.131436+0000","last_fullsized":"2024-04-25T10:32:50.131436+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:00.965450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1752,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"22'151","reported_seq":168,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.967451+0000","last_change":"2024-04-25T10:30:32.441438+0000","last_active":"2024-04-25T10:34:43.967451+0000","last_peered":"2024-04-25T10:34:43.967451+0000","last_clean":"2024-04-25T10:34:43.967451+0000","last_became_active":"2024-04-25T10:30:32.440011+0000","last_became_peered":"2024-04-25T10:30:32.440011+0000","last_unstale":"2024-04-25T10:34:43.967451+0000","last_undegraded":"2024-04-25T10:34:43.967451+0000","last_fullsized":"2024-04-25T10:34:43.967451+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:09.127517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":116,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:32:25.132929+0000","last_change":"2024-04-25T10:30:21.429098+0000","last_active":"2024-04-25T10:32:25.132929+0000","last_peered":"2024-04-25T10:32:25.132929+0000","last_clean":"2024-04-25T10:32:25.132929+0000","last_became_active":"2024-04-25T10:30:21.427958+0000","last_became_peered":"2024-04-25T10:30:21.427958+0000","last_unstale":"2024-04-25T10:32:25.132929+0000","last_undegraded":"2024-04-25T10:32:25.132929+0000","last_fullsized":"2024-04-25T10:32:25.132929+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:20.186206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:20.186206+0000","last_clean_scrub_stamp":"2024-04-25T10:30:20.186206+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:46.046488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"20'6","reported_seq":26,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:32:40.133596+0000","last_change":"2024-04-25T10:30:31.404974+0000","last_active":"2024-04-25T10:32:40.133596+0000","last_peered":"2024-04-25T10:32:40.133596+0000","last_clean":"2024-04-25T10:32:40.133596+0000","last_became_active":"2024-04-25T10:30:31.404549+0000","last_became_peered":"2024-04-25T10:30:31.404549+0000","last_unstale":"2024-04-25T10:32:40.133596+0000","last_undegraded":"2024-04-25T10:32:40.133596+0000","last_fullsized":"2024-04-25T10:32:40.133596+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:06.191321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"22'158","reported_seq":176,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.960745+0000","last_change":"2024-04-25T10:30:32.434409+0000","last_active":"2024-04-25T10:34:43.960745+0000","last_peered":"2024-04-25T10:34:43.960745+0000","last_clean":"2024-04-25T10:34:43.960745+0000","last_became_active":"2024-04-25T10:30:32.433830+0000","last_became_peered":"2024-04-25T10:30:32.433830+0000","last_unstale":"2024-04-25T10:34:43.960745+0000","last_undegraded":"2024-04-25T10:34:43.960745+0000","last_fullsized":"2024-04-25T10:34:43.960745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:57.864627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":409600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"20'4","reported_seq":24,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:32:35.134131+0000","last_change":"2024-04-25T10:30:31.416025+0000","last_active":"2024-04-25T10:32:35.134131+0000","last_peered":"2024-04-25T10:32:35.134131+0000","last_clean":"2024-04-25T10:32:35.134131+0000","last_became_active":"2024-04-25T10:30:31.415675+0000","last_became_peered":"2024-04-25T10:30:31.415675+0000","last_unstale":"2024-04-25T10:32:35.134131+0000","last_undegraded":"2024-04-25T10:32:35.134131+0000","last_fullsized":"2024-04-25T10:32:35.134131+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:28:48.185715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"22'131","reported_seq":149,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.933599+0000","last_change":"2024-04-25T10:30:32.424630+0000","last_active":"2024-04-25T10:34:43.933599+0000","last_peered":"2024-04-25T10:34:43.933599+0000","last_clean":"2024-04-25T10:34:43.933599+0000","last_became_active":"2024-04-25T10:30:32.424303+0000","last_became_peered":"2024-04-25T10:30:32.424303+0000","last_unstale":"2024-04-25T10:34:43.933599+0000","last_undegraded":"2024-04-25T10:34:43.933599+0000","last_fullsized":"2024-04-25T10:34:43.933599+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:55.535319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"22'13","reported_seq":38,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.901417+0000","last_change":"2024-04-25T10:30:31.419145+0000","last_active":"2024-04-25T10:34:43.901417+0000","last_peered":"2024-04-25T10:34:43.901417+0000","last_clean":"2024-04-25T10:34:43.901417+0000","last_became_active":"2024-04-25T10:30:31.418817+0000","last_became_peered":"2024-04-25T10:30:31.418817+0000","last_unstale":"2024-04-25T10:34:43.901417+0000","last_undegraded":"2024-04-25T10:34:43.901417+0000","last_fullsized":"2024-04-25T10:34:43.901417+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:01.608205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":12,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"22'131","reported_seq":149,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.929954+0000","last_change":"2024-04-25T10:30:32.419911+0000","last_active":"2024-04-25T10:34:43.929954+0000","last_peered":"2024-04-25T10:34:43.929954+0000","last_clean":"2024-04-25T10:34:43.929954+0000","last_became_active":"2024-04-25T10:30:32.419486+0000","last_became_peered":"2024-04-25T10:30:32.419486+0000","last_unstale":"2024-04-25T10:34:43.929954+0000","last_undegraded":"2024-04-25T10:34:43.929954+0000","last_fullsized":"2024-04-25T10:34:43.929954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:54.967873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"20'5","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:32:40.133576+0000","last_change":"2024-04-25T10:30:31.405646+0000","last_active":"2024-04-25T10:32:40.133576+0000","last_peered":"2024-04-25T10:32:40.133576+0000","last_clean":"2024-04-25T10:32:40.133576+0000","last_became_active":"2024-04-25T10:30:31.405238+0000","last_became_peered":"2024-04-25T10:30:31.405238+0000","last_unstale":"2024-04-25T10:32:40.133576+0000","last_undegraded":"2024-04-25T10:32:40.133576+0000","last_fullsized":"2024-04-25T10:32:40.133576+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:13.159787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"22'145","reported_seq":163,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.936816+0000","last_change":"2024-04-25T10:30:32.426130+0000","last_active":"2024-04-25T10:34:43.936816+0000","last_peered":"2024-04-25T10:34:43.936816+0000","last_clean":"2024-04-25T10:34:43.936816+0000","last_became_active":"2024-04-25T10:30:32.425783+0000","last_became_peered":"2024-04-25T10:30:32.425783+0000","last_unstale":"2024-04-25T10:34:43.936816+0000","last_undegraded":"2024-04-25T10:34:43.936816+0000","last_fullsized":"2024-04-25T10:34:43.936816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:30.844640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"20'1","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:32:40.131788+0000","last_change":"2024-04-25T10:30:31.412736+0000","last_active":"2024-04-25T10:32:40.131788+0000","last_peered":"2024-04-25T10:32:40.131788+0000","last_clean":"2024-04-25T10:32:40.131788+0000","last_became_active":"2024-04-25T10:30:31.412413+0000","last_became_peered":"2024-04-25T10:30:31.412413+0000","last_unstale":"2024-04-25T10:32:40.131788+0000","last_undegraded":"2024-04-25T10:32:40.131788+0000","last_fullsized":"2024-04-25T10:32:40.131788+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:19:56.236327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"22'143","reported_seq":166,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.951783+0000","last_change":"2024-04-25T10:31:13.640815+0000","last_active":"2024-04-25T10:34:43.951783+0000","last_peered":"2024-04-25T10:34:43.951783+0000","last_clean":"2024-04-25T10:34:43.951783+0000","last_became_active":"2024-04-25T10:31:13.640368+0000","last_became_peered":"2024-04-25T10:31:13.640368+0000","last_unstale":"2024-04-25T10:34:43.951783+0000","last_undegraded":"2024-04-25T10:34:43.951783+0000","last_fullsized":"2024-04-25T10:34:43.951783+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:15.845263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"20'4","reported_seq":24,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:32:40.134663+0000","last_change":"2024-04-25T10:30:31.417958+0000","last_active":"2024-04-25T10:32:40.134663+0000","last_peered":"2024-04-25T10:32:40.134663+0000","last_clean":"2024-04-25T10:32:40.134663+0000","last_became_active":"2024-04-25T10:30:31.417541+0000","last_became_peered":"2024-04-25T10:30:31.417541+0000","last_unstale":"2024-04-25T10:32:40.134663+0000","last_undegraded":"2024-04-25T10:32:40.134663+0000","last_fullsized":"2024-04-25T10:32:40.134663+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:35.373351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"22'144","reported_seq":162,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.937182+0000","last_change":"2024-04-25T10:30:32.420451+0000","last_active":"2024-04-25T10:34:43.937182+0000","last_peered":"2024-04-25T10:34:43.937182+0000","last_clean":"2024-04-25T10:34:43.937182+0000","last_became_active":"2024-04-25T10:30:32.420109+0000","last_became_peered":"2024-04-25T10:30:32.420109+0000","last_unstale":"2024-04-25T10:34:43.937182+0000","last_undegraded":"2024-04-25T10:34:43.937182+0000","last_fullsized":"2024-04-25T10:34:43.937182+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:22.067828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'23","reported_seq":42,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:28.491838+0000","last_change":"2024-04-25T10:30:31.406005+0000","last_active":"2024-04-25T10:34:28.491838+0000","last_peered":"2024-04-25T10:34:28.491838+0000","last_clean":"2024-04-25T10:34:28.491838+0000","last_became_active":"2024-04-25T10:30:31.404739+0000","last_became_peered":"2024-04-25T10:30:31.404739+0000","last_unstale":"2024-04-25T10:34:28.491838+0000","last_undegraded":"2024-04-25T10:34:28.491838+0000","last_fullsized":"2024-04-25T10:34:28.491838+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:31.771123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"22'143","reported_seq":160,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.955595+0000","last_change":"2024-04-25T10:30:32.437513+0000","last_active":"2024-04-25T10:34:43.955595+0000","last_peered":"2024-04-25T10:34:43.955595+0000","last_clean":"2024-04-25T10:34:43.955595+0000","last_became_active":"2024-04-25T10:30:32.437171+0000","last_became_peered":"2024-04-25T10:30:32.437171+0000","last_unstale":"2024-04-25T10:34:43.955595+0000","last_undegraded":"2024-04-25T10:34:43.955595+0000","last_fullsized":"2024-04-25T10:34:43.955595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:50.924843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"22'8","reported_seq":27,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:32:45.134201+0000","last_change":"2024-04-25T10:30:31.413140+0000","last_active":"2024-04-25T10:32:45.134201+0000","last_peered":"2024-04-25T10:32:45.134201+0000","last_clean":"2024-04-25T10:32:45.134201+0000","last_became_active":"2024-04-25T10:30:31.412821+0000","last_became_peered":"2024-04-25T10:30:31.412821+0000","last_unstale":"2024-04-25T10:32:45.134201+0000","last_undegraded":"2024-04-25T10:32:45.134201+0000","last_fullsized":"2024-04-25T10:32:45.134201+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:37.177254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"22'163","reported_seq":180,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.936828+0000","last_change":"2024-04-25T10:30:32.434333+0000","last_active":"2024-04-25T10:34:43.936828+0000","last_peered":"2024-04-25T10:34:43.936828+0000","last_clean":"2024-04-25T10:34:43.936828+0000","last_became_active":"2024-04-25T10:30:32.434000+0000","last_became_peered":"2024-04-25T10:30:32.434000+0000","last_unstale":"2024-04-25T10:34:43.936828+0000","last_undegraded":"2024-04-25T10:34:43.936828+0000","last_fullsized":"2024-04-25T10:34:43.936828+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:54.057457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"20'2","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:31:13.675909+0000","last_change":"2024-04-25T10:30:31.419475+0000","last_active":"2024-04-25T10:31:13.675909+0000","last_peered":"2024-04-25T10:31:13.675909+0000","last_clean":"2024-04-25T10:31:13.675909+0000","last_became_active":"2024-04-25T10:30:31.417605+0000","last_became_peered":"2024-04-25T10:30:31.417605+0000","last_unstale":"2024-04-25T10:31:13.675909+0000","last_undegraded":"2024-04-25T10:31:13.675909+0000","last_fullsized":"2024-04-25T10:31:13.675909+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:00.948269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"22'134","reported_seq":151,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.911130+0000","last_change":"2024-04-25T10:30:32.419693+0000","last_active":"2024-04-25T10:34:43.911130+0000","last_peered":"2024-04-25T10:34:43.911130+0000","last_clean":"2024-04-25T10:34:43.911130+0000","last_became_active":"2024-04-25T10:30:32.419370+0000","last_became_peered":"2024-04-25T10:30:32.419370+0000","last_unstale":"2024-04-25T10:34:43.911130+0000","last_undegraded":"2024-04-25T10:34:43.911130+0000","last_fullsized":"2024-04-25T10:34:43.911130+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:31.199065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"20'7","reported_seq":26,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:31:13.635773+0000","last_change":"2024-04-25T10:30:31.410767+0000","last_active":"2024-04-25T10:31:13.635773+0000","last_peered":"2024-04-25T10:31:13.635773+0000","last_clean":"2024-04-25T10:31:13.635773+0000","last_became_active":"2024-04-25T10:30:31.410404+0000","last_became_peered":"2024-04-25T10:30:31.410404+0000","last_unstale":"2024-04-25T10:31:13.635773+0000","last_undegraded":"2024-04-25T10:31:13.635773+0000","last_fullsized":"2024-04-25T10:31:13.635773+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:37.431532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"22'153","reported_seq":170,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.938119+0000","last_change":"2024-04-25T10:30:32.433687+0000","last_active":"2024-04-25T10:34:43.938119+0000","last_peered":"2024-04-25T10:34:43.938119+0000","last_clean":"2024-04-25T10:34:43.938119+0000","last_became_active":"2024-04-25T10:30:32.433389+0000","last_became_peered":"2024-04-25T10:30:32.433389+0000","last_unstale":"2024-04-25T10:34:43.938119+0000","last_undegraded":"2024-04-25T10:34:43.938119+0000","last_fullsized":"2024-04-25T10:34:43.938119+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:39.155317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"20'3","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:31:13.636188+0000","last_change":"2024-04-25T10:30:31.419118+0000","last_active":"2024-04-25T10:31:13.636188+0000","last_peered":"2024-04-25T10:31:13.636188+0000","last_clean":"2024-04-25T10:31:13.636188+0000","last_became_active":"2024-04-25T10:30:31.418822+0000","last_became_peered":"2024-04-25T10:30:31.418822+0000","last_unstale":"2024-04-25T10:31:13.636188+0000","last_undegraded":"2024-04-25T10:31:13.636188+0000","last_fullsized":"2024-04-25T10:31:13.636188+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:22.755042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"22'143","reported_seq":160,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.938567+0000","last_change":"2024-04-25T10:30:32.427894+0000","last_active":"2024-04-25T10:34:43.938567+0000","last_peered":"2024-04-25T10:34:43.938567+0000","last_clean":"2024-04-25T10:34:43.938567+0000","last_became_active":"2024-04-25T10:30:32.427504+0000","last_became_peered":"2024-04-25T10:30:32.427504+0000","last_unstale":"2024-04-25T10:34:43.938567+0000","last_undegraded":"2024-04-25T10:34:43.938567+0000","last_fullsized":"2024-04-25T10:34:43.938567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:53.877484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'5","reported_seq":24,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:32:50.135474+0000","last_change":"2024-04-25T10:30:31.417115+0000","last_active":"2024-04-25T10:32:50.135474+0000","last_peered":"2024-04-25T10:32:50.135474+0000","last_clean":"2024-04-25T10:32:50.135474+0000","last_became_active":"2024-04-25T10:30:31.416766+0000","last_became_peered":"2024-04-25T10:30:31.416766+0000","last_unstale":"2024-04-25T10:32:50.135474+0000","last_undegraded":"2024-04-25T10:32:50.135474+0000","last_fullsized":"2024-04-25T10:32:50.135474+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:06.021270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"22'128","reported_seq":145,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.962450+0000","last_change":"2024-04-25T10:30:32.431674+0000","last_active":"2024-04-25T10:34:43.962450+0000","last_peered":"2024-04-25T10:34:43.962450+0000","last_clean":"2024-04-25T10:34:43.962450+0000","last_became_active":"2024-04-25T10:30:32.431395+0000","last_became_peered":"2024-04-25T10:30:32.431395+0000","last_unstale":"2024-04-25T10:34:43.962450+0000","last_undegraded":"2024-04-25T10:34:43.962450+0000","last_fullsized":"2024-04-25T10:34:43.962450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:49.726323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"22'183","reported_seq":200,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.949651+0000","last_change":"2024-04-25T10:30:32.437700+0000","last_active":"2024-04-25T10:34:43.949651+0000","last_peered":"2024-04-25T10:34:43.949651+0000","last_clean":"2024-04-25T10:34:43.949651+0000","last_became_active":"2024-04-25T10:30:32.437242+0000","last_became_peered":"2024-04-25T10:30:32.437242+0000","last_unstale":"2024-04-25T10:34:43.949651+0000","last_undegraded":"2024-04-25T10:34:43.949651+0000","last_fullsized":"2024-04-25T10:34:43.949651+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:06.499414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":168,"num_write_kb":458752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'5","reported_seq":24,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:31:13.675847+0000","last_change":"2024-04-25T10:30:31.419678+0000","last_active":"2024-04-25T10:31:13.675847+0000","last_peered":"2024-04-25T10:31:13.675847+0000","last_clean":"2024-04-25T10:31:13.675847+0000","last_became_active":"2024-04-25T10:30:31.419269+0000","last_became_peered":"2024-04-25T10:30:31.419269+0000","last_unstale":"2024-04-25T10:31:13.675847+0000","last_undegraded":"2024-04-25T10:31:13.675847+0000","last_fullsized":"2024-04-25T10:31:13.675847+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:33.034260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"22'141","reported_seq":158,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.966109+0000","last_change":"2024-04-25T10:30:32.434410+0000","last_active":"2024-04-25T10:34:43.966109+0000","last_peered":"2024-04-25T10:34:43.966109+0000","last_clean":"2024-04-25T10:34:43.966109+0000","last_became_active":"2024-04-25T10:30:32.434093+0000","last_became_peered":"2024-04-25T10:30:32.434093+0000","last_unstale":"2024-04-25T10:34:43.966109+0000","last_undegraded":"2024-04-25T10:34:43.966109+0000","last_fullsized":"2024-04-25T10:34:43.966109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:46.964704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'13","reported_seq":32,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:33:05.136991+0000","last_change":"2024-04-25T10:30:31.418360+0000","last_active":"2024-04-25T10:33:05.136991+0000","last_peered":"2024-04-25T10:33:05.136991+0000","last_clean":"2024-04-25T10:33:05.136991+0000","last_became_active":"2024-04-25T10:30:31.418005+0000","last_became_peered":"2024-04-25T10:30:31.418005+0000","last_unstale":"2024-04-25T10:33:05.136991+0000","last_undegraded":"2024-04-25T10:33:05.136991+0000","last_fullsized":"2024-04-25T10:33:05.136991+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:15.556859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"22'135","reported_seq":153,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.953049+0000","last_change":"2024-04-25T10:30:32.439515+0000","last_active":"2024-04-25T10:34:43.953049+0000","last_peered":"2024-04-25T10:34:43.953049+0000","last_clean":"2024-04-25T10:34:43.953049+0000","last_became_active":"2024-04-25T10:30:32.439101+0000","last_became_peered":"2024-04-25T10:30:32.439101+0000","last_unstale":"2024-04-25T10:34:43.953049+0000","last_undegraded":"2024-04-25T10:34:43.953049+0000","last_fullsized":"2024-04-25T10:34:43.953049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:33.797903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'2","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:32:35.131979+0000","last_change":"2024-04-25T10:30:31.418070+0000","last_active":"2024-04-25T10:32:35.131979+0000","last_peered":"2024-04-25T10:32:35.131979+0000","last_clean":"2024-04-25T10:32:35.131979+0000","last_became_active":"2024-04-25T10:30:31.417737+0000","last_became_peered":"2024-04-25T10:30:31.417737+0000","last_unstale":"2024-04-25T10:32:35.131979+0000","last_undegraded":"2024-04-25T10:32:35.131979+0000","last_fullsized":"2024-04-25T10:32:35.131979+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:05.664855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"22'152","reported_seq":170,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.960186+0000","last_change":"2024-04-25T10:30:32.435258+0000","last_active":"2024-04-25T10:34:43.960186+0000","last_peered":"2024-04-25T10:34:43.960186+0000","last_clean":"2024-04-25T10:34:43.960186+0000","last_became_active":"2024-04-25T10:30:32.434930+0000","last_became_peered":"2024-04-25T10:30:32.434930+0000","last_unstale":"2024-04-25T10:34:43.960186+0000","last_undegraded":"2024-04-25T10:34:43.960186+0000","last_fullsized":"2024-04-25T10:34:43.960186+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:22.727778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'13","reported_seq":33,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:32:40.130599+0000","last_change":"2024-04-25T10:30:31.415611+0000","last_active":"2024-04-25T10:32:40.130599+0000","last_peered":"2024-04-25T10:32:40.130599+0000","last_clean":"2024-04-25T10:32:40.130599+0000","last_became_active":"2024-04-25T10:30:31.415071+0000","last_became_peered":"2024-04-25T10:30:31.415071+0000","last_unstale":"2024-04-25T10:32:40.130599+0000","last_undegraded":"2024-04-25T10:32:40.130599+0000","last_fullsized":"2024-04-25T10:32:40.130599+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:30:54.077864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9216,"num_write_kb":25165824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9717,"ondisk_log_size":9717,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":210463,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":345,"num_write_kb":324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":671744,"data_stored":420926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":308,"ondisk_log_size":308,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574906,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40336,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331504,"statfs":{"total":96636764160,"available":96595460096,"internally_reserved":0,"allocated":860160,"data_stored":676182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574906,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35784,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336056,"statfs":{"total":96636764160,"available":96600121344,"internally_reserved":0,"allocated":458752,"data_stored":286294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574907,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39768,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94332072,"statfs":{"total":96636764160,"available":96596041728,"internally_reserved":0,"allocated":278528,"data_stored":87646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574907,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39744,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94332096,"statfs":{"total":96636764160,"available":96596066304,"internally_reserved":0,"allocated":253952,"data_stored":85044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574906,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39952,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331888,"statfs":{"total":96636764160,"available":96595853312,"internally_reserved":0,"allocated":466944,"data_stored":286833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574906,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35596,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336244,"statfs":{"total":96636764160,"available":96600313856,"internally_reserved":0,"allocated":266240,"data_stored":85681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574906,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35588,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336252,"statfs":{"total":96636764160,"available":96600322048,"internally_reserved":0,"allocated":258048,"data_stored":84664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574906,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40328,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331512,"statfs":{"total":96636764160,"available":96595468288,"internally_reserved":0,"allocated":851968,"data_stored":674374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":237568,"data_stored":203701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":229376,"data_stored":203162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:34:50.945 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T10:34:50.945 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:34:51.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.087+0000 7f24154e2640 1 Processor -- start 2024-04-25T10:34:51.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.087+0000 7f24154e2640 1 -- start start 2024-04-25T10:34:51.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.088+0000 7f24154e2640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f24100c8d90 0x7f2410156ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:51.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.088+0000 7f24154e2640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f24100b4ce0 0x7f241015ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:51.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.088+0000 7f24154e2640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24100ce630 con 0x7f24100c7e30 2024-04-25T10:34:51.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.088+0000 7f24154e2640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f24100b50f0 con 0x7f24100b4ce0 2024-04-25T10:34:51.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.088+0000 7f24154e2640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f24100c7c60 con 0x7f24100c8d90 2024-04-25T10:34:51.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.088+0000 7f240effd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f24100c7e30 0x7f241006d000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:39710/0 (socket says 172.21.15.106:39710) 2024-04-25T10:34:51.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.088+0000 7f240effd640 1 -- 172.21.15.106:0/387706631 learned_addr learned my addr 172.21.15.106:0/387706631 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:34:51.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.088+0000 7f240f7fe640 1 --2- 172.21.15.106:0/387706631 >> v2:172.21.15.134:3300/0 conn(0x7f24100b4ce0 0x7f241015ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:51.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.088+0000 7f240e7fc640 1 --2- 172.21.15.106:0/387706631 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f24100c8d90 0x7f2410156ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:51.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.088+0000 7f240dffb640 1 -- 172.21.15.106:0/387706631 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3800864408 0 0) 0x7f24100ce630 con 0x7f24100c7e30 2024-04-25T10:34:51.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.089+0000 7f240dffb640 1 -- 172.21.15.106:0/387706631 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f23fc003520 con 0x7f24100c7e30 2024-04-25T10:34:51.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.089+0000 7f240f7fe640 1 -- 172.21.15.106:0/387706631 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f24100c8d90 msgr2=0x7f2410156ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:51.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.089+0000 7f240dffb640 1 -- 172.21.15.106:0/387706631 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 939223345 0 0) 0x7f23fc003520 con 0x7f24100c7e30 2024-04-25T10:34:51.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.089+0000 7f240f7fe640 1 --2- 172.21.15.106:0/387706631 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f24100c8d90 0x7f2410156ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:51.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.089+0000 7f240f7fe640 1 -- 172.21.15.106:0/387706631 >> v1:172.21.15.106:6789/0 conn(0x7f24100c7e30 legacy=0x7f241006d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:51.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.089+0000 7f240f7fe640 1 -- 172.21.15.106:0/387706631 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24100c7980 con 0x7f24100b4ce0 2024-04-25T10:34:51.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.089+0000 7f240e7fc640 1 --2- 172.21.15.106:0/387706631 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f24100c8d90 0x7f2410156ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:34:51.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.090+0000 7f240f7fe640 1 --2- 172.21.15.106:0/387706631 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24100b4ce0 0x7f241015ce30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f24100c7de0 tx=0x7f240002a080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83b9d1efaa269064 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:51.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.090+0000 7f240dffb640 1 -- 172.21.15.106:0/387706631 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2400037070 con 0x7f24100b4ce0 2024-04-25T10:34:51.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.090+0000 7f240dffb640 1 -- 172.21.15.106:0/387706631 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f240002ab10 con 0x7f24100b4ce0 2024-04-25T10:34:51.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.090+0000 7f240dffb640 1 -- 172.21.15.106:0/387706631 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24080019f0 con 0x7f24100b4ce0 2024-04-25T10:34:51.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.090+0000 7f24154e2640 1 -- 172.21.15.106:0/387706631 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24100b4ce0 msgr2=0x7f241015ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:51.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.090+0000 7f24154e2640 1 --2- 172.21.15.106:0/387706631 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24100b4ce0 0x7f241015ce30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f24100c7de0 tx=0x7f240002a080 comp rx=0 tx=0).stop 2024-04-25T10:34:51.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.090+0000 7f24154e2640 1 -- 172.21.15.106:0/387706631 shutdown_connections 2024-04-25T10:34:51.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.090+0000 7f24154e2640 1 --2- 172.21.15.106:0/387706631 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24100b4ce0 0x7f241015ce30 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:51.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.090+0000 7f24154e2640 1 --2- 172.21.15.106:0/387706631 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f24100c8d90 0x7f2410156ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:51.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.090+0000 7f24154e2640 1 -- 172.21.15.106:0/387706631 >> 172.21.15.106:0/387706631 conn(0x7f2410068b10 msgr2=0x7f24100d1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:51.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.090+0000 7f24154e2640 1 -- 172.21.15.106:0/387706631 shutdown_connections 2024-04-25T10:34:51.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.091+0000 7f24154e2640 1 -- 172.21.15.106:0/387706631 wait complete. 2024-04-25T10:34:51.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.092+0000 7f24154e2640 1 Processor -- start 2024-04-25T10:34:51.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.092+0000 7f24154e2640 1 -- start start 2024-04-25T10:34:51.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.094+0000 7f24154e2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24100b4ce0 0x7f24101f6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:51.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.094+0000 7f24154e2640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f24100c7e30 0x7f2410151a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:51.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.094+0000 7f24154e2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24100c8d90 0x7f241014f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:51.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.094+0000 7f24154e2640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f241006d710 con 0x7f24100b4ce0 2024-04-25T10:34:51.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.094+0000 7f24154e2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f24100c72f0 con 0x7f24100c8d90 2024-04-25T10:34:51.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.094+0000 7f24154e2640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f241006d890 con 0x7f24100c7e30 2024-04-25T10:34:51.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.094+0000 7f240effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24100b4ce0 0x7f24101f6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:51.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.094+0000 7f240effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24100b4ce0 0x7f24101f6940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56020/0 (socket says 172.21.15.106:56020) 2024-04-25T10:34:51.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.094+0000 7f240effd640 1 -- 172.21.15.106:0/1995751375 learned_addr learned my addr 172.21.15.106:0/1995751375 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:34:51.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.094+0000 7f240e7fc640 1 --2- 172.21.15.106:0/1995751375 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f24100c7e30 0x7f2410151a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:51.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.094+0000 7f240f7fe640 1 --2- 172.21.15.106:0/1995751375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24100c8d90 0x7f241014f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:51.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.094+0000 7f240effd640 1 -- 172.21.15.106:0/1995751375 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f24100c7e30 msgr2=0x7f2410151a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:51.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.094+0000 7f240effd640 1 --2- 172.21.15.106:0/1995751375 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f24100c7e30 0x7f2410151a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:51.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.094+0000 7f240effd640 1 -- 172.21.15.106:0/1995751375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24100c8d90 msgr2=0x7f241014f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:51.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.094+0000 7f240effd640 1 --2- 172.21.15.106:0/1995751375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24100c8d90 0x7f241014f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:51.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.094+0000 7f240effd640 1 -- 172.21.15.106:0/1995751375 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24101f6e80 con 0x7f24100b4ce0 2024-04-25T10:34:51.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.094+0000 7f240effd640 1 --2- 172.21.15.106:0/1995751375 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24100b4ce0 0x7f24101f6940 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f240800cc70 tx=0x7f240800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:51.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.094+0000 7f24077fe640 1 -- 172.21.15.106:0/1995751375 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2408017070 con 0x7f24100b4ce0 2024-04-25T10:34:51.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.094+0000 7f24077fe640 1 -- 172.21.15.106:0/1995751375 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2408012400 con 0x7f24100b4ce0 2024-04-25T10:34:51.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.094+0000 7f24154e2640 1 -- 172.21.15.106:0/1995751375 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24101526b0 con 0x7f24100b4ce0 2024-04-25T10:34:51.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.094+0000 7f24077fe640 1 -- 172.21.15.106:0/1995751375 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24080127f0 con 0x7f24100b4ce0 2024-04-25T10:34:51.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.094+0000 7f24154e2640 1 -- 172.21.15.106:0/1995751375 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24101f3720 con 0x7f24100b4ce0 2024-04-25T10:34:51.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.096+0000 7f24154e2640 1 -- 172.21.15.106:0/1995751375 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f241006f650 con 0x7f24100b4ce0 2024-04-25T10:34:51.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.096+0000 7f24077fe640 1 -- 172.21.15.106:0/1995751375 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f240801a070 con 0x7f24100b4ce0 2024-04-25T10:34:51.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.096+0000 7f24077fe640 1 --2- 172.21.15.106:0/1995751375 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f23f00744e0 0x7f23f00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:51.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.097+0000 7f240e7fc640 1 --2- 172.21.15.106:0/1995751375 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f23f00744e0 0x7f23f00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:51.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.097+0000 7f24077fe640 1 -- 172.21.15.106:0/1995751375 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f2408095060 con 0x7f24100b4ce0 2024-04-25T10:34:51.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.097+0000 7f240e7fc640 1 --2- 172.21.15.106:0/1995751375 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f23f00744e0 0x7f23f00769f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2410160fe0 tx=0x7f23f8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:51.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.101+0000 7f24077fe640 1 -- 172.21.15.106:0/1995751375 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f241006f650 con 0x7f24100b4ce0 2024-04-25T10:34:51.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.274+0000 7f24154e2640 1 -- 172.21.15.106:0/1995751375 --> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f241006a3d0 con 0x7f23f00744e0 2024-04-25T10:34:51.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.280+0000 7f24077fe640 1 -- 172.21.15.106:0/1995751375 <== mgr.4107 v2:172.21.15.106:6832/1744210167 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252928 (secure 0 0 0) 0x7f241006a3d0 con 0x7f23f00744e0 2024-04-25T10:34:51.285 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:34:51.286 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-25T10:34:51.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.283+0000 7f24154e2640 1 -- 172.21.15.106:0/1995751375 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f23f00744e0 msgr2=0x7f23f00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:51.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.283+0000 7f24154e2640 1 --2- 172.21.15.106:0/1995751375 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f23f00744e0 0x7f23f00769f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2410160fe0 tx=0x7f23f8008040 comp rx=0 tx=0).stop 2024-04-25T10:34:51.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.283+0000 7f24154e2640 1 -- 172.21.15.106:0/1995751375 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24100b4ce0 msgr2=0x7f24101f6940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:51.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.283+0000 7f24154e2640 1 --2- 172.21.15.106:0/1995751375 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24100b4ce0 0x7f24101f6940 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f240800cc70 tx=0x7f240800a430 comp rx=0 tx=0).stop 2024-04-25T10:34:51.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.283+0000 7f24154e2640 1 -- 172.21.15.106:0/1995751375 shutdown_connections 2024-04-25T10:34:51.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.283+0000 7f24154e2640 1 --2- 172.21.15.106:0/1995751375 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f23f00744e0 0x7f23f00769f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:51.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.283+0000 7f24154e2640 1 --2- 172.21.15.106:0/1995751375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24100c8d90 0x7f241014f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:51.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.283+0000 7f24154e2640 1 --2- 172.21.15.106:0/1995751375 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f24100c7e30 0x7f2410151a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:51.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.283+0000 7f24154e2640 1 --2- 172.21.15.106:0/1995751375 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24100b4ce0 0x7f24101f6940 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:51.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.283+0000 7f24154e2640 1 -- 172.21.15.106:0/1995751375 >> 172.21.15.106:0/1995751375 conn(0x7f2410068b10 msgr2=0x7f24100cd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:51.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.283+0000 7f24154e2640 1 -- 172.21.15.106:0/1995751375 shutdown_connections 2024-04-25T10:34:51.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.283+0000 7f24154e2640 1 -- 172.21.15.106:0/1995751375 wait complete. 2024-04-25T10:34:51.297 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":150,"stamp":"2024-04-25T10:34:50.160934+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":800831,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":71,"num_read_kb":60,"num_write":9662,"num_write_kb":25167776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10097,"ondisk_log_size":10097,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":307096,"kb_used_data":3608,"kb_used_omap":12,"kb_used_meta":303411,"kb_avail":754667624,"statfs":{"total":773094113280,"available":772779646976,"internally_reserved":0,"allocated":3694592,"data_stored":2266718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":310693424},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":672,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002917"},"pg_stats":[{"pgid":"3.3f","version":"22'146","reported_seq":164,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.965117+0000","last_change":"2024-04-25T10:30:32.440983+0000","last_active":"2024-04-25T10:34:43.965117+0000","last_peered":"2024-04-25T10:34:43.965117+0000","last_clean":"2024-04-25T10:34:43.965117+0000","last_became_active":"2024-04-25T10:30:32.440552+0000","last_became_peered":"2024-04-25T10:30:32.440552+0000","last_unstale":"2024-04-25T10:34:43.965117+0000","last_undegraded":"2024-04-25T10:34:43.965117+0000","last_fullsized":"2024-04-25T10:34:43.965117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:47.963061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"22'130","reported_seq":148,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.937860+0000","last_change":"2024-04-25T10:30:32.425695+0000","last_active":"2024-04-25T10:34:43.937860+0000","last_peered":"2024-04-25T10:34:43.937860+0000","last_clean":"2024-04-25T10:34:43.937860+0000","last_became_active":"2024-04-25T10:30:32.425368+0000","last_became_peered":"2024-04-25T10:30:32.425368+0000","last_unstale":"2024-04-25T10:34:43.937860+0000","last_undegraded":"2024-04-25T10:34:43.937860+0000","last_fullsized":"2024-04-25T10:34:43.937860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:39.120522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"22'151","reported_seq":169,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.929471+0000","last_change":"2024-04-25T10:30:32.430144+0000","last_active":"2024-04-25T10:34:43.929471+0000","last_peered":"2024-04-25T10:34:43.929471+0000","last_clean":"2024-04-25T10:34:43.929471+0000","last_became_active":"2024-04-25T10:30:32.429754+0000","last_became_peered":"2024-04-25T10:30:32.429754+0000","last_unstale":"2024-04-25T10:34:43.929471+0000","last_undegraded":"2024-04-25T10:34:43.929471+0000","last_fullsized":"2024-04-25T10:34:43.929471+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:11.448907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"22'173","reported_seq":191,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.964702+0000","last_change":"2024-04-25T10:30:32.442478+0000","last_active":"2024-04-25T10:34:43.964702+0000","last_peered":"2024-04-25T10:34:43.964702+0000","last_clean":"2024-04-25T10:34:43.964702+0000","last_became_active":"2024-04-25T10:30:32.442182+0000","last_became_peered":"2024-04-25T10:30:32.442182+0000","last_unstale":"2024-04-25T10:34:43.964702+0000","last_undegraded":"2024-04-25T10:34:43.964702+0000","last_fullsized":"2024-04-25T10:34:43.964702+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":173,"log_dups_size":0,"ondisk_log_size":173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:22.185978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":450560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"22'148","reported_seq":165,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.934476+0000","last_change":"2024-04-25T10:30:32.438231+0000","last_active":"2024-04-25T10:34:43.934476+0000","last_peered":"2024-04-25T10:34:43.934476+0000","last_clean":"2024-04-25T10:34:43.934476+0000","last_became_active":"2024-04-25T10:30:32.437656+0000","last_became_peered":"2024-04-25T10:30:32.437656+0000","last_unstale":"2024-04-25T10:34:43.934476+0000","last_undegraded":"2024-04-25T10:34:43.934476+0000","last_fullsized":"2024-04-25T10:34:43.934476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:56.915999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"22'135","reported_seq":146,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.925882+0000","last_change":"2024-04-25T10:31:13.676329+0000","last_active":"2024-04-25T10:34:43.925882+0000","last_peered":"2024-04-25T10:34:43.925882+0000","last_clean":"2024-04-25T10:34:43.925882+0000","last_became_active":"2024-04-25T10:31:13.676013+0000","last_became_peered":"2024-04-25T10:31:13.676013+0000","last_unstale":"2024-04-25T10:34:43.925882+0000","last_undegraded":"2024-04-25T10:34:43.925882+0000","last_fullsized":"2024-04-25T10:34:43.925882+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:07:15.646068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"22'164","reported_seq":181,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.941596+0000","last_change":"2024-04-25T10:30:32.429453+0000","last_active":"2024-04-25T10:34:43.941596+0000","last_peered":"2024-04-25T10:34:43.941596+0000","last_clean":"2024-04-25T10:34:43.941596+0000","last_became_active":"2024-04-25T10:30:32.429131+0000","last_became_peered":"2024-04-25T10:30:32.429131+0000","last_unstale":"2024-04-25T10:34:43.941596+0000","last_undegraded":"2024-04-25T10:34:43.941596+0000","last_fullsized":"2024-04-25T10:34:43.941596+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:26.785922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159,"num_write_kb":434176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"22'186","reported_seq":203,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.963510+0000","last_change":"2024-04-25T10:30:32.438860+0000","last_active":"2024-04-25T10:34:43.963510+0000","last_peered":"2024-04-25T10:34:43.963510+0000","last_clean":"2024-04-25T10:34:43.963510+0000","last_became_active":"2024-04-25T10:30:32.438491+0000","last_became_peered":"2024-04-25T10:30:32.438491+0000","last_unstale":"2024-04-25T10:34:43.963510+0000","last_undegraded":"2024-04-25T10:34:43.963510+0000","last_fullsized":"2024-04-25T10:34:43.963510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:08.610707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":177,"num_write_kb":483328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"22'152","reported_seq":169,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.967681+0000","last_change":"2024-04-25T10:30:32.440331+0000","last_active":"2024-04-25T10:34:43.967681+0000","last_peered":"2024-04-25T10:34:43.967681+0000","last_clean":"2024-04-25T10:34:43.967681+0000","last_became_active":"2024-04-25T10:30:32.440043+0000","last_became_peered":"2024-04-25T10:30:32.440043+0000","last_unstale":"2024-04-25T10:34:43.967681+0000","last_undegraded":"2024-04-25T10:34:43.967681+0000","last_fullsized":"2024-04-25T10:34:43.967681+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:17.684685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"22'156","reported_seq":173,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.968251+0000","last_change":"2024-04-25T10:30:32.435388+0000","last_active":"2024-04-25T10:34:43.968251+0000","last_peered":"2024-04-25T10:34:43.968251+0000","last_clean":"2024-04-25T10:34:43.968251+0000","last_became_active":"2024-04-25T10:30:32.434983+0000","last_became_peered":"2024-04-25T10:30:32.434983+0000","last_unstale":"2024-04-25T10:34:43.968251+0000","last_undegraded":"2024-04-25T10:34:43.968251+0000","last_fullsized":"2024-04-25T10:34:43.968251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:39.256604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"22'189","reported_seq":211,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.953182+0000","last_change":"2024-04-25T10:31:13.640959+0000","last_active":"2024-04-25T10:34:43.953182+0000","last_peered":"2024-04-25T10:34:43.953182+0000","last_clean":"2024-04-25T10:34:43.953182+0000","last_became_active":"2024-04-25T10:31:13.640582+0000","last_became_peered":"2024-04-25T10:31:13.640582+0000","last_unstale":"2024-04-25T10:34:43.953182+0000","last_undegraded":"2024-04-25T10:34:43.953182+0000","last_fullsized":"2024-04-25T10:34:43.953182+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:30.907171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"22'142","reported_seq":159,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.955837+0000","last_change":"2024-04-25T10:30:32.436403+0000","last_active":"2024-04-25T10:34:43.955837+0000","last_peered":"2024-04-25T10:34:43.955837+0000","last_clean":"2024-04-25T10:34:43.955837+0000","last_became_active":"2024-04-25T10:30:32.435908+0000","last_became_peered":"2024-04-25T10:30:32.435908+0000","last_unstale":"2024-04-25T10:34:43.955837+0000","last_undegraded":"2024-04-25T10:34:43.955837+0000","last_fullsized":"2024-04-25T10:34:43.955837+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:03.339316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"22'151","reported_seq":168,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.951346+0000","last_change":"2024-04-25T10:30:32.432110+0000","last_active":"2024-04-25T10:34:43.951346+0000","last_peered":"2024-04-25T10:34:43.951346+0000","last_clean":"2024-04-25T10:34:43.951346+0000","last_became_active":"2024-04-25T10:30:32.431792+0000","last_became_peered":"2024-04-25T10:30:32.431792+0000","last_unstale":"2024-04-25T10:34:43.951346+0000","last_undegraded":"2024-04-25T10:34:43.951346+0000","last_fullsized":"2024-04-25T10:34:43.951346+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:34.185989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"22'149","reported_seq":166,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.940258+0000","last_change":"2024-04-25T10:30:32.433676+0000","last_active":"2024-04-25T10:34:43.940258+0000","last_peered":"2024-04-25T10:34:43.940258+0000","last_clean":"2024-04-25T10:34:43.940258+0000","last_became_active":"2024-04-25T10:30:32.433329+0000","last_became_peered":"2024-04-25T10:30:32.433329+0000","last_unstale":"2024-04-25T10:34:43.940258+0000","last_undegraded":"2024-04-25T10:34:43.940258+0000","last_fullsized":"2024-04-25T10:34:43.940258+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:59.942368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"22'151","reported_seq":168,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.962728+0000","last_change":"2024-04-25T10:30:32.423582+0000","last_active":"2024-04-25T10:34:43.962728+0000","last_peered":"2024-04-25T10:34:43.962728+0000","last_clean":"2024-04-25T10:34:43.962728+0000","last_became_active":"2024-04-25T10:30:32.422991+0000","last_became_peered":"2024-04-25T10:30:32.422991+0000","last_unstale":"2024-04-25T10:34:43.962728+0000","last_undegraded":"2024-04-25T10:34:43.962728+0000","last_fullsized":"2024-04-25T10:34:43.962728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:11:47.132928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"22'147","reported_seq":164,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.965990+0000","last_change":"2024-04-25T10:30:32.440235+0000","last_active":"2024-04-25T10:34:43.965990+0000","last_peered":"2024-04-25T10:34:43.965990+0000","last_clean":"2024-04-25T10:34:43.965990+0000","last_became_active":"2024-04-25T10:30:32.439191+0000","last_became_peered":"2024-04-25T10:30:32.439191+0000","last_unstale":"2024-04-25T10:34:43.965990+0000","last_undegraded":"2024-04-25T10:34:43.965990+0000","last_fullsized":"2024-04-25T10:34:43.965990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":147,"log_dups_size":0,"ondisk_log_size":147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:51.423086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"22'146","reported_seq":163,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.959761+0000","last_change":"2024-04-25T10:30:32.442418+0000","last_active":"2024-04-25T10:34:43.959761+0000","last_peered":"2024-04-25T10:34:43.959761+0000","last_clean":"2024-04-25T10:34:43.959761+0000","last_became_active":"2024-04-25T10:30:32.442125+0000","last_became_peered":"2024-04-25T10:30:32.442125+0000","last_unstale":"2024-04-25T10:34:43.959761+0000","last_undegraded":"2024-04-25T10:34:43.959761+0000","last_fullsized":"2024-04-25T10:34:43.959761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:14.213809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"22'157","reported_seq":174,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.925082+0000","last_change":"2024-04-25T10:30:32.434758+0000","last_active":"2024-04-25T10:34:43.925082+0000","last_peered":"2024-04-25T10:34:43.925082+0000","last_clean":"2024-04-25T10:34:43.925082+0000","last_became_active":"2024-04-25T10:30:32.434492+0000","last_became_peered":"2024-04-25T10:30:32.434492+0000","last_unstale":"2024-04-25T10:34:43.925082+0000","last_undegraded":"2024-04-25T10:34:43.925082+0000","last_fullsized":"2024-04-25T10:34:43.925082+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:41.454407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"22'172","reported_seq":189,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.948403+0000","last_change":"2024-04-25T10:30:32.440399+0000","last_active":"2024-04-25T10:34:43.948403+0000","last_peered":"2024-04-25T10:34:43.948403+0000","last_clean":"2024-04-25T10:34:43.948403+0000","last_became_active":"2024-04-25T10:30:32.438227+0000","last_became_peered":"2024-04-25T10:30:32.438227+0000","last_unstale":"2024-04-25T10:34:43.948403+0000","last_undegraded":"2024-04-25T10:34:43.948403+0000","last_fullsized":"2024-04-25T10:34:43.948403+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:49.571671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":450560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"22'161","reported_seq":178,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.963147+0000","last_change":"2024-04-25T10:30:32.431609+0000","last_active":"2024-04-25T10:34:43.963147+0000","last_peered":"2024-04-25T10:34:43.963147+0000","last_clean":"2024-04-25T10:34:43.963147+0000","last_became_active":"2024-04-25T10:30:32.431314+0000","last_became_peered":"2024-04-25T10:30:32.431314+0000","last_unstale":"2024-04-25T10:34:43.963147+0000","last_undegraded":"2024-04-25T10:34:43.963147+0000","last_fullsized":"2024-04-25T10:34:43.963147+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:33.421989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":153,"num_write_kb":417792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"22'137","reported_seq":148,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.956088+0000","last_change":"2024-04-25T10:31:13.677426+0000","last_active":"2024-04-25T10:34:43.956088+0000","last_peered":"2024-04-25T10:34:43.956088+0000","last_clean":"2024-04-25T10:34:43.956088+0000","last_became_active":"2024-04-25T10:31:13.677164+0000","last_became_peered":"2024-04-25T10:31:13.677164+0000","last_unstale":"2024-04-25T10:34:43.956088+0000","last_undegraded":"2024-04-25T10:34:43.956088+0000","last_fullsized":"2024-04-25T10:34:43.956088+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:35.850650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2a","version":"22'166","reported_seq":183,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.968866+0000","last_change":"2024-04-25T10:30:32.444618+0000","last_active":"2024-04-25T10:34:43.968866+0000","last_peered":"2024-04-25T10:34:43.968866+0000","last_clean":"2024-04-25T10:34:43.968866+0000","last_became_active":"2024-04-25T10:30:32.444273+0000","last_became_peered":"2024-04-25T10:30:32.444273+0000","last_unstale":"2024-04-25T10:34:43.968866+0000","last_undegraded":"2024-04-25T10:34:43.968866+0000","last_fullsized":"2024-04-25T10:34:43.968866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:09.295251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"22'134","reported_seq":151,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.960474+0000","last_change":"2024-04-25T10:30:32.426578+0000","last_active":"2024-04-25T10:34:43.960474+0000","last_peered":"2024-04-25T10:34:43.960474+0000","last_clean":"2024-04-25T10:34:43.960474+0000","last_became_active":"2024-04-25T10:30:32.426223+0000","last_became_peered":"2024-04-25T10:30:32.426223+0000","last_unstale":"2024-04-25T10:34:43.960474+0000","last_undegraded":"2024-04-25T10:34:43.960474+0000","last_fullsized":"2024-04-25T10:34:43.960474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:53.700585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"22'180","reported_seq":197,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.964675+0000","last_change":"2024-04-25T10:30:32.439767+0000","last_active":"2024-04-25T10:34:43.964675+0000","last_peered":"2024-04-25T10:34:43.964675+0000","last_clean":"2024-04-25T10:34:43.964675+0000","last_became_active":"2024-04-25T10:30:32.438801+0000","last_became_peered":"2024-04-25T10:30:32.438801+0000","last_unstale":"2024-04-25T10:34:43.964675+0000","last_undegraded":"2024-04-25T10:34:43.964675+0000","last_fullsized":"2024-04-25T10:34:43.964675+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:58.231512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":174,"num_write_kb":475136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"22'134","reported_seq":151,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.959780+0000","last_change":"2024-04-25T10:30:32.427424+0000","last_active":"2024-04-25T10:34:43.959780+0000","last_peered":"2024-04-25T10:34:43.959780+0000","last_clean":"2024-04-25T10:34:43.959780+0000","last_became_active":"2024-04-25T10:30:32.427081+0000","last_became_peered":"2024-04-25T10:30:32.427081+0000","last_unstale":"2024-04-25T10:34:43.959780+0000","last_undegraded":"2024-04-25T10:34:43.959780+0000","last_fullsized":"2024-04-25T10:34:43.959780+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:47.372451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"22'198","reported_seq":215,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.968855+0000","last_change":"2024-04-25T10:30:32.432472+0000","last_active":"2024-04-25T10:34:43.968855+0000","last_peered":"2024-04-25T10:34:43.968855+0000","last_clean":"2024-04-25T10:34:43.968855+0000","last_became_active":"2024-04-25T10:30:32.432121+0000","last_became_peered":"2024-04-25T10:30:32.432121+0000","last_unstale":"2024-04-25T10:34:43.968855+0000","last_undegraded":"2024-04-25T10:34:43.968855+0000","last_fullsized":"2024-04-25T10:34:43.968855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":198,"log_dups_size":0,"ondisk_log_size":198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:34.191339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"22'144","reported_seq":161,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.933640+0000","last_change":"2024-04-25T10:30:32.425108+0000","last_active":"2024-04-25T10:34:43.933640+0000","last_peered":"2024-04-25T10:34:43.933640+0000","last_clean":"2024-04-25T10:34:43.933640+0000","last_became_active":"2024-04-25T10:30:32.424794+0000","last_became_peered":"2024-04-25T10:30:32.424794+0000","last_unstale":"2024-04-25T10:34:43.933640+0000","last_undegraded":"2024-04-25T10:34:43.933640+0000","last_fullsized":"2024-04-25T10:34:43.933640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:01.282758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"22'133","reported_seq":150,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.960152+0000","last_change":"2024-04-25T10:30:32.426672+0000","last_active":"2024-04-25T10:34:43.960152+0000","last_peered":"2024-04-25T10:34:43.960152+0000","last_clean":"2024-04-25T10:34:43.960152+0000","last_became_active":"2024-04-25T10:30:32.426348+0000","last_became_peered":"2024-04-25T10:30:32.426348+0000","last_unstale":"2024-04-25T10:34:43.960152+0000","last_undegraded":"2024-04-25T10:34:43.960152+0000","last_fullsized":"2024-04-25T10:34:43.960152+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:57.645772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"22'118","reported_seq":136,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.939753+0000","last_change":"2024-04-25T10:30:32.427989+0000","last_active":"2024-04-25T10:34:43.939753+0000","last_peered":"2024-04-25T10:34:43.939753+0000","last_clean":"2024-04-25T10:34:43.939753+0000","last_became_active":"2024-04-25T10:30:32.427676+0000","last_became_peered":"2024-04-25T10:30:32.427676+0000","last_unstale":"2024-04-25T10:34:43.939753+0000","last_undegraded":"2024-04-25T10:34:43.939753+0000","last_fullsized":"2024-04-25T10:34:43.939753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:15:02.332186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":111,"num_write_kb":303104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"22'131","reported_seq":149,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.960144+0000","last_change":"2024-04-25T10:30:32.441112+0000","last_active":"2024-04-25T10:34:43.960144+0000","last_peered":"2024-04-25T10:34:43.960144+0000","last_clean":"2024-04-25T10:34:43.960144+0000","last_became_active":"2024-04-25T10:30:32.440815+0000","last_became_peered":"2024-04-25T10:30:32.440815+0000","last_unstale":"2024-04-25T10:34:43.960144+0000","last_undegraded":"2024-04-25T10:34:43.960144+0000","last_fullsized":"2024-04-25T10:34:43.960144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:02.956759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"22'135","reported_seq":153,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.938625+0000","last_change":"2024-04-25T10:30:32.430902+0000","last_active":"2024-04-25T10:34:43.938625+0000","last_peered":"2024-04-25T10:34:43.938625+0000","last_clean":"2024-04-25T10:34:43.938625+0000","last_became_active":"2024-04-25T10:30:32.430615+0000","last_became_peered":"2024-04-25T10:30:32.430615+0000","last_unstale":"2024-04-25T10:34:43.938625+0000","last_undegraded":"2024-04-25T10:34:43.938625+0000","last_fullsized":"2024-04-25T10:34:43.938625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:12.747595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"22'150","reported_seq":168,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.959087+0000","last_change":"2024-04-25T10:30:32.435379+0000","last_active":"2024-04-25T10:34:43.959087+0000","last_peered":"2024-04-25T10:34:43.959087+0000","last_clean":"2024-04-25T10:34:43.959087+0000","last_became_active":"2024-04-25T10:30:32.435055+0000","last_became_peered":"2024-04-25T10:30:32.435055+0000","last_unstale":"2024-04-25T10:34:43.959087+0000","last_undegraded":"2024-04-25T10:34:43.959087+0000","last_fullsized":"2024-04-25T10:34:43.959087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:13.623762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"22'130","reported_seq":147,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.959593+0000","last_change":"2024-04-25T10:30:32.422102+0000","last_active":"2024-04-25T10:34:43.959593+0000","last_peered":"2024-04-25T10:34:43.959593+0000","last_clean":"2024-04-25T10:34:43.959593+0000","last_became_active":"2024-04-25T10:30:32.421766+0000","last_became_peered":"2024-04-25T10:30:32.421766+0000","last_unstale":"2024-04-25T10:34:43.959593+0000","last_undegraded":"2024-04-25T10:34:43.959593+0000","last_fullsized":"2024-04-25T10:34:43.959593+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:30.300572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'7","reported_seq":26,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:33:05.134865+0000","last_change":"2024-04-25T10:30:31.418065+0000","last_active":"2024-04-25T10:33:05.134865+0000","last_peered":"2024-04-25T10:33:05.134865+0000","last_clean":"2024-04-25T10:33:05.134865+0000","last_became_active":"2024-04-25T10:30:31.417451+0000","last_became_peered":"2024-04-25T10:30:31.417451+0000","last_unstale":"2024-04-25T10:33:05.134865+0000","last_undegraded":"2024-04-25T10:33:05.134865+0000","last_fullsized":"2024-04-25T10:33:05.134865+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:17.262861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1204,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"22'137","reported_seq":154,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.941470+0000","last_change":"2024-04-25T10:30:32.426913+0000","last_active":"2024-04-25T10:34:43.941470+0000","last_peered":"2024-04-25T10:34:43.941470+0000","last_clean":"2024-04-25T10:34:43.941470+0000","last_became_active":"2024-04-25T10:30:32.426616+0000","last_became_peered":"2024-04-25T10:30:32.426616+0000","last_unstale":"2024-04-25T10:34:43.941470+0000","last_undegraded":"2024-04-25T10:34:43.941470+0000","last_fullsized":"2024-04-25T10:34:43.941470+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:50.614932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'7","reported_seq":26,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:31:13.638951+0000","last_change":"2024-04-25T10:30:31.416432+0000","last_active":"2024-04-25T10:31:13.638951+0000","last_peered":"2024-04-25T10:31:13.638951+0000","last_clean":"2024-04-25T10:31:13.638951+0000","last_became_active":"2024-04-25T10:30:31.415112+0000","last_became_peered":"2024-04-25T10:30:31.415112+0000","last_unstale":"2024-04-25T10:31:13.638951+0000","last_undegraded":"2024-04-25T10:31:13.638951+0000","last_fullsized":"2024-04-25T10:31:13.638951+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:22.674365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"20'3","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:31:13.636650+0000","last_change":"2024-04-25T10:30:31.418059+0000","last_active":"2024-04-25T10:31:13.636650+0000","last_peered":"2024-04-25T10:31:13.636650+0000","last_clean":"2024-04-25T10:31:13.636650+0000","last_became_active":"2024-04-25T10:30:31.417716+0000","last_became_peered":"2024-04-25T10:30:31.417716+0000","last_unstale":"2024-04-25T10:31:13.636650+0000","last_undegraded":"2024-04-25T10:31:13.636650+0000","last_fullsized":"2024-04-25T10:31:13.636650+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:57.752218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"22'170","reported_seq":187,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.938474+0000","last_change":"2024-04-25T10:30:32.428078+0000","last_active":"2024-04-25T10:34:43.938474+0000","last_peered":"2024-04-25T10:34:43.938474+0000","last_clean":"2024-04-25T10:34:43.938474+0000","last_became_active":"2024-04-25T10:30:32.427700+0000","last_became_peered":"2024-04-25T10:30:32.427700+0000","last_unstale":"2024-04-25T10:34:43.938474+0000","last_undegraded":"2024-04-25T10:34:43.938474+0000","last_fullsized":"2024-04-25T10:34:43.938474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:26.931791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":162,"num_write_kb":442368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"20'3","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:31:13.638118+0000","last_change":"2024-04-25T10:30:31.415030+0000","last_active":"2024-04-25T10:31:13.638118+0000","last_peered":"2024-04-25T10:31:13.638118+0000","last_clean":"2024-04-25T10:31:13.638118+0000","last_became_active":"2024-04-25T10:30:31.414542+0000","last_became_peered":"2024-04-25T10:30:31.414542+0000","last_unstale":"2024-04-25T10:31:13.638118+0000","last_undegraded":"2024-04-25T10:31:13.638118+0000","last_fullsized":"2024-04-25T10:31:13.638118+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:20.082420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"22'157","reported_seq":174,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.939532+0000","last_change":"2024-04-25T10:30:32.425207+0000","last_active":"2024-04-25T10:34:43.939532+0000","last_peered":"2024-04-25T10:34:43.939532+0000","last_clean":"2024-04-25T10:34:43.939532+0000","last_became_active":"2024-04-25T10:30:32.424849+0000","last_became_peered":"2024-04-25T10:30:32.424849+0000","last_unstale":"2024-04-25T10:34:43.939532+0000","last_undegraded":"2024-04-25T10:34:43.939532+0000","last_fullsized":"2024-04-25T10:34:43.939532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:11.533686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"22'8","reported_seq":27,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.902339+0000","last_change":"2024-04-25T10:30:31.407700+0000","last_active":"2024-04-25T10:34:43.902339+0000","last_peered":"2024-04-25T10:34:43.902339+0000","last_clean":"2024-04-25T10:34:43.902339+0000","last_became_active":"2024-04-25T10:30:31.407376+0000","last_became_peered":"2024-04-25T10:30:31.407376+0000","last_unstale":"2024-04-25T10:34:43.902339+0000","last_undegraded":"2024-04-25T10:34:43.902339+0000","last_fullsized":"2024-04-25T10:34:43.902339+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:50.112742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"22'229","reported_seq":246,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.960310+0000","last_change":"2024-04-25T10:30:32.444599+0000","last_active":"2024-04-25T10:34:43.960310+0000","last_peered":"2024-04-25T10:34:43.960310+0000","last_clean":"2024-04-25T10:34:43.960310+0000","last_became_active":"2024-04-25T10:30:32.444250+0000","last_became_peered":"2024-04-25T10:30:32.444250+0000","last_unstale":"2024-04-25T10:34:43.960310+0000","last_undegraded":"2024-04-25T10:34:43.960310+0000","last_fullsized":"2024-04-25T10:34:43.960310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":229,"log_dups_size":0,"ondisk_log_size":229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:55.711525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":219,"num_write_kb":598016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"20'4","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:31:13.638996+0000","last_change":"2024-04-25T10:30:31.405051+0000","last_active":"2024-04-25T10:31:13.638996+0000","last_peered":"2024-04-25T10:31:13.638996+0000","last_clean":"2024-04-25T10:31:13.638996+0000","last_became_active":"2024-04-25T10:30:31.404669+0000","last_became_peered":"2024-04-25T10:30:31.404669+0000","last_unstale":"2024-04-25T10:31:13.638996+0000","last_undegraded":"2024-04-25T10:31:13.638996+0000","last_fullsized":"2024-04-25T10:31:13.638996+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:45.260847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"22'141","reported_seq":158,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.953061+0000","last_change":"2024-04-25T10:30:32.435804+0000","last_active":"2024-04-25T10:34:43.953061+0000","last_peered":"2024-04-25T10:34:43.953061+0000","last_clean":"2024-04-25T10:34:43.953061+0000","last_became_active":"2024-04-25T10:30:32.435471+0000","last_became_peered":"2024-04-25T10:30:32.435471+0000","last_unstale":"2024-04-25T10:34:43.953061+0000","last_undegraded":"2024-04-25T10:34:43.953061+0000","last_fullsized":"2024-04-25T10:34:43.953061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:48.213846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"22'11","reported_seq":30,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:27.461358+0000","last_change":"2024-04-25T10:30:31.403560+0000","last_active":"2024-04-25T10:34:27.461358+0000","last_peered":"2024-04-25T10:34:27.461358+0000","last_clean":"2024-04-25T10:34:27.461358+0000","last_became_active":"2024-04-25T10:30:31.403167+0000","last_became_peered":"2024-04-25T10:30:31.403167+0000","last_unstale":"2024-04-25T10:34:27.461358+0000","last_undegraded":"2024-04-25T10:34:27.461358+0000","last_fullsized":"2024-04-25T10:34:27.461358+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:54.592832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"22'149","reported_seq":166,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.967685+0000","last_change":"2024-04-25T10:30:32.424101+0000","last_active":"2024-04-25T10:34:43.967685+0000","last_peered":"2024-04-25T10:34:43.967685+0000","last_clean":"2024-04-25T10:34:43.967685+0000","last_became_active":"2024-04-25T10:30:32.423780+0000","last_became_peered":"2024-04-25T10:30:32.423780+0000","last_unstale":"2024-04-25T10:34:43.967685+0000","last_undegraded":"2024-04-25T10:34:43.967685+0000","last_fullsized":"2024-04-25T10:34:43.967685+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:29.175348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"22'95","reported_seq":114,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.905445+0000","last_change":"2024-04-25T10:30:31.418201+0000","last_active":"2024-04-25T10:34:43.905445+0000","last_peered":"2024-04-25T10:34:43.905445+0000","last_clean":"2024-04-25T10:34:43.905445+0000","last_became_active":"2024-04-25T10:30:31.417824+0000","last_became_peered":"2024-04-25T10:30:31.417824+0000","last_unstale":"2024-04-25T10:34:43.905445+0000","last_undegraded":"2024-04-25T10:34:43.905445+0000","last_fullsized":"2024-04-25T10:34:43.905445+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:38.584225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":201790,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":96,"num_write_kb":233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"22'150","reported_seq":167,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.951616+0000","last_change":"2024-04-25T10:30:32.431139+0000","last_active":"2024-04-25T10:34:43.951616+0000","last_peered":"2024-04-25T10:34:43.951616+0000","last_clean":"2024-04-25T10:34:43.951616+0000","last_became_active":"2024-04-25T10:30:32.429622+0000","last_became_peered":"2024-04-25T10:30:32.429622+0000","last_unstale":"2024-04-25T10:34:43.951616+0000","last_undegraded":"2024-04-25T10:34:43.951616+0000","last_fullsized":"2024-04-25T10:34:43.951616+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:39.993647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'9","reported_seq":28,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:31:13.675426+0000","last_change":"2024-04-25T10:30:31.414775+0000","last_active":"2024-04-25T10:31:13.675426+0000","last_peered":"2024-04-25T10:31:13.675426+0000","last_clean":"2024-04-25T10:31:13.675426+0000","last_became_active":"2024-04-25T10:30:31.414422+0000","last_became_peered":"2024-04-25T10:30:31.414422+0000","last_unstale":"2024-04-25T10:31:13.675426+0000","last_undegraded":"2024-04-25T10:31:13.675426+0000","last_fullsized":"2024-04-25T10:31:13.675426+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:37.803505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"22'157","reported_seq":174,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.944852+0000","last_change":"2024-04-25T10:30:32.432884+0000","last_active":"2024-04-25T10:34:43.944852+0000","last_peered":"2024-04-25T10:34:43.944852+0000","last_clean":"2024-04-25T10:34:43.944852+0000","last_became_active":"2024-04-25T10:30:32.431743+0000","last_became_peered":"2024-04-25T10:30:32.431743+0000","last_unstale":"2024-04-25T10:34:43.944852+0000","last_undegraded":"2024-04-25T10:34:43.944852+0000","last_fullsized":"2024-04-25T10:34:43.944852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:15:06.737631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"20'5","reported_seq":24,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:31:13.638219+0000","last_change":"2024-04-25T10:30:31.419497+0000","last_active":"2024-04-25T10:31:13.638219+0000","last_peered":"2024-04-25T10:31:13.638219+0000","last_clean":"2024-04-25T10:31:13.638219+0000","last_became_active":"2024-04-25T10:30:31.419135+0000","last_became_peered":"2024-04-25T10:30:31.419135+0000","last_unstale":"2024-04-25T10:31:13.638219+0000","last_undegraded":"2024-04-25T10:31:13.638219+0000","last_fullsized":"2024-04-25T10:31:13.638219+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:59.403927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"22'171","reported_seq":188,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.964294+0000","last_change":"2024-04-25T10:30:32.434197+0000","last_active":"2024-04-25T10:34:43.964294+0000","last_peered":"2024-04-25T10:34:43.964294+0000","last_clean":"2024-04-25T10:34:43.964294+0000","last_became_active":"2024-04-25T10:30:32.433623+0000","last_became_peered":"2024-04-25T10:30:32.433623+0000","last_unstale":"2024-04-25T10:34:43.964294+0000","last_undegraded":"2024-04-25T10:34:43.964294+0000","last_fullsized":"2024-04-25T10:34:43.964294+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:08.027472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"22'7","reported_seq":26,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:32:55.135185+0000","last_change":"2024-04-25T10:30:31.416901+0000","last_active":"2024-04-25T10:32:55.135185+0000","last_peered":"2024-04-25T10:32:55.135185+0000","last_clean":"2024-04-25T10:32:55.135185+0000","last_became_active":"2024-04-25T10:30:31.415706+0000","last_became_peered":"2024-04-25T10:30:31.415706+0000","last_unstale":"2024-04-25T10:32:55.135185+0000","last_undegraded":"2024-04-25T10:32:55.135185+0000","last_fullsized":"2024-04-25T10:32:55.135185+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:23.243039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"22'165","reported_seq":182,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.946112+0000","last_change":"2024-04-25T10:30:32.435089+0000","last_active":"2024-04-25T10:34:43.946112+0000","last_peered":"2024-04-25T10:34:43.946112+0000","last_clean":"2024-04-25T10:34:43.946112+0000","last_became_active":"2024-04-25T10:30:32.434737+0000","last_became_peered":"2024-04-25T10:30:32.434737+0000","last_unstale":"2024-04-25T10:34:43.946112+0000","last_undegraded":"2024-04-25T10:34:43.946112+0000","last_fullsized":"2024-04-25T10:34:43.946112+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:19.738850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159,"num_write_kb":434176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"22'18","reported_seq":37,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:32:55.131953+0000","last_change":"2024-04-25T10:30:31.405163+0000","last_active":"2024-04-25T10:32:55.131953+0000","last_peered":"2024-04-25T10:32:55.131953+0000","last_clean":"2024-04-25T10:32:55.131953+0000","last_became_active":"2024-04-25T10:30:31.404775+0000","last_became_peered":"2024-04-25T10:30:31.404775+0000","last_unstale":"2024-04-25T10:32:55.131953+0000","last_undegraded":"2024-04-25T10:32:55.131953+0000","last_fullsized":"2024-04-25T10:32:55.131953+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:10:21.595653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"22'189","reported_seq":206,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.961535+0000","last_change":"2024-04-25T10:30:32.432061+0000","last_active":"2024-04-25T10:34:43.961535+0000","last_peered":"2024-04-25T10:34:43.961535+0000","last_clean":"2024-04-25T10:34:43.961535+0000","last_became_active":"2024-04-25T10:30:32.431651+0000","last_became_peered":"2024-04-25T10:30:32.431651+0000","last_unstale":"2024-04-25T10:34:43.961535+0000","last_undegraded":"2024-04-25T10:34:43.961535+0000","last_fullsized":"2024-04-25T10:34:43.961535+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:39.062490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'2","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:31:13.676443+0000","last_change":"2024-04-25T10:30:31.420258+0000","last_active":"2024-04-25T10:31:13.676443+0000","last_peered":"2024-04-25T10:31:13.676443+0000","last_clean":"2024-04-25T10:31:13.676443+0000","last_became_active":"2024-04-25T10:30:31.419902+0000","last_became_peered":"2024-04-25T10:30:31.419902+0000","last_unstale":"2024-04-25T10:31:13.676443+0000","last_undegraded":"2024-04-25T10:31:13.676443+0000","last_fullsized":"2024-04-25T10:31:13.676443+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:43.919867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"22'146","reported_seq":163,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.963108+0000","last_change":"2024-04-25T10:30:32.433568+0000","last_active":"2024-04-25T10:34:43.963108+0000","last_peered":"2024-04-25T10:34:43.963108+0000","last_clean":"2024-04-25T10:34:43.963108+0000","last_became_active":"2024-04-25T10:30:32.432572+0000","last_became_peered":"2024-04-25T10:30:32.432572+0000","last_unstale":"2024-04-25T10:34:43.963108+0000","last_undegraded":"2024-04-25T10:34:43.963108+0000","last_fullsized":"2024-04-25T10:34:43.963108+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:31.771919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'5","reported_seq":24,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:31:13.636179+0000","last_change":"2024-04-25T10:30:31.419308+0000","last_active":"2024-04-25T10:31:13.636179+0000","last_peered":"2024-04-25T10:31:13.636179+0000","last_clean":"2024-04-25T10:31:13.636179+0000","last_became_active":"2024-04-25T10:30:31.418978+0000","last_became_peered":"2024-04-25T10:30:31.418978+0000","last_unstale":"2024-04-25T10:31:13.636179+0000","last_undegraded":"2024-04-25T10:31:13.636179+0000","last_fullsized":"2024-04-25T10:31:13.636179+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:33.784804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"22'138","reported_seq":155,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.966851+0000","last_change":"2024-04-25T10:30:32.436109+0000","last_active":"2024-04-25T10:34:43.966851+0000","last_peered":"2024-04-25T10:34:43.966851+0000","last_clean":"2024-04-25T10:34:43.966851+0000","last_became_active":"2024-04-25T10:30:32.434701+0000","last_became_peered":"2024-04-25T10:30:32.434701+0000","last_unstale":"2024-04-25T10:34:43.966851+0000","last_undegraded":"2024-04-25T10:34:43.966851+0000","last_fullsized":"2024-04-25T10:34:43.966851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:19.471867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'3","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:31:13.636578+0000","last_change":"2024-04-25T10:30:31.416044+0000","last_active":"2024-04-25T10:31:13.636578+0000","last_peered":"2024-04-25T10:31:13.636578+0000","last_clean":"2024-04-25T10:31:13.636578+0000","last_became_active":"2024-04-25T10:30:31.415707+0000","last_became_peered":"2024-04-25T10:30:31.415707+0000","last_unstale":"2024-04-25T10:31:13.636578+0000","last_undegraded":"2024-04-25T10:31:13.636578+0000","last_fullsized":"2024-04-25T10:31:13.636578+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:31.424135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"22'144","reported_seq":161,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.958006+0000","last_change":"2024-04-25T10:30:32.442971+0000","last_active":"2024-04-25T10:34:43.958006+0000","last_peered":"2024-04-25T10:34:43.958006+0000","last_clean":"2024-04-25T10:34:43.958006+0000","last_became_active":"2024-04-25T10:30:32.442656+0000","last_became_peered":"2024-04-25T10:30:32.442656+0000","last_unstale":"2024-04-25T10:34:43.958006+0000","last_undegraded":"2024-04-25T10:34:43.958006+0000","last_fullsized":"2024-04-25T10:34:43.958006+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:29.951142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"22'7","reported_seq":26,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:32:50.131436+0000","last_change":"2024-04-25T10:30:31.404793+0000","last_active":"2024-04-25T10:32:50.131436+0000","last_peered":"2024-04-25T10:32:50.131436+0000","last_clean":"2024-04-25T10:32:50.131436+0000","last_became_active":"2024-04-25T10:30:31.404392+0000","last_became_peered":"2024-04-25T10:30:31.404392+0000","last_unstale":"2024-04-25T10:32:50.131436+0000","last_undegraded":"2024-04-25T10:32:50.131436+0000","last_fullsized":"2024-04-25T10:32:50.131436+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:00.965450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1752,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"22'151","reported_seq":168,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.967451+0000","last_change":"2024-04-25T10:30:32.441438+0000","last_active":"2024-04-25T10:34:43.967451+0000","last_peered":"2024-04-25T10:34:43.967451+0000","last_clean":"2024-04-25T10:34:43.967451+0000","last_became_active":"2024-04-25T10:30:32.440011+0000","last_became_peered":"2024-04-25T10:30:32.440011+0000","last_unstale":"2024-04-25T10:34:43.967451+0000","last_undegraded":"2024-04-25T10:34:43.967451+0000","last_fullsized":"2024-04-25T10:34:43.967451+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:09.127517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":116,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:32:25.132929+0000","last_change":"2024-04-25T10:30:21.429098+0000","last_active":"2024-04-25T10:32:25.132929+0000","last_peered":"2024-04-25T10:32:25.132929+0000","last_clean":"2024-04-25T10:32:25.132929+0000","last_became_active":"2024-04-25T10:30:21.427958+0000","last_became_peered":"2024-04-25T10:30:21.427958+0000","last_unstale":"2024-04-25T10:32:25.132929+0000","last_undegraded":"2024-04-25T10:32:25.132929+0000","last_fullsized":"2024-04-25T10:32:25.132929+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:20.186206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:20.186206+0000","last_clean_scrub_stamp":"2024-04-25T10:30:20.186206+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:46.046488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"20'6","reported_seq":26,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:32:40.133596+0000","last_change":"2024-04-25T10:30:31.404974+0000","last_active":"2024-04-25T10:32:40.133596+0000","last_peered":"2024-04-25T10:32:40.133596+0000","last_clean":"2024-04-25T10:32:40.133596+0000","last_became_active":"2024-04-25T10:30:31.404549+0000","last_became_peered":"2024-04-25T10:30:31.404549+0000","last_unstale":"2024-04-25T10:32:40.133596+0000","last_undegraded":"2024-04-25T10:32:40.133596+0000","last_fullsized":"2024-04-25T10:32:40.133596+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:06.191321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"22'158","reported_seq":176,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.960745+0000","last_change":"2024-04-25T10:30:32.434409+0000","last_active":"2024-04-25T10:34:43.960745+0000","last_peered":"2024-04-25T10:34:43.960745+0000","last_clean":"2024-04-25T10:34:43.960745+0000","last_became_active":"2024-04-25T10:30:32.433830+0000","last_became_peered":"2024-04-25T10:30:32.433830+0000","last_unstale":"2024-04-25T10:34:43.960745+0000","last_undegraded":"2024-04-25T10:34:43.960745+0000","last_fullsized":"2024-04-25T10:34:43.960745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:57.864627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":409600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"20'4","reported_seq":24,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:32:35.134131+0000","last_change":"2024-04-25T10:30:31.416025+0000","last_active":"2024-04-25T10:32:35.134131+0000","last_peered":"2024-04-25T10:32:35.134131+0000","last_clean":"2024-04-25T10:32:35.134131+0000","last_became_active":"2024-04-25T10:30:31.415675+0000","last_became_peered":"2024-04-25T10:30:31.415675+0000","last_unstale":"2024-04-25T10:32:35.134131+0000","last_undegraded":"2024-04-25T10:32:35.134131+0000","last_fullsized":"2024-04-25T10:32:35.134131+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:28:48.185715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"22'131","reported_seq":149,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.933599+0000","last_change":"2024-04-25T10:30:32.424630+0000","last_active":"2024-04-25T10:34:43.933599+0000","last_peered":"2024-04-25T10:34:43.933599+0000","last_clean":"2024-04-25T10:34:43.933599+0000","last_became_active":"2024-04-25T10:30:32.424303+0000","last_became_peered":"2024-04-25T10:30:32.424303+0000","last_unstale":"2024-04-25T10:34:43.933599+0000","last_undegraded":"2024-04-25T10:34:43.933599+0000","last_fullsized":"2024-04-25T10:34:43.933599+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:55.535319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"22'13","reported_seq":38,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.901417+0000","last_change":"2024-04-25T10:30:31.419145+0000","last_active":"2024-04-25T10:34:43.901417+0000","last_peered":"2024-04-25T10:34:43.901417+0000","last_clean":"2024-04-25T10:34:43.901417+0000","last_became_active":"2024-04-25T10:30:31.418817+0000","last_became_peered":"2024-04-25T10:30:31.418817+0000","last_unstale":"2024-04-25T10:34:43.901417+0000","last_undegraded":"2024-04-25T10:34:43.901417+0000","last_fullsized":"2024-04-25T10:34:43.901417+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:01.608205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":12,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"22'131","reported_seq":149,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.929954+0000","last_change":"2024-04-25T10:30:32.419911+0000","last_active":"2024-04-25T10:34:43.929954+0000","last_peered":"2024-04-25T10:34:43.929954+0000","last_clean":"2024-04-25T10:34:43.929954+0000","last_became_active":"2024-04-25T10:30:32.419486+0000","last_became_peered":"2024-04-25T10:30:32.419486+0000","last_unstale":"2024-04-25T10:34:43.929954+0000","last_undegraded":"2024-04-25T10:34:43.929954+0000","last_fullsized":"2024-04-25T10:34:43.929954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:54.967873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"20'5","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:32:40.133576+0000","last_change":"2024-04-25T10:30:31.405646+0000","last_active":"2024-04-25T10:32:40.133576+0000","last_peered":"2024-04-25T10:32:40.133576+0000","last_clean":"2024-04-25T10:32:40.133576+0000","last_became_active":"2024-04-25T10:30:31.405238+0000","last_became_peered":"2024-04-25T10:30:31.405238+0000","last_unstale":"2024-04-25T10:32:40.133576+0000","last_undegraded":"2024-04-25T10:32:40.133576+0000","last_fullsized":"2024-04-25T10:32:40.133576+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:13.159787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"22'145","reported_seq":163,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.936816+0000","last_change":"2024-04-25T10:30:32.426130+0000","last_active":"2024-04-25T10:34:43.936816+0000","last_peered":"2024-04-25T10:34:43.936816+0000","last_clean":"2024-04-25T10:34:43.936816+0000","last_became_active":"2024-04-25T10:30:32.425783+0000","last_became_peered":"2024-04-25T10:30:32.425783+0000","last_unstale":"2024-04-25T10:34:43.936816+0000","last_undegraded":"2024-04-25T10:34:43.936816+0000","last_fullsized":"2024-04-25T10:34:43.936816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:30.844640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"20'1","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:32:40.131788+0000","last_change":"2024-04-25T10:30:31.412736+0000","last_active":"2024-04-25T10:32:40.131788+0000","last_peered":"2024-04-25T10:32:40.131788+0000","last_clean":"2024-04-25T10:32:40.131788+0000","last_became_active":"2024-04-25T10:30:31.412413+0000","last_became_peered":"2024-04-25T10:30:31.412413+0000","last_unstale":"2024-04-25T10:32:40.131788+0000","last_undegraded":"2024-04-25T10:32:40.131788+0000","last_fullsized":"2024-04-25T10:32:40.131788+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:19:56.236327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"22'143","reported_seq":166,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.951783+0000","last_change":"2024-04-25T10:31:13.640815+0000","last_active":"2024-04-25T10:34:43.951783+0000","last_peered":"2024-04-25T10:34:43.951783+0000","last_clean":"2024-04-25T10:34:43.951783+0000","last_became_active":"2024-04-25T10:31:13.640368+0000","last_became_peered":"2024-04-25T10:31:13.640368+0000","last_unstale":"2024-04-25T10:34:43.951783+0000","last_undegraded":"2024-04-25T10:34:43.951783+0000","last_fullsized":"2024-04-25T10:34:43.951783+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:15.845263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"20'4","reported_seq":24,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:32:40.134663+0000","last_change":"2024-04-25T10:30:31.417958+0000","last_active":"2024-04-25T10:32:40.134663+0000","last_peered":"2024-04-25T10:32:40.134663+0000","last_clean":"2024-04-25T10:32:40.134663+0000","last_became_active":"2024-04-25T10:30:31.417541+0000","last_became_peered":"2024-04-25T10:30:31.417541+0000","last_unstale":"2024-04-25T10:32:40.134663+0000","last_undegraded":"2024-04-25T10:32:40.134663+0000","last_fullsized":"2024-04-25T10:32:40.134663+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:35.373351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"22'144","reported_seq":162,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.937182+0000","last_change":"2024-04-25T10:30:32.420451+0000","last_active":"2024-04-25T10:34:43.937182+0000","last_peered":"2024-04-25T10:34:43.937182+0000","last_clean":"2024-04-25T10:34:43.937182+0000","last_became_active":"2024-04-25T10:30:32.420109+0000","last_became_peered":"2024-04-25T10:30:32.420109+0000","last_unstale":"2024-04-25T10:34:43.937182+0000","last_undegraded":"2024-04-25T10:34:43.937182+0000","last_fullsized":"2024-04-25T10:34:43.937182+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:22.067828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'23","reported_seq":42,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:28.491838+0000","last_change":"2024-04-25T10:30:31.406005+0000","last_active":"2024-04-25T10:34:28.491838+0000","last_peered":"2024-04-25T10:34:28.491838+0000","last_clean":"2024-04-25T10:34:28.491838+0000","last_became_active":"2024-04-25T10:30:31.404739+0000","last_became_peered":"2024-04-25T10:30:31.404739+0000","last_unstale":"2024-04-25T10:34:28.491838+0000","last_undegraded":"2024-04-25T10:34:28.491838+0000","last_fullsized":"2024-04-25T10:34:28.491838+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:31.771123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"22'143","reported_seq":160,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.955595+0000","last_change":"2024-04-25T10:30:32.437513+0000","last_active":"2024-04-25T10:34:43.955595+0000","last_peered":"2024-04-25T10:34:43.955595+0000","last_clean":"2024-04-25T10:34:43.955595+0000","last_became_active":"2024-04-25T10:30:32.437171+0000","last_became_peered":"2024-04-25T10:30:32.437171+0000","last_unstale":"2024-04-25T10:34:43.955595+0000","last_undegraded":"2024-04-25T10:34:43.955595+0000","last_fullsized":"2024-04-25T10:34:43.955595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:50.924843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"22'8","reported_seq":27,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:32:45.134201+0000","last_change":"2024-04-25T10:30:31.413140+0000","last_active":"2024-04-25T10:32:45.134201+0000","last_peered":"2024-04-25T10:32:45.134201+0000","last_clean":"2024-04-25T10:32:45.134201+0000","last_became_active":"2024-04-25T10:30:31.412821+0000","last_became_peered":"2024-04-25T10:30:31.412821+0000","last_unstale":"2024-04-25T10:32:45.134201+0000","last_undegraded":"2024-04-25T10:32:45.134201+0000","last_fullsized":"2024-04-25T10:32:45.134201+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:37.177254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"22'163","reported_seq":180,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.936828+0000","last_change":"2024-04-25T10:30:32.434333+0000","last_active":"2024-04-25T10:34:43.936828+0000","last_peered":"2024-04-25T10:34:43.936828+0000","last_clean":"2024-04-25T10:34:43.936828+0000","last_became_active":"2024-04-25T10:30:32.434000+0000","last_became_peered":"2024-04-25T10:30:32.434000+0000","last_unstale":"2024-04-25T10:34:43.936828+0000","last_undegraded":"2024-04-25T10:34:43.936828+0000","last_fullsized":"2024-04-25T10:34:43.936828+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:54.057457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"20'2","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:31:13.675909+0000","last_change":"2024-04-25T10:30:31.419475+0000","last_active":"2024-04-25T10:31:13.675909+0000","last_peered":"2024-04-25T10:31:13.675909+0000","last_clean":"2024-04-25T10:31:13.675909+0000","last_became_active":"2024-04-25T10:30:31.417605+0000","last_became_peered":"2024-04-25T10:30:31.417605+0000","last_unstale":"2024-04-25T10:31:13.675909+0000","last_undegraded":"2024-04-25T10:31:13.675909+0000","last_fullsized":"2024-04-25T10:31:13.675909+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:00.948269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"22'134","reported_seq":151,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.911130+0000","last_change":"2024-04-25T10:30:32.419693+0000","last_active":"2024-04-25T10:34:43.911130+0000","last_peered":"2024-04-25T10:34:43.911130+0000","last_clean":"2024-04-25T10:34:43.911130+0000","last_became_active":"2024-04-25T10:30:32.419370+0000","last_became_peered":"2024-04-25T10:30:32.419370+0000","last_unstale":"2024-04-25T10:34:43.911130+0000","last_undegraded":"2024-04-25T10:34:43.911130+0000","last_fullsized":"2024-04-25T10:34:43.911130+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:31.199065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"20'7","reported_seq":26,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:31:13.635773+0000","last_change":"2024-04-25T10:30:31.410767+0000","last_active":"2024-04-25T10:31:13.635773+0000","last_peered":"2024-04-25T10:31:13.635773+0000","last_clean":"2024-04-25T10:31:13.635773+0000","last_became_active":"2024-04-25T10:30:31.410404+0000","last_became_peered":"2024-04-25T10:30:31.410404+0000","last_unstale":"2024-04-25T10:31:13.635773+0000","last_undegraded":"2024-04-25T10:31:13.635773+0000","last_fullsized":"2024-04-25T10:31:13.635773+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:37.431532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"22'153","reported_seq":170,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.938119+0000","last_change":"2024-04-25T10:30:32.433687+0000","last_active":"2024-04-25T10:34:43.938119+0000","last_peered":"2024-04-25T10:34:43.938119+0000","last_clean":"2024-04-25T10:34:43.938119+0000","last_became_active":"2024-04-25T10:30:32.433389+0000","last_became_peered":"2024-04-25T10:30:32.433389+0000","last_unstale":"2024-04-25T10:34:43.938119+0000","last_undegraded":"2024-04-25T10:34:43.938119+0000","last_fullsized":"2024-04-25T10:34:43.938119+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:39.155317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"20'3","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:31:13.636188+0000","last_change":"2024-04-25T10:30:31.419118+0000","last_active":"2024-04-25T10:31:13.636188+0000","last_peered":"2024-04-25T10:31:13.636188+0000","last_clean":"2024-04-25T10:31:13.636188+0000","last_became_active":"2024-04-25T10:30:31.418822+0000","last_became_peered":"2024-04-25T10:30:31.418822+0000","last_unstale":"2024-04-25T10:31:13.636188+0000","last_undegraded":"2024-04-25T10:31:13.636188+0000","last_fullsized":"2024-04-25T10:31:13.636188+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:22.755042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"22'143","reported_seq":160,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.938567+0000","last_change":"2024-04-25T10:30:32.427894+0000","last_active":"2024-04-25T10:34:43.938567+0000","last_peered":"2024-04-25T10:34:43.938567+0000","last_clean":"2024-04-25T10:34:43.938567+0000","last_became_active":"2024-04-25T10:30:32.427504+0000","last_became_peered":"2024-04-25T10:30:32.427504+0000","last_unstale":"2024-04-25T10:34:43.938567+0000","last_undegraded":"2024-04-25T10:34:43.938567+0000","last_fullsized":"2024-04-25T10:34:43.938567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:53.877484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'5","reported_seq":24,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:32:50.135474+0000","last_change":"2024-04-25T10:30:31.417115+0000","last_active":"2024-04-25T10:32:50.135474+0000","last_peered":"2024-04-25T10:32:50.135474+0000","last_clean":"2024-04-25T10:32:50.135474+0000","last_became_active":"2024-04-25T10:30:31.416766+0000","last_became_peered":"2024-04-25T10:30:31.416766+0000","last_unstale":"2024-04-25T10:32:50.135474+0000","last_undegraded":"2024-04-25T10:32:50.135474+0000","last_fullsized":"2024-04-25T10:32:50.135474+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:06.021270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"22'128","reported_seq":145,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.962450+0000","last_change":"2024-04-25T10:30:32.431674+0000","last_active":"2024-04-25T10:34:43.962450+0000","last_peered":"2024-04-25T10:34:43.962450+0000","last_clean":"2024-04-25T10:34:43.962450+0000","last_became_active":"2024-04-25T10:30:32.431395+0000","last_became_peered":"2024-04-25T10:30:32.431395+0000","last_unstale":"2024-04-25T10:34:43.962450+0000","last_undegraded":"2024-04-25T10:34:43.962450+0000","last_fullsized":"2024-04-25T10:34:43.962450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:49.726323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"22'183","reported_seq":200,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.949651+0000","last_change":"2024-04-25T10:30:32.437700+0000","last_active":"2024-04-25T10:34:43.949651+0000","last_peered":"2024-04-25T10:34:43.949651+0000","last_clean":"2024-04-25T10:34:43.949651+0000","last_became_active":"2024-04-25T10:30:32.437242+0000","last_became_peered":"2024-04-25T10:30:32.437242+0000","last_unstale":"2024-04-25T10:34:43.949651+0000","last_undegraded":"2024-04-25T10:34:43.949651+0000","last_fullsized":"2024-04-25T10:34:43.949651+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:06.499414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":168,"num_write_kb":458752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'5","reported_seq":24,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:31:13.675847+0000","last_change":"2024-04-25T10:30:31.419678+0000","last_active":"2024-04-25T10:31:13.675847+0000","last_peered":"2024-04-25T10:31:13.675847+0000","last_clean":"2024-04-25T10:31:13.675847+0000","last_became_active":"2024-04-25T10:30:31.419269+0000","last_became_peered":"2024-04-25T10:30:31.419269+0000","last_unstale":"2024-04-25T10:31:13.675847+0000","last_undegraded":"2024-04-25T10:31:13.675847+0000","last_fullsized":"2024-04-25T10:31:13.675847+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:33.034260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"22'141","reported_seq":158,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.966109+0000","last_change":"2024-04-25T10:30:32.434410+0000","last_active":"2024-04-25T10:34:43.966109+0000","last_peered":"2024-04-25T10:34:43.966109+0000","last_clean":"2024-04-25T10:34:43.966109+0000","last_became_active":"2024-04-25T10:30:32.434093+0000","last_became_peered":"2024-04-25T10:30:32.434093+0000","last_unstale":"2024-04-25T10:34:43.966109+0000","last_undegraded":"2024-04-25T10:34:43.966109+0000","last_fullsized":"2024-04-25T10:34:43.966109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:46.964704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'13","reported_seq":32,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:33:05.136991+0000","last_change":"2024-04-25T10:30:31.418360+0000","last_active":"2024-04-25T10:33:05.136991+0000","last_peered":"2024-04-25T10:33:05.136991+0000","last_clean":"2024-04-25T10:33:05.136991+0000","last_became_active":"2024-04-25T10:30:31.418005+0000","last_became_peered":"2024-04-25T10:30:31.418005+0000","last_unstale":"2024-04-25T10:33:05.136991+0000","last_undegraded":"2024-04-25T10:33:05.136991+0000","last_fullsized":"2024-04-25T10:33:05.136991+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:15.556859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"22'135","reported_seq":153,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.953049+0000","last_change":"2024-04-25T10:30:32.439515+0000","last_active":"2024-04-25T10:34:43.953049+0000","last_peered":"2024-04-25T10:34:43.953049+0000","last_clean":"2024-04-25T10:34:43.953049+0000","last_became_active":"2024-04-25T10:30:32.439101+0000","last_became_peered":"2024-04-25T10:30:32.439101+0000","last_unstale":"2024-04-25T10:34:43.953049+0000","last_undegraded":"2024-04-25T10:34:43.953049+0000","last_fullsized":"2024-04-25T10:34:43.953049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:33.797903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'2","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:32:35.131979+0000","last_change":"2024-04-25T10:30:31.418070+0000","last_active":"2024-04-25T10:32:35.131979+0000","last_peered":"2024-04-25T10:32:35.131979+0000","last_clean":"2024-04-25T10:32:35.131979+0000","last_became_active":"2024-04-25T10:30:31.417737+0000","last_became_peered":"2024-04-25T10:30:31.417737+0000","last_unstale":"2024-04-25T10:32:35.131979+0000","last_undegraded":"2024-04-25T10:32:35.131979+0000","last_fullsized":"2024-04-25T10:32:35.131979+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:05.664855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"22'152","reported_seq":170,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.960186+0000","last_change":"2024-04-25T10:30:32.435258+0000","last_active":"2024-04-25T10:34:43.960186+0000","last_peered":"2024-04-25T10:34:43.960186+0000","last_clean":"2024-04-25T10:34:43.960186+0000","last_became_active":"2024-04-25T10:30:32.434930+0000","last_became_peered":"2024-04-25T10:30:32.434930+0000","last_unstale":"2024-04-25T10:34:43.960186+0000","last_undegraded":"2024-04-25T10:34:43.960186+0000","last_fullsized":"2024-04-25T10:34:43.960186+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:22.727778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'13","reported_seq":33,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:32:40.130599+0000","last_change":"2024-04-25T10:30:31.415611+0000","last_active":"2024-04-25T10:32:40.130599+0000","last_peered":"2024-04-25T10:32:40.130599+0000","last_clean":"2024-04-25T10:32:40.130599+0000","last_became_active":"2024-04-25T10:30:31.415071+0000","last_became_peered":"2024-04-25T10:30:31.415071+0000","last_unstale":"2024-04-25T10:32:40.130599+0000","last_undegraded":"2024-04-25T10:32:40.130599+0000","last_fullsized":"2024-04-25T10:32:40.130599+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:30:54.077864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9216,"num_write_kb":25165824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9717,"ondisk_log_size":9717,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":210463,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":345,"num_write_kb":324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":671744,"data_stored":420926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":308,"ondisk_log_size":308,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574906,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40336,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331504,"statfs":{"total":96636764160,"available":96595460096,"internally_reserved":0,"allocated":860160,"data_stored":676182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574906,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35784,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336056,"statfs":{"total":96636764160,"available":96600121344,"internally_reserved":0,"allocated":458752,"data_stored":286294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574907,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39768,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94332072,"statfs":{"total":96636764160,"available":96596041728,"internally_reserved":0,"allocated":278528,"data_stored":87646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574907,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39744,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94332096,"statfs":{"total":96636764160,"available":96596066304,"internally_reserved":0,"allocated":253952,"data_stored":85044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574906,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39952,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331888,"statfs":{"total":96636764160,"available":96595853312,"internally_reserved":0,"allocated":466944,"data_stored":286833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574906,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35596,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336244,"statfs":{"total":96636764160,"available":96600313856,"internally_reserved":0,"allocated":266240,"data_stored":85681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574906,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35588,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336252,"statfs":{"total":96636764160,"available":96600322048,"internally_reserved":0,"allocated":258048,"data_stored":84664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574906,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40328,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331512,"statfs":{"total":96636764160,"available":96595468288,"internally_reserved":0,"allocated":851968,"data_stored":674374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":237568,"data_stored":203701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":229376,"data_stored":203162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:34:51.301 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:34:51.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.444+0000 7fec0a76c640 1 Processor -- start 2024-04-25T10:34:51.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.444+0000 7fec0a76c640 1 -- start start 2024-04-25T10:34:51.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.445+0000 7fec0a76c640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fec040cc600 0x7fec040c5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:51.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.445+0000 7fec0a76c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec040c58d0 0x7fec040cce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:51.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.445+0000 7fec0a76c640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec0415b8c0 con 0x7fec040cb2b0 2024-04-25T10:34:51.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.445+0000 7fec0a76c640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fec0415eea0 con 0x7fec040cc600 2024-04-25T10:34:51.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.445+0000 7fec0a76c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fec0415b5e0 con 0x7fec040c58d0 2024-04-25T10:34:51.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.446+0000 7fec037fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec040c58d0 0x7fec040cce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:51.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.446+0000 7fec037fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec040c58d0 0x7fec040cce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.106:59820/0 (socket says 172.21.15.106:59820) 2024-04-25T10:34:51.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.446+0000 7fec03fff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fec040cc600 0x7fec040c5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:51.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.446+0000 7fec037fe640 1 -- 172.21.15.106:0/1231007704 learned_addr learned my addr 172.21.15.106:0/1231007704 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:34:51.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.446+0000 7fec08d09640 1 --1- 172.21.15.106:0/1231007704 >> v1:172.21.15.134:6789/0 conn(0x7fec040cb2b0 0x7fec04070bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.106:48588/0 (socket says 172.21.15.106:48588) 2024-04-25T10:34:51.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.446+0000 7fec037fe640 1 -- 172.21.15.106:0/1231007704 >> v2:172.21.15.134:3300/0 conn(0x7fec040cc600 msgr2=0x7fec040c5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:51.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.446+0000 7fec037fe640 1 --2- 172.21.15.106:0/1231007704 >> v2:172.21.15.134:3300/0 conn(0x7fec040cc600 0x7fec040c5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:51.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.446+0000 7fec037fe640 1 -- 172.21.15.106:0/1231007704 >> v1:172.21.15.134:6789/0 conn(0x7fec040cb2b0 legacy=0x7fec04070bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:51.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.446+0000 7fec037fe640 1 -- 172.21.15.106:0/1231007704 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec0415e0e0 con 0x7fec040c58d0 2024-04-25T10:34:51.449 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.446+0000 7fec03fff640 1 --2- 172.21.15.106:0/1231007704 >> v2:172.21.15.134:3300/0 conn(0x7fec040cc600 0x7fec040c5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:34:51.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.447+0000 7fec037fe640 1 --2- 172.21.15.106:0/1231007704 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec040c58d0 0x7fec040cce90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7febf8009870 tx=0x7febf802a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=779d1c37f8dc9320 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:51.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.447+0000 7fec02ffd640 1 -- 172.21.15.106:0/1231007704 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf8037070 con 0x7fec040c58d0 2024-04-25T10:34:51.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.447+0000 7fec02ffd640 1 -- 172.21.15.106:0/1231007704 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febf8032440 con 0x7fec040c58d0 2024-04-25T10:34:51.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.447+0000 7fec02ffd640 1 -- 172.21.15.106:0/1231007704 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf8032810 con 0x7fec040c58d0 2024-04-25T10:34:51.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.448+0000 7fec0a76c640 1 -- 172.21.15.106:0/1231007704 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec040c58d0 msgr2=0x7fec040cce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:51.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.448+0000 7fec0a76c640 1 --2- 172.21.15.106:0/1231007704 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec040c58d0 0x7fec040cce90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7febf8009870 tx=0x7febf802a250 comp rx=0 tx=0).stop 2024-04-25T10:34:51.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.448+0000 7fec0a76c640 1 -- 172.21.15.106:0/1231007704 shutdown_connections 2024-04-25T10:34:51.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.448+0000 7fec0a76c640 1 --2- 172.21.15.106:0/1231007704 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec040c58d0 0x7fec040cce90 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:51.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.448+0000 7fec0a76c640 1 --2- 172.21.15.106:0/1231007704 >> v2:172.21.15.134:3300/0 conn(0x7fec040cc600 0x7fec040c5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:51.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.448+0000 7fec0a76c640 1 -- 172.21.15.106:0/1231007704 >> 172.21.15.106:0/1231007704 conn(0x7fec040727d0 msgr2=0x7fec04072c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:51.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.448+0000 7fec0a76c640 1 -- 172.21.15.106:0/1231007704 shutdown_connections 2024-04-25T10:34:51.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.448+0000 7fec0a76c640 1 -- 172.21.15.106:0/1231007704 wait complete. 2024-04-25T10:34:51.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.449+0000 7fec0a76c640 1 Processor -- start 2024-04-25T10:34:51.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.450+0000 7fec0a76c640 1 -- start start 2024-04-25T10:34:51.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.451+0000 7fec0a76c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec040c58d0 0x7fec0406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:51.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.451+0000 7fec0a76c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec040cb2b0 0x7fec0414d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:51.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.451+0000 7fec0a76c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec040cc600 0x7fec04151090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:51.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.451+0000 7fec0a76c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fec0415e980 con 0x7fec040cb2b0 2024-04-25T10:34:51.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.451+0000 7fec0a76c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fec0415e6a0 con 0x7fec040c58d0 2024-04-25T10:34:51.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.451+0000 7fec0a76c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fec040cbff0 con 0x7fec040cc600 2024-04-25T10:34:51.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.451+0000 7fec037fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec040cb2b0 0x7fec0414d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:51.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.451+0000 7fec037fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec040cb2b0 0x7fec0414d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56022/0 (socket says 172.21.15.106:56022) 2024-04-25T10:34:51.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.451+0000 7fec037fe640 1 -- 172.21.15.106:0/2650931148 learned_addr learned my addr 172.21.15.106:0/2650931148 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:34:51.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.451+0000 7fec03fff640 1 --2- 172.21.15.106:0/2650931148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec040c58d0 0x7fec0406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:51.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.451+0000 7fec08d09640 1 --2- 172.21.15.106:0/2650931148 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec040cc600 0x7fec04151090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:51.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.451+0000 7fec037fe640 1 -- 172.21.15.106:0/2650931148 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec040cc600 msgr2=0x7fec04151090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:51.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.451+0000 7fec037fe640 1 --2- 172.21.15.106:0/2650931148 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec040cc600 0x7fec04151090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:51.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.451+0000 7fec037fe640 1 -- 172.21.15.106:0/2650931148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec040c58d0 msgr2=0x7fec0406edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:51.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.451+0000 7fec037fe640 1 --2- 172.21.15.106:0/2650931148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec040c58d0 0x7fec0406edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:51.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.451+0000 7fec037fe640 1 -- 172.21.15.106:0/2650931148 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec0414d600 con 0x7fec040cb2b0 2024-04-25T10:34:51.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.451+0000 7fec037fe640 1 --2- 172.21.15.106:0/2650931148 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec040cb2b0 0x7fec0414d0c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7febf8000c00 tx=0x7febf80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:51.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.452+0000 7fec00ff9640 1 -- 172.21.15.106:0/2650931148 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf8037040 con 0x7fec040cb2b0 2024-04-25T10:34:51.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.452+0000 7fec00ff9640 1 -- 172.21.15.106:0/2650931148 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febf8042070 con 0x7fec040cb2b0 2024-04-25T10:34:51.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.452+0000 7fec0a76c640 1 -- 172.21.15.106:0/2650931148 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec0406dd10 con 0x7fec040cb2b0 2024-04-25T10:34:51.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.452+0000 7fec00ff9640 1 -- 172.21.15.106:0/2650931148 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf8030860 con 0x7fec040cb2b0 2024-04-25T10:34:51.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.452+0000 7fec0a76c640 1 -- 172.21.15.106:0/2650931148 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec041f6ef0 con 0x7fec040cb2b0 2024-04-25T10:34:51.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.453+0000 7fec0a76c640 1 -- 172.21.15.106:0/2650931148 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec0406da80 con 0x7fec040cb2b0 2024-04-25T10:34:51.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.453+0000 7fec00ff9640 1 -- 172.21.15.106:0/2650931148 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7febf8030a00 con 0x7fec040cb2b0 2024-04-25T10:34:51.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.454+0000 7fec00ff9640 1 --2- 172.21.15.106:0/2650931148 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7febf00744e0 0x7febf00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:51.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.454+0000 7fec00ff9640 1 -- 172.21.15.106:0/2650931148 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7febf80b5930 con 0x7fec040cb2b0 2024-04-25T10:34:51.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.454+0000 7fec03fff640 1 --2- 172.21.15.106:0/2650931148 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7febf00744e0 0x7febf00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:51.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.454+0000 7fec03fff640 1 --2- 172.21.15.106:0/2650931148 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7febf00744e0 0x7febf00769f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7febf4009560 tx=0x7febf4009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:51.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.458+0000 7fec00ff9640 1 -- 172.21.15.106:0/2650931148 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7febf807fb10 con 0x7fec040cb2b0 2024-04-25T10:34:51.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.636+0000 7fec0a76c640 1 -- 172.21.15.106:0/2650931148 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fec0406a3d0 con 0x7fec040cb2b0 2024-04-25T10:34:51.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.637+0000 7fec00ff9640 1 -- 172.21.15.106:0/2650931148 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16890 (secure 0 0 0) 0x7febf8083e10 con 0x7fec040cb2b0 2024-04-25T10:34:51.640 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:34:51.640 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":22,"fsid":"3ee77985-e8e4-41cf-81e9-7c774cf7891f","created":"2024-04-25T10:30:07.981054+0000","modified":"2024-04-25T10:31:13.621853+0000","last_up_change":"2024-04-25T10:30:19.184325+0000","last_in_change":"2024-04-25T10:30:10.217553+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:30:20.111046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:30:30.314914+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:30:30.715500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a5962cb1-dbbd-47b1-8d15-9bae114bfdf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1240717079},{"type":"v1","addr":"172.21.15.106:6801","nonce":1240717079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1240717079},{"type":"v1","addr":"172.21.15.106:6803","nonce":1240717079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1240717079},{"type":"v1","addr":"172.21.15.106:6807","nonce":1240717079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1240717079},{"type":"v1","addr":"172.21.15.106:6805","nonce":1240717079}]},"public_addr":"172.21.15.106:6801/1240717079","cluster_addr":"172.21.15.106:6803/1240717079","heartbeat_back_addr":"172.21.15.106:6807/1240717079","heartbeat_front_addr":"172.21.15.106:6805/1240717079","state":["exists","up"]},{"osd":1,"uuid":"94436e13-4769-4752-8397-b5e396a6f7ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4089653051},{"type":"v1","addr":"172.21.15.106:6825","nonce":4089653051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4089653051},{"type":"v1","addr":"172.21.15.106:6827","nonce":4089653051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4089653051},{"type":"v1","addr":"172.21.15.106:6831","nonce":4089653051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4089653051},{"type":"v1","addr":"172.21.15.106:6829","nonce":4089653051}]},"public_addr":"172.21.15.106:6825/4089653051","cluster_addr":"172.21.15.106:6827/4089653051","heartbeat_back_addr":"172.21.15.106:6831/4089653051","heartbeat_front_addr":"172.21.15.106:6829/4089653051","state":["exists","up"]},{"osd":2,"uuid":"429aae0b-70c6-4e7b-9043-6a11334c7efc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":1441018239},{"type":"v1","addr":"172.21.15.106:6809","nonce":1441018239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1441018239},{"type":"v1","addr":"172.21.15.106:6811","nonce":1441018239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1441018239},{"type":"v1","addr":"172.21.15.106:6815","nonce":1441018239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":1441018239},{"type":"v1","addr":"172.21.15.106:6813","nonce":1441018239}]},"public_addr":"172.21.15.106:6809/1441018239","cluster_addr":"172.21.15.106:6811/1441018239","heartbeat_back_addr":"172.21.15.106:6815/1441018239","heartbeat_front_addr":"172.21.15.106:6813/1441018239","state":["exists","up"]},{"osd":3,"uuid":"9593a4d1-772f-4667-ab0d-2982f841e342","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":1846968711},{"type":"v1","addr":"172.21.15.106:6817","nonce":1846968711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1846968711},{"type":"v1","addr":"172.21.15.106:6819","nonce":1846968711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":1846968711},{"type":"v1","addr":"172.21.15.106:6823","nonce":1846968711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":1846968711},{"type":"v1","addr":"172.21.15.106:6821","nonce":1846968711}]},"public_addr":"172.21.15.106:6817/1846968711","cluster_addr":"172.21.15.106:6819/1846968711","heartbeat_back_addr":"172.21.15.106:6823/1846968711","heartbeat_front_addr":"172.21.15.106:6821/1846968711","state":["exists","up"]},{"osd":4,"uuid":"1b61f16d-5c60-480a-b449-c745aa3f6b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":729224739},{"type":"v1","addr":"172.21.15.134:6825","nonce":729224739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":729224739},{"type":"v1","addr":"172.21.15.134:6827","nonce":729224739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":729224739},{"type":"v1","addr":"172.21.15.134:6831","nonce":729224739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":729224739},{"type":"v1","addr":"172.21.15.134:6829","nonce":729224739}]},"public_addr":"172.21.15.134:6825/729224739","cluster_addr":"172.21.15.134:6827/729224739","heartbeat_back_addr":"172.21.15.134:6831/729224739","heartbeat_front_addr":"172.21.15.134:6829/729224739","state":["exists","up"]},{"osd":5,"uuid":"944cdd7f-47c0-4b71-b7e0-d093542f4528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3132275652},{"type":"v1","addr":"172.21.15.134:6809","nonce":3132275652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3132275652},{"type":"v1","addr":"172.21.15.134:6811","nonce":3132275652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3132275652},{"type":"v1","addr":"172.21.15.134:6815","nonce":3132275652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3132275652},{"type":"v1","addr":"172.21.15.134:6813","nonce":3132275652}]},"public_addr":"172.21.15.134:6809/3132275652","cluster_addr":"172.21.15.134:6811/3132275652","heartbeat_back_addr":"172.21.15.134:6815/3132275652","heartbeat_front_addr":"172.21.15.134:6813/3132275652","state":["exists","up"]},{"osd":6,"uuid":"cc2446e2-1d21-4135-ac04-6af7c990058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":992702668},{"type":"v1","addr":"172.21.15.134:6801","nonce":992702668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":992702668},{"type":"v1","addr":"172.21.15.134:6803","nonce":992702668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":992702668},{"type":"v1","addr":"172.21.15.134:6807","nonce":992702668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":992702668},{"type":"v1","addr":"172.21.15.134:6805","nonce":992702668}]},"public_addr":"172.21.15.134:6801/992702668","cluster_addr":"172.21.15.134:6803/992702668","heartbeat_back_addr":"172.21.15.134:6807/992702668","heartbeat_front_addr":"172.21.15.134:6805/992702668","state":["exists","up"]},{"osd":7,"uuid":"3df1fc7a-82fc-4257-8629-98027b6e2aed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":3291062284},{"type":"v1","addr":"172.21.15.134:6817","nonce":3291062284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":3291062284},{"type":"v1","addr":"172.21.15.134:6819","nonce":3291062284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":3291062284},{"type":"v1","addr":"172.21.15.134:6823","nonce":3291062284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":3291062284},{"type":"v1","addr":"172.21.15.134:6821","nonce":3291062284}]},"public_addr":"172.21.15.134:6817/3291062284","cluster_addr":"172.21.15.134:6819/3291062284","heartbeat_back_addr":"172.21.15.134:6823/3291062284","heartbeat_front_addr":"172.21.15.134:6821/3291062284","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.403941+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.382563+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.411544+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.330444+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.405430+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.387737+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.408922+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.454450+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":2}]},{"pgid":"3.2b","mappings":[{"from":0,"to":1}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:34:51.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.639+0000 7fec0a76c640 1 -- 172.21.15.106:0/2650931148 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7febf00744e0 msgr2=0x7febf00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:51.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.639+0000 7fec0a76c640 1 --2- 172.21.15.106:0/2650931148 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7febf00744e0 0x7febf00769f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7febf4009560 tx=0x7febf4009190 comp rx=0 tx=0).stop 2024-04-25T10:34:51.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.639+0000 7fec0a76c640 1 -- 172.21.15.106:0/2650931148 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec040cb2b0 msgr2=0x7fec0414d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:51.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.639+0000 7fec0a76c640 1 --2- 172.21.15.106:0/2650931148 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec040cb2b0 0x7fec0414d0c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7febf8000c00 tx=0x7febf80046c0 comp rx=0 tx=0).stop 2024-04-25T10:34:51.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.639+0000 7fec0a76c640 1 -- 172.21.15.106:0/2650931148 shutdown_connections 2024-04-25T10:34:51.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.639+0000 7fec0a76c640 1 --2- 172.21.15.106:0/2650931148 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7febf00744e0 0x7febf00769f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:51.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.639+0000 7fec0a76c640 1 --2- 172.21.15.106:0/2650931148 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec040cc600 0x7fec04151090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:51.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.639+0000 7fec0a76c640 1 --2- 172.21.15.106:0/2650931148 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec040cb2b0 0x7fec0414d0c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:51.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.639+0000 7fec0a76c640 1 --2- 172.21.15.106:0/2650931148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec040c58d0 0x7fec0406edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:51.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.639+0000 7fec0a76c640 1 -- 172.21.15.106:0/2650931148 >> 172.21.15.106:0/2650931148 conn(0x7fec040727d0 msgr2=0x7fec04153450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:51.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.639+0000 7fec0a76c640 1 -- 172.21.15.106:0/2650931148 shutdown_connections 2024-04-25T10:34:51.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:51.640+0000 7fec0a76c640 1 -- 172.21.15.106:0/2650931148 wait complete. 2024-04-25T10:34:52.654 INFO:tasks.ceph:Scrubbing osd.0 2024-04-25T10:34:52.654 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:34:52.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.776+0000 7f9082a9f640 1 Processor -- start 2024-04-25T10:34:52.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.776+0000 7f9082a9f640 1 -- start start 2024-04-25T10:34:52.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.777+0000 7f9082a9f640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f907c0cb550 0x7f907c154a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:52.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.777+0000 7f9082a9f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f907c0cb960 0x7f907c0c5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:52.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.777+0000 7f9082a9f640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f907c0c67c0 con 0x7f907c15a930 2024-04-25T10:34:52.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.777+0000 7f9082a9f640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f907c0c6aa0 con 0x7f907c0cb550 2024-04-25T10:34:52.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.777+0000 7f9082a9f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f907c0c6d80 con 0x7f907c0cb960 2024-04-25T10:34:52.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.777+0000 7f907bfff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f907c0cb960 0x7f907c0c5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:52.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.777+0000 7f907bfff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f907c0cb960 0x7f907c0c5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.106:59830/0 (socket says 172.21.15.106:59830) 2024-04-25T10:34:52.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.777+0000 7f907bfff640 1 -- 172.21.15.106:0/1775655824 learned_addr learned my addr 172.21.15.106:0/1775655824 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:34:52.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.777+0000 7f908083b640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f907c0cb550 0x7f907c154a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:52.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.778+0000 7f907bfff640 1 -- 172.21.15.106:0/1775655824 >> v2:172.21.15.134:3300/0 conn(0x7f907c0cb550 msgr2=0x7f907c154a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:52.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.778+0000 7f907bfff640 1 --2- 172.21.15.106:0/1775655824 >> v2:172.21.15.134:3300/0 conn(0x7f907c0cb550 0x7f907c154a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:52.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.778+0000 7f907bfff640 1 -- 172.21.15.106:0/1775655824 >> v1:172.21.15.134:6789/0 conn(0x7f907c15a930 legacy=0x7f907c15ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:52.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.778+0000 7f907bfff640 1 -- 172.21.15.106:0/1775655824 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f907c0caa70 con 0x7f907c0cb960 2024-04-25T10:34:52.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.779+0000 7f907bfff640 1 --2- 172.21.15.106:0/1775655824 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f907c0cb960 0x7f907c0c5e30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9068009870 tx=0x7f906802a300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d782f0484ce4b46 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:52.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.779+0000 7f907b7fe640 1 -- 172.21.15.106:0/1775655824 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9068037070 con 0x7f907c0cb960 2024-04-25T10:34:52.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.779+0000 7f907b7fe640 1 -- 172.21.15.106:0/1775655824 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f906802ae50 con 0x7f907c0cb960 2024-04-25T10:34:52.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.779+0000 7f907b7fe640 1 -- 172.21.15.106:0/1775655824 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9068032860 con 0x7f907c0cb960 2024-04-25T10:34:52.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.779+0000 7f9082a9f640 1 -- 172.21.15.106:0/1775655824 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f907c0cb960 msgr2=0x7f907c0c5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:52.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.779+0000 7f9082a9f640 1 --2- 172.21.15.106:0/1775655824 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f907c0cb960 0x7f907c0c5e30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9068009870 tx=0x7f906802a300 comp rx=0 tx=0).stop 2024-04-25T10:34:52.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.780+0000 7f9082a9f640 1 -- 172.21.15.106:0/1775655824 shutdown_connections 2024-04-25T10:34:52.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.780+0000 7f9082a9f640 1 --2- 172.21.15.106:0/1775655824 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f907c0cb960 0x7f907c0c5e30 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:52.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.780+0000 7f9082a9f640 1 --2- 172.21.15.106:0/1775655824 >> v2:172.21.15.134:3300/0 conn(0x7f907c0cb550 0x7f907c154a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:52.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.780+0000 7f9082a9f640 1 -- 172.21.15.106:0/1775655824 >> 172.21.15.106:0/1775655824 conn(0x7f907c0cffd0 msgr2=0x7f907c0d03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:52.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.780+0000 7f9082a9f640 1 -- 172.21.15.106:0/1775655824 shutdown_connections 2024-04-25T10:34:52.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.780+0000 7f9082a9f640 1 -- 172.21.15.106:0/1775655824 wait complete. 2024-04-25T10:34:52.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.781+0000 7f9082a9f640 1 Processor -- start 2024-04-25T10:34:52.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.781+0000 7f9082a9f640 1 -- start start 2024-04-25T10:34:52.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.783+0000 7f9082a9f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f907c0cb550 0x7f907c1f45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:52.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.783+0000 7f9082a9f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f907c0cb960 0x7f907c1467e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:52.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.783+0000 7f9082a9f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f907c15a930 0x7f907c146d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:52.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.783+0000 7f9082a9f640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f907c0cc780 con 0x7f907c15a930 2024-04-25T10:34:52.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.783+0000 7f9082a9f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f907c0cb030 con 0x7f907c0cb550 2024-04-25T10:34:52.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.783+0000 7f9082a9f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f907c154fc0 con 0x7f907c0cb960 2024-04-25T10:34:52.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.783+0000 7f908103c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f907c15a930 0x7f907c146d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:52.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.783+0000 7f908103c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f907c15a930 0x7f907c146d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56030/0 (socket says 172.21.15.106:56030) 2024-04-25T10:34:52.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.783+0000 7f908103c640 1 -- 172.21.15.106:0/1051830130 learned_addr learned my addr 172.21.15.106:0/1051830130 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:34:52.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.783+0000 7f907bfff640 1 --2- 172.21.15.106:0/1051830130 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f907c0cb960 0x7f907c1467e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:52.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.783+0000 7f908083b640 1 --2- 172.21.15.106:0/1051830130 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f907c0cb550 0x7f907c1f45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:52.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.783+0000 7f908103c640 1 -- 172.21.15.106:0/1051830130 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f907c0cb960 msgr2=0x7f907c1467e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:52.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.783+0000 7f908103c640 1 --2- 172.21.15.106:0/1051830130 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f907c0cb960 0x7f907c1467e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:52.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.783+0000 7f908103c640 1 -- 172.21.15.106:0/1051830130 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f907c0cb550 msgr2=0x7f907c1f45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:52.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.783+0000 7f908103c640 1 --2- 172.21.15.106:0/1051830130 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f907c0cb550 0x7f907c1f45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:52.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.783+0000 7f908103c640 1 -- 172.21.15.106:0/1051830130 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f907c0ce900 con 0x7f907c15a930 2024-04-25T10:34:52.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.783+0000 7f908103c640 1 --2- 172.21.15.106:0/1051830130 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f907c15a930 0x7f907c146d20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f907400c6b0 tx=0x7f907400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:52.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.783+0000 7f90797fa640 1 -- 172.21.15.106:0/1051830130 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9074003b70 con 0x7f907c15a930 2024-04-25T10:34:52.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.783+0000 7f90797fa640 1 -- 172.21.15.106:0/1051830130 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9074003d30 con 0x7f907c15a930 2024-04-25T10:34:52.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.783+0000 7f9082a9f640 1 -- 172.21.15.106:0/1051830130 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f907c149040 con 0x7f907c15a930 2024-04-25T10:34:52.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.783+0000 7f90797fa640 1 -- 172.21.15.106:0/1051830130 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9074005310 con 0x7f907c15a930 2024-04-25T10:34:52.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.784+0000 7f9082a9f640 1 -- 172.21.15.106:0/1051830130 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f907c0b3210 con 0x7f907c15a930 2024-04-25T10:34:52.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.784+0000 7f9082a9f640 1 -- 172.21.15.106:0/1051830130 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f907c1f5300 con 0x7f907c15a930 2024-04-25T10:34:52.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.785+0000 7f90797fa640 1 -- 172.21.15.106:0/1051830130 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f90740054b0 con 0x7f907c15a930 2024-04-25T10:34:52.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.786+0000 7f90797fa640 1 --2- 172.21.15.106:0/1051830130 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f90600744b0 0x7f90600769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:52.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.786+0000 7f908083b640 1 --2- 172.21.15.106:0/1051830130 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f90600744b0 0x7f90600769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:52.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.786+0000 7f90797fa640 1 -- 172.21.15.106:0/1051830130 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f9074095980 con 0x7f907c15a930 2024-04-25T10:34:52.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.786+0000 7f90797fa640 1 --2- 172.21.15.106:0/1051830130 >> [v2:172.21.15.106:6800/1240717079,v1:172.21.15.106:6801/1240717079] conn(0x7f906007c880 0x7f906007ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:52.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.786+0000 7f90797fa640 1 -- 172.21.15.106:0/1051830130 --> [v2:172.21.15.106:6800/1240717079,v1:172.21.15.106:6801/1240717079] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f906007f420 con 0x7f906007c880 2024-04-25T10:34:52.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.786+0000 7f90797fa640 1 -- 172.21.15.106:0/1051830130 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f9074095e10 con 0x7f907c15a930 2024-04-25T10:34:52.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.786+0000 7f907bfff640 1 --2- 172.21.15.106:0/1051830130 >> [v2:172.21.15.106:6800/1240717079,v1:172.21.15.106:6801/1240717079] conn(0x7f906007c880 0x7f906007ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:52.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.786+0000 7f908083b640 1 --2- 172.21.15.106:0/1051830130 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f90600744b0 0x7f90600769c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9070009aa0 tx=0x7f9070009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:52.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.786+0000 7f907bfff640 1 --2- 172.21.15.106:0/1051830130 >> [v2:172.21.15.106:6800/1240717079,v1:172.21.15.106:6801/1240717079] conn(0x7f906007c880 0x7f906007ed50 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:52.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.787+0000 7f90797fa640 1 -- 172.21.15.106:0/1051830130 <== osd.0 v2:172.21.15.106:6800/1240717079 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f906007f420 con 0x7f906007c880 2024-04-25T10:34:52.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.800+0000 7f9082a9f640 1 -- 172.21.15.106:0/1051830130 --> [v2:172.21.15.106:6800/1240717079,v1:172.21.15.106:6801/1240717079] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f907c1f50b0 con 0x7f906007c880 2024-04-25T10:34:52.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.800+0000 7f90797fa640 1 -- 172.21.15.106:0/1051830130 <== osd.0 v2:172.21.15.106:6800/1240717079 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f907c1f50b0 con 0x7f906007c880 2024-04-25T10:34:52.803 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-25T10:34:52.803 INFO:teuthology.orchestra.run.smithi106.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:34:52.803 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-25T10:34:52.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.800+0000 7f9082a9f640 1 -- 172.21.15.106:0/1051830130 >> [v2:172.21.15.106:6800/1240717079,v1:172.21.15.106:6801/1240717079] conn(0x7f906007c880 msgr2=0x7f906007ed50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:52.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.800+0000 7f9082a9f640 1 --2- 172.21.15.106:0/1051830130 >> [v2:172.21.15.106:6800/1240717079,v1:172.21.15.106:6801/1240717079] conn(0x7f906007c880 0x7f906007ed50 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:52.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.800+0000 7f9082a9f640 1 -- 172.21.15.106:0/1051830130 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f90600744b0 msgr2=0x7f90600769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:52.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.800+0000 7f9082a9f640 1 --2- 172.21.15.106:0/1051830130 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f90600744b0 0x7f90600769c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9070009aa0 tx=0x7f9070009190 comp rx=0 tx=0).stop 2024-04-25T10:34:52.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.800+0000 7f9082a9f640 1 -- 172.21.15.106:0/1051830130 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f907c15a930 msgr2=0x7f907c146d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:52.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.800+0000 7f9082a9f640 1 --2- 172.21.15.106:0/1051830130 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f907c15a930 0x7f907c146d20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f907400c6b0 tx=0x7f907400cb70 comp rx=0 tx=0).stop 2024-04-25T10:34:52.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.800+0000 7f908103c640 1 -- 172.21.15.106:0/1051830130 reap_dead start 2024-04-25T10:34:52.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.801+0000 7f9082a9f640 1 -- 172.21.15.106:0/1051830130 shutdown_connections 2024-04-25T10:34:52.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.801+0000 7f9082a9f640 1 -- 172.21.15.106:0/1051830130 >> 172.21.15.106:0/1051830130 conn(0x7f907c0cffd0 msgr2=0x7f907c0d03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:52.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.801+0000 7f9082a9f640 1 -- 172.21.15.106:0/1051830130 shutdown_connections 2024-04-25T10:34:52.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.801+0000 7f9082a9f640 1 -- 172.21.15.106:0/1051830130 wait complete. 2024-04-25T10:34:52.814 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-25T10:34:52.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.921+0000 7f09d2cb1640 1 Processor -- start 2024-04-25T10:34:52.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.921+0000 7f09d2cb1640 1 -- start start 2024-04-25T10:34:52.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.923+0000 7f09d2cb1640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09cc15d540 0x7f09cc156ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:52.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.923+0000 7f09d2cb1640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09cc0ce630 con 0x7f09cc0c7e30 2024-04-25T10:34:52.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.923+0000 7f09d2cb1640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09cc0c7980 con 0x7f09cc0c8d90 2024-04-25T10:34:52.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.923+0000 7f09d2cb1640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f09cc0c7c60 con 0x7f09cc15d540 2024-04-25T10:34:52.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.923+0000 7f09d0a4d640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f09cc0c7e30 0x7f09cc06d000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:39726/0 (socket says 172.21.15.106:39726) 2024-04-25T10:34:52.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.923+0000 7f09d0a4d640 1 -- 172.21.15.106:0/2559170854 learned_addr learned my addr 172.21.15.106:0/2559170854 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:34:52.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.923+0000 7f09d124e640 1 --2- 172.21.15.106:0/2559170854 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09cc15d540 0x7f09cc156ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:52.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.923+0000 7f09cb7fe640 1 -- 172.21.15.106:0/2559170854 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3036304178 0 0) 0x7f09cc0ce630 con 0x7f09cc0c7e30 2024-04-25T10:34:52.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.923+0000 7f09cb7fe640 1 -- 172.21.15.106:0/2559170854 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f09bc003520 con 0x7f09cc0c7e30 2024-04-25T10:34:52.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.923+0000 7f09cb7fe640 1 -- 172.21.15.106:0/2559170854 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 392311907 0 0) 0x7f09cc0c7980 con 0x7f09cc0c8d90 2024-04-25T10:34:52.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.923+0000 7f09cb7fe640 1 -- 172.21.15.106:0/2559170854 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f09cc0ce630 con 0x7f09cc0c8d90 2024-04-25T10:34:52.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.923+0000 7f09d124e640 1 -- 172.21.15.106:0/2559170854 >> v1:172.21.15.134:6789/0 conn(0x7f09cc0c8d90 legacy=0x7f09cc15ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:52.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.923+0000 7f09d124e640 1 -- 172.21.15.106:0/2559170854 >> v1:172.21.15.106:6789/0 conn(0x7f09cc0c7e30 legacy=0x7f09cc06d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:52.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.923+0000 7f09d124e640 1 -- 172.21.15.106:0/2559170854 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09cc0c7010 con 0x7f09cc15d540 2024-04-25T10:34:52.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.924+0000 7f09d124e640 1 --2- 172.21.15.106:0/2559170854 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09cc15d540 0x7f09cc156ef0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f09c4009dc0 tx=0x7f09c402a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2304645fdea28952 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:52.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.924+0000 7f09cb7fe640 1 -- 172.21.15.106:0/2559170854 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c402ae60 con 0x7f09cc15d540 2024-04-25T10:34:52.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.924+0000 7f09cb7fe640 1 -- 172.21.15.106:0/2559170854 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09c4007740 con 0x7f09cc15d540 2024-04-25T10:34:52.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.924+0000 7f09cb7fe640 1 -- 172.21.15.106:0/2559170854 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c4007af0 con 0x7f09cc15d540 2024-04-25T10:34:52.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.925+0000 7f09d2cb1640 1 -- 172.21.15.106:0/2559170854 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09cc15d540 msgr2=0x7f09cc156ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:52.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.925+0000 7f09d2cb1640 1 --2- 172.21.15.106:0/2559170854 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09cc15d540 0x7f09cc156ef0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f09c4009dc0 tx=0x7f09c402a640 comp rx=0 tx=0).stop 2024-04-25T10:34:52.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.925+0000 7f09d2cb1640 1 -- 172.21.15.106:0/2559170854 shutdown_connections 2024-04-25T10:34:52.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.925+0000 7f09d2cb1640 1 --2- 172.21.15.106:0/2559170854 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09cc15d540 0x7f09cc156ef0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:52.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.925+0000 7f09d2cb1640 1 -- 172.21.15.106:0/2559170854 >> 172.21.15.106:0/2559170854 conn(0x7f09cc068b10 msgr2=0x7f09cc0d1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:52.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.925+0000 7f09d2cb1640 1 -- 172.21.15.106:0/2559170854 shutdown_connections 2024-04-25T10:34:52.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.925+0000 7f09d2cb1640 1 -- 172.21.15.106:0/2559170854 wait complete. 2024-04-25T10:34:52.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.927+0000 7f09d2cb1640 1 Processor -- start 2024-04-25T10:34:52.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.927+0000 7f09d2cb1640 1 -- start start 2024-04-25T10:34:52.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.928+0000 7f09d2cb1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09cc0c7e30 0x7f09cc0d0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:52.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.928+0000 7f09d2cb1640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09cc0c8d90 0x7f09cc0d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:52.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.928+0000 7f09d2cb1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09cc15d540 0x7f09cc1596e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:52.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.928+0000 7f09d2cb1640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f09cc06d890 con 0x7f09cc0c7e30 2024-04-25T10:34:52.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.928+0000 7f09d2cb1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f09cc06d710 con 0x7f09cc15d540 2024-04-25T10:34:52.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.928+0000 7f09d2cb1640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f09cc0b4ce0 con 0x7f09cc0c8d90 2024-04-25T10:34:52.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.928+0000 7f09d0a4d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09cc0c7e30 0x7f09cc0d0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:52.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.928+0000 7f09d0a4d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09cc0c7e30 0x7f09cc0d0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56040/0 (socket says 172.21.15.106:56040) 2024-04-25T10:34:52.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.928+0000 7f09d0a4d640 1 -- 172.21.15.106:0/3434090889 learned_addr learned my addr 172.21.15.106:0/3434090889 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:34:52.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.928+0000 7f09cbfff640 1 --2- 172.21.15.106:0/3434090889 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09cc0c8d90 0x7f09cc0d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:52.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.928+0000 7f09d124e640 1 --2- 172.21.15.106:0/3434090889 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09cc15d540 0x7f09cc1596e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:52.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.929+0000 7f09d0a4d640 1 -- 172.21.15.106:0/3434090889 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09cc0c8d90 msgr2=0x7f09cc0d0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:52.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.929+0000 7f09d0a4d640 1 --2- 172.21.15.106:0/3434090889 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09cc0c8d90 0x7f09cc0d0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:52.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.929+0000 7f09d0a4d640 1 -- 172.21.15.106:0/3434090889 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09cc15d540 msgr2=0x7f09cc1596e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:52.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.929+0000 7f09d0a4d640 1 --2- 172.21.15.106:0/3434090889 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09cc15d540 0x7f09cc1596e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:52.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.929+0000 7f09d0a4d640 1 -- 172.21.15.106:0/3434090889 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09cc155a50 con 0x7f09cc0c7e30 2024-04-25T10:34:52.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.929+0000 7f09d0a4d640 1 --2- 172.21.15.106:0/3434090889 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09cc0c7e30 0x7f09cc0d0780 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f09c000c410 tx=0x7f09c000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:52.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.929+0000 7f09c97fa640 1 -- 172.21.15.106:0/3434090889 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c0017070 con 0x7f09cc0c7e30 2024-04-25T10:34:52.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.929+0000 7f09c97fa640 1 -- 172.21.15.106:0/3434090889 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09c0005ce0 con 0x7f09cc0c7e30 2024-04-25T10:34:52.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.929+0000 7f09d2cb1640 1 -- 172.21.15.106:0/3434090889 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09cc159c20 con 0x7f09cc0c7e30 2024-04-25T10:34:52.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.929+0000 7f09c97fa640 1 -- 172.21.15.106:0/3434090889 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c00072d0 con 0x7f09cc0c7e30 2024-04-25T10:34:52.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.929+0000 7f09d2cb1640 1 -- 172.21.15.106:0/3434090889 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09cc0d1c00 con 0x7f09cc0c7e30 2024-04-25T10:34:52.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.930+0000 7f09d2cb1640 1 -- 172.21.15.106:0/3434090889 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09cc06a3d0 con 0x7f09cc0c7e30 2024-04-25T10:34:52.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.931+0000 7f09c97fa640 1 -- 172.21.15.106:0/3434090889 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f09c001a070 con 0x7f09cc0c7e30 2024-04-25T10:34:52.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.931+0000 7f09c97fa640 1 --2- 172.21.15.106:0/3434090889 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f09b00744b0 0x7f09b00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:52.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.931+0000 7f09c97fa640 1 -- 172.21.15.106:0/3434090889 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f09c0094500 con 0x7f09cc0c7e30 2024-04-25T10:34:52.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.931+0000 7f09cbfff640 1 --2- 172.21.15.106:0/3434090889 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f09b00744b0 0x7f09b00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:52.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.932+0000 7f09cbfff640 1 --2- 172.21.15.106:0/3434090889 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f09b00744b0 0x7f09b00769c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f09cc1541e0 tx=0x7f09b8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:52.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:52.936+0000 7f09c97fa640 1 -- 172.21.15.106:0/3434090889 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f09cc155a50 con 0x7f09cc0c7e30 2024-04-25T10:34:53.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.134+0000 7f09d2cb1640 1 -- 172.21.15.106:0/3434090889 --> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f09cc15a5a0 con 0x7f09b00744b0 2024-04-25T10:34:53.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.134+0000 7f09c97fa640 1 -- 172.21.15.106:0/3434090889 <== mgr.4107 v2:172.21.15.106:6832/1744210167 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f09cc15a5a0 con 0x7f09b00744b0 2024-04-25T10:34:53.137 INFO:teuthology.orchestra.run.smithi106.stderr:instructed osd(s) 0 to deep-scrub 2024-04-25T10:34:53.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.136+0000 7f09d2cb1640 1 -- 172.21.15.106:0/3434090889 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f09b00744b0 msgr2=0x7f09b00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:53.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.136+0000 7f09d2cb1640 1 --2- 172.21.15.106:0/3434090889 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f09b00744b0 0x7f09b00769c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f09cc1541e0 tx=0x7f09b8009040 comp rx=0 tx=0).stop 2024-04-25T10:34:53.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.136+0000 7f09d2cb1640 1 -- 172.21.15.106:0/3434090889 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09cc0c7e30 msgr2=0x7f09cc0d0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:53.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.136+0000 7f09d2cb1640 1 --2- 172.21.15.106:0/3434090889 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09cc0c7e30 0x7f09cc0d0780 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f09c000c410 tx=0x7f09c000c8d0 comp rx=0 tx=0).stop 2024-04-25T10:34:53.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.136+0000 7f09d2cb1640 1 -- 172.21.15.106:0/3434090889 shutdown_connections 2024-04-25T10:34:53.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.136+0000 7f09d2cb1640 1 --2- 172.21.15.106:0/3434090889 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f09b00744b0 0x7f09b00769c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:53.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.137+0000 7f09d2cb1640 1 --2- 172.21.15.106:0/3434090889 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09cc15d540 0x7f09cc1596e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:53.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.137+0000 7f09d2cb1640 1 --2- 172.21.15.106:0/3434090889 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09cc0c8d90 0x7f09cc0d0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:53.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.137+0000 7f09d2cb1640 1 --2- 172.21.15.106:0/3434090889 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09cc0c7e30 0x7f09cc0d0780 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:53.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.137+0000 7f09d2cb1640 1 -- 172.21.15.106:0/3434090889 >> 172.21.15.106:0/3434090889 conn(0x7f09cc068b10 msgr2=0x7f09cc06d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:53.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.137+0000 7f09d2cb1640 1 -- 172.21.15.106:0/3434090889 shutdown_connections 2024-04-25T10:34:53.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.137+0000 7f09d2cb1640 1 -- 172.21.15.106:0/3434090889 wait complete. 2024-04-25T10:34:53.151 INFO:tasks.ceph:Scrubbing osd.1 2024-04-25T10:34:53.151 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:34:53.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.264+0000 7f5b5b29a640 1 Processor -- start 2024-04-25T10:34:53.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.264+0000 7f5b5b29a640 1 -- start start 2024-04-25T10:34:53.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.266+0000 7f5b5b29a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5b5415cec0 0x7f5b5415a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:53.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.266+0000 7f5b5b29a640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f5b54073d40 0x7f5b54153a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:53.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.266+0000 7f5b5b29a640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f5b540cab50 0x7f5b540caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:53.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.266+0000 7f5b5b29a640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f5b54072f80 con 0x7f5b540cab50 2024-04-25T10:34:53.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.266+0000 7f5b5b29a640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f5b5415a840 con 0x7f5b54073d40 2024-04-25T10:34:53.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.266+0000 7f5b5b29a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f5b540742c0 con 0x7f5b5415cec0 2024-04-25T10:34:53.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.266+0000 7f5b59837640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f5b540cab50 0x7f5b540caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:53.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.266+0000 7f5b59837640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f5b540cab50 0x7f5b540caf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56044/0 (socket says 172.21.15.106:56044) 2024-04-25T10:34:53.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.266+0000 7f5b59837640 1 -- 172.21.15.106:0/3282617881 learned_addr learned my addr 172.21.15.106:0/3282617881 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:34:53.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.266+0000 7f5b59036640 1 --2- 172.21.15.106:0/3282617881 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5b5415cec0 0x7f5b5415a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:53.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.266+0000 7f5b58835640 1 --2- 172.21.15.106:0/3282617881 >> v2:172.21.15.134:3300/0 conn(0x7f5b54073d40 0x7f5b54153a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:53.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.266+0000 7f5b59837640 1 -- 172.21.15.106:0/3282617881 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5b5415cec0 msgr2=0x7f5b5415a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:53.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.266+0000 7f5b59837640 1 --2- 172.21.15.106:0/3282617881 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5b5415cec0 0x7f5b5415a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:53.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.267+0000 7f5b59837640 1 -- 172.21.15.106:0/3282617881 >> v2:172.21.15.134:3300/0 conn(0x7f5b54073d40 msgr2=0x7f5b54153a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:53.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.267+0000 7f5b59837640 1 --2- 172.21.15.106:0/3282617881 >> v2:172.21.15.134:3300/0 conn(0x7f5b54073d40 0x7f5b54153a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:53.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.267+0000 7f5b59837640 1 -- 172.21.15.106:0/3282617881 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b540745a0 con 0x7f5b540cab50 2024-04-25T10:34:53.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.268+0000 7f5b59837640 1 --2- 172.21.15.106:0/3282617881 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b540cab50 0x7f5b540caf60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5b54071330 tx=0x7f5b4c02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=141a04401fb250c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:53.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.268+0000 7f5b4bfff640 1 -- 172.21.15.106:0/3282617881 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c036070 con 0x7f5b540cab50 2024-04-25T10:34:53.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.268+0000 7f5b4bfff640 1 -- 172.21.15.106:0/3282617881 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b4c031600 con 0x7f5b540cab50 2024-04-25T10:34:53.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.268+0000 7f5b4bfff640 1 -- 172.21.15.106:0/3282617881 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c0319d0 con 0x7f5b540cab50 2024-04-25T10:34:53.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.269+0000 7f5b5b29a640 1 -- 172.21.15.106:0/3282617881 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b540cab50 msgr2=0x7f5b540caf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:53.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.269+0000 7f5b5b29a640 1 --2- 172.21.15.106:0/3282617881 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b540cab50 0x7f5b540caf60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5b54071330 tx=0x7f5b4c02a730 comp rx=0 tx=0).stop 2024-04-25T10:34:53.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.269+0000 7f5b5b29a640 1 -- 172.21.15.106:0/3282617881 shutdown_connections 2024-04-25T10:34:53.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.269+0000 7f5b5b29a640 1 --2- 172.21.15.106:0/3282617881 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b540cab50 0x7f5b540caf60 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:53.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.269+0000 7f5b5b29a640 1 --2- 172.21.15.106:0/3282617881 >> v2:172.21.15.134:3300/0 conn(0x7f5b54073d40 0x7f5b54153a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:53.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.269+0000 7f5b5b29a640 1 --2- 172.21.15.106:0/3282617881 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5b5415cec0 0x7f5b5415a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:53.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.269+0000 7f5b5b29a640 1 -- 172.21.15.106:0/3282617881 >> 172.21.15.106:0/3282617881 conn(0x7f5b54070570 msgr2=0x7f5b54070980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:53.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.269+0000 7f5b5b29a640 1 -- 172.21.15.106:0/3282617881 shutdown_connections 2024-04-25T10:34:53.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.269+0000 7f5b5b29a640 1 -- 172.21.15.106:0/3282617881 wait complete. 2024-04-25T10:34:53.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.270+0000 7f5b5b29a640 1 Processor -- start 2024-04-25T10:34:53.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.271+0000 7f5b5b29a640 1 -- start start 2024-04-25T10:34:53.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.272+0000 7f5b5b29a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5b54073d40 0x7f5b541f48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:53.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.272+0000 7f5b5b29a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b540cab50 0x7f5b540ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:53.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.272+0000 7f5b5b29a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b5415cec0 0x7f5b541558f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:53.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.272+0000 7f5b5b29a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5b54072c20 con 0x7f5b540cab50 2024-04-25T10:34:53.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.272+0000 7f5b5b29a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5b54159de0 con 0x7f5b5415cec0 2024-04-25T10:34:53.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.272+0000 7f5b5b29a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f5b54153f80 con 0x7f5b54073d40 2024-04-25T10:34:53.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.272+0000 7f5b58835640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b540cab50 0x7f5b540ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:53.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.272+0000 7f5b58835640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b540cab50 0x7f5b540ce0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56058/0 (socket says 172.21.15.106:56058) 2024-04-25T10:34:53.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.272+0000 7f5b58835640 1 -- 172.21.15.106:0/750258375 learned_addr learned my addr 172.21.15.106:0/750258375 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:34:53.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.272+0000 7f5b58835640 1 -- 172.21.15.106:0/750258375 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5b54073d40 msgr2=0x7f5b541f48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:53.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.272+0000 7f5b58835640 1 --2- 172.21.15.106:0/750258375 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5b54073d40 0x7f5b541f48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:53.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.272+0000 7f5b58835640 1 -- 172.21.15.106:0/750258375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b5415cec0 msgr2=0x7f5b541558f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:53.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.272+0000 7f5b59837640 1 --2- 172.21.15.106:0/750258375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b5415cec0 0x7f5b541558f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:53.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.272+0000 7f5b58835640 1 --2- 172.21.15.106:0/750258375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b5415cec0 0x7f5b541558f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:53.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.272+0000 7f5b58835640 1 -- 172.21.15.106:0/750258375 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b541f4e20 con 0x7f5b540cab50 2024-04-25T10:34:53.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.272+0000 7f5b59837640 1 --2- 172.21.15.106:0/750258375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b5415cec0 0x7f5b541558f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:34:53.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.272+0000 7f5b58835640 1 --2- 172.21.15.106:0/750258375 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b540cab50 0x7f5b540ce0e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5b4400e870 tx=0x7f5b4400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:53.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.272+0000 7f5b49ffb640 1 -- 172.21.15.106:0/750258375 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b44017070 con 0x7f5b540cab50 2024-04-25T10:34:53.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.272+0000 7f5b49ffb640 1 -- 172.21.15.106:0/750258375 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b4400c940 con 0x7f5b540cab50 2024-04-25T10:34:53.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.272+0000 7f5b5b29a640 1 -- 172.21.15.106:0/750258375 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b541e2320 con 0x7f5b540cab50 2024-04-25T10:34:53.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.272+0000 7f5b49ffb640 1 -- 172.21.15.106:0/750258375 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b4400cd10 con 0x7f5b540cab50 2024-04-25T10:34:53.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.273+0000 7f5b5b29a640 1 -- 172.21.15.106:0/750258375 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b541f40e0 con 0x7f5b540cab50 2024-04-25T10:34:53.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.274+0000 7f5b5b29a640 1 -- 172.21.15.106:0/750258375 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5b5406a3d0 con 0x7f5b540cab50 2024-04-25T10:34:53.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.274+0000 7f5b49ffb640 1 -- 172.21.15.106:0/750258375 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5b4401a070 con 0x7f5b540cab50 2024-04-25T10:34:53.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.275+0000 7f5b49ffb640 1 --2- 172.21.15.106:0/750258375 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f5b380744b0 0x7f5b380769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:53.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.275+0000 7f5b49ffb640 1 -- 172.21.15.106:0/750258375 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f5b44095110 con 0x7f5b540cab50 2024-04-25T10:34:53.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.275+0000 7f5b59036640 1 --2- 172.21.15.106:0/750258375 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f5b380744b0 0x7f5b380769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:53.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.275+0000 7f5b49ffb640 1 --2- 172.21.15.106:0/750258375 >> [v2:172.21.15.106:6824/4089653051,v1:172.21.15.106:6825/4089653051] conn(0x7f5b3807c880 0x7f5b3807ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:53.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.275+0000 7f5b49ffb640 1 -- 172.21.15.106:0/750258375 --> [v2:172.21.15.106:6824/4089653051,v1:172.21.15.106:6825/4089653051] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5b3807f420 con 0x7f5b3807c880 2024-04-25T10:34:53.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.275+0000 7f5b49ffb640 1 -- 172.21.15.106:0/750258375 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f5b4409c050 con 0x7f5b540cab50 2024-04-25T10:34:53.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.275+0000 7f5b59837640 1 --2- 172.21.15.106:0/750258375 >> [v2:172.21.15.106:6824/4089653051,v1:172.21.15.106:6825/4089653051] conn(0x7f5b3807c880 0x7f5b3807ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:53.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.275+0000 7f5b59036640 1 --2- 172.21.15.106:0/750258375 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f5b380744b0 0x7f5b380769c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5b50006a90 tx=0x7f5b50006a20 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:53.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.276+0000 7f5b59837640 1 --2- 172.21.15.106:0/750258375 >> [v2:172.21.15.106:6824/4089653051,v1:172.21.15.106:6825/4089653051] conn(0x7f5b3807c880 0x7f5b3807ed50 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:53.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.276+0000 7f5b49ffb640 1 -- 172.21.15.106:0/750258375 <== osd.1 v2:172.21.15.106:6824/4089653051 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5b3807f420 con 0x7f5b3807c880 2024-04-25T10:34:53.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.289+0000 7f5b5b29a640 1 -- 172.21.15.106:0/750258375 --> [v2:172.21.15.106:6824/4089653051,v1:172.21.15.106:6825/4089653051] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f5b540cb2e0 con 0x7f5b3807c880 2024-04-25T10:34:53.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.289+0000 7f5b49ffb640 1 -- 172.21.15.106:0/750258375 <== osd.1 v2:172.21.15.106:6824/4089653051 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f5b540cb2e0 con 0x7f5b3807c880 2024-04-25T10:34:53.292 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-25T10:34:53.292 INFO:teuthology.orchestra.run.smithi106.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:34:53.292 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-25T10:34:53.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.290+0000 7f5b5b29a640 1 -- 172.21.15.106:0/750258375 >> [v2:172.21.15.106:6824/4089653051,v1:172.21.15.106:6825/4089653051] conn(0x7f5b3807c880 msgr2=0x7f5b3807ed50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:53.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.290+0000 7f5b5b29a640 1 --2- 172.21.15.106:0/750258375 >> [v2:172.21.15.106:6824/4089653051,v1:172.21.15.106:6825/4089653051] conn(0x7f5b3807c880 0x7f5b3807ed50 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:53.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.290+0000 7f5b5b29a640 1 -- 172.21.15.106:0/750258375 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f5b380744b0 msgr2=0x7f5b380769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:53.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.290+0000 7f5b5b29a640 1 --2- 172.21.15.106:0/750258375 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f5b380744b0 0x7f5b380769c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5b50006a90 tx=0x7f5b50006a20 comp rx=0 tx=0).stop 2024-04-25T10:34:53.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.290+0000 7f5b5b29a640 1 -- 172.21.15.106:0/750258375 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b540cab50 msgr2=0x7f5b540ce0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:53.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.290+0000 7f5b5b29a640 1 --2- 172.21.15.106:0/750258375 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b540cab50 0x7f5b540ce0e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5b4400e870 tx=0x7f5b4400ed30 comp rx=0 tx=0).stop 2024-04-25T10:34:53.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.290+0000 7f5b59837640 1 -- 172.21.15.106:0/750258375 reap_dead start 2024-04-25T10:34:53.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.290+0000 7f5b5b29a640 1 -- 172.21.15.106:0/750258375 shutdown_connections 2024-04-25T10:34:53.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.290+0000 7f5b5b29a640 1 -- 172.21.15.106:0/750258375 >> 172.21.15.106:0/750258375 conn(0x7f5b54070570 msgr2=0x7f5b54070980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:53.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.290+0000 7f5b5b29a640 1 -- 172.21.15.106:0/750258375 shutdown_connections 2024-04-25T10:34:53.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.290+0000 7f5b5b29a640 1 -- 172.21.15.106:0/750258375 wait complete. 2024-04-25T10:34:53.303 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-25T10:34:53.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.412+0000 7f68f6a92640 1 Processor -- start 2024-04-25T10:34:53.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.412+0000 7f68f6a92640 1 -- start start 2024-04-25T10:34:53.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.413+0000 7f68f6a92640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f68f00c7e30 0x7f68f00c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:53.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.413+0000 7f68f6a92640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f68f0156ef0 0x7f68f00b4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:53.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.413+0000 7f68f6a92640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f68f006d000 0x7f68f006d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:53.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.413+0000 7f68f6a92640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f68f00c7c60 con 0x7f68f006d000 2024-04-25T10:34:53.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.413+0000 7f68f6a92640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f68f00ce630 con 0x7f68f00c7e30 2024-04-25T10:34:53.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.413+0000 7f68f6a92640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f68f00c7980 con 0x7f68f0156ef0 2024-04-25T10:34:53.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.413+0000 7f68f502f640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f68f006d000 0x7f68f006d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:53.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.413+0000 7f68f502f640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f68f006d000 0x7f68f006d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56062/0 (socket says 172.21.15.106:56062) 2024-04-25T10:34:53.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.413+0000 7f68f502f640 1 -- 172.21.15.106:0/1830364954 learned_addr learned my addr 172.21.15.106:0/1830364954 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:34:53.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.414+0000 7f68f482e640 1 --2- 172.21.15.106:0/1830364954 >> v2:172.21.15.134:3300/0 conn(0x7f68f00c7e30 0x7f68f00c8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:53.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.414+0000 7f68effff640 1 --2- 172.21.15.106:0/1830364954 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f68f0156ef0 0x7f68f00b4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:53.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.414+0000 7f68f502f640 1 -- 172.21.15.106:0/1830364954 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f68f0156ef0 msgr2=0x7f68f00b4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:53.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.414+0000 7f68f502f640 1 --2- 172.21.15.106:0/1830364954 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f68f0156ef0 0x7f68f00b4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:53.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.414+0000 7f68f502f640 1 -- 172.21.15.106:0/1830364954 >> v2:172.21.15.134:3300/0 conn(0x7f68f00c7e30 msgr2=0x7f68f00c8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:53.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.414+0000 7f68f502f640 1 --2- 172.21.15.106:0/1830364954 >> v2:172.21.15.134:3300/0 conn(0x7f68f00c7e30 0x7f68f00c8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:53.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.414+0000 7f68f502f640 1 -- 172.21.15.106:0/1830364954 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68f00c7010 con 0x7f68f006d000 2024-04-25T10:34:53.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.414+0000 7f68f502f640 1 --2- 172.21.15.106:0/1830364954 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f68f006d000 0x7f68f006d410 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f68f008fcb0 tx=0x7f68e802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a19296ddb795a773 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:53.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.414+0000 7f68ef7fe640 1 -- 172.21.15.106:0/1830364954 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68e8036070 con 0x7f68f006d000 2024-04-25T10:34:53.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.415+0000 7f68ef7fe640 1 -- 172.21.15.106:0/1830364954 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68e8031600 con 0x7f68f006d000 2024-04-25T10:34:53.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.415+0000 7f68ef7fe640 1 -- 172.21.15.106:0/1830364954 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68e80319d0 con 0x7f68f006d000 2024-04-25T10:34:53.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.415+0000 7f68f6a92640 1 -- 172.21.15.106:0/1830364954 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f68f006d000 msgr2=0x7f68f006d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:53.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.415+0000 7f68f6a92640 1 --2- 172.21.15.106:0/1830364954 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f68f006d000 0x7f68f006d410 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f68f008fcb0 tx=0x7f68e802a610 comp rx=0 tx=0).stop 2024-04-25T10:34:53.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.415+0000 7f68f6a92640 1 -- 172.21.15.106:0/1830364954 shutdown_connections 2024-04-25T10:34:53.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.415+0000 7f68f6a92640 1 --2- 172.21.15.106:0/1830364954 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f68f006d000 0x7f68f006d410 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:53.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.415+0000 7f68f6a92640 1 --2- 172.21.15.106:0/1830364954 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f68f0156ef0 0x7f68f00b4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:53.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.415+0000 7f68f6a92640 1 --2- 172.21.15.106:0/1830364954 >> v2:172.21.15.134:3300/0 conn(0x7f68f00c7e30 0x7f68f00c8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:53.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.415+0000 7f68f6a92640 1 -- 172.21.15.106:0/1830364954 >> 172.21.15.106:0/1830364954 conn(0x7f68f0068b10 msgr2=0x7f68f00d1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:53.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.415+0000 7f68f6a92640 1 -- 172.21.15.106:0/1830364954 shutdown_connections 2024-04-25T10:34:53.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.415+0000 7f68f6a92640 1 -- 172.21.15.106:0/1830364954 wait complete. 2024-04-25T10:34:53.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.417+0000 7f68f6a92640 1 Processor -- start 2024-04-25T10:34:53.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.417+0000 7f68f6a92640 1 -- start start 2024-04-25T10:34:53.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.418+0000 7f68f6a92640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68f006d000 0x7f68f0159b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:53.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.418+0000 7f68f6a92640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f68f00c7e30 0x7f68f0151a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:53.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.418+0000 7f68f6a92640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f68f0156ef0 0x7f68f0151fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:53.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.418+0000 7f68f6a92640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f68f015cfb0 con 0x7f68f0156ef0 2024-04-25T10:34:53.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.418+0000 7f68f6a92640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f68f015ce30 con 0x7f68f006d000 2024-04-25T10:34:53.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.418+0000 7f68f6a92640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f68f015d130 con 0x7f68f00c7e30 2024-04-25T10:34:53.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.418+0000 7f68f502f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f68f0156ef0 0x7f68f0151fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:53.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.418+0000 7f68f502f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f68f0156ef0 0x7f68f0151fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56072/0 (socket says 172.21.15.106:56072) 2024-04-25T10:34:53.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.418+0000 7f68f502f640 1 -- 172.21.15.106:0/3588358896 learned_addr learned my addr 172.21.15.106:0/3588358896 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:34:53.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.419+0000 7f68f482e640 1 --2- 172.21.15.106:0/3588358896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68f006d000 0x7f68f0159b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:53.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.419+0000 7f68effff640 1 --2- 172.21.15.106:0/3588358896 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f68f00c7e30 0x7f68f0151a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:53.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.419+0000 7f68f502f640 1 -- 172.21.15.106:0/3588358896 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f68f00c7e30 msgr2=0x7f68f0151a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:53.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.419+0000 7f68f502f640 1 --2- 172.21.15.106:0/3588358896 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f68f00c7e30 0x7f68f0151a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:53.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.419+0000 7f68f502f640 1 -- 172.21.15.106:0/3588358896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68f006d000 msgr2=0x7f68f0159b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:53.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.419+0000 7f68f502f640 1 --2- 172.21.15.106:0/3588358896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68f006d000 0x7f68f0159b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:53.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.419+0000 7f68f502f640 1 -- 172.21.15.106:0/3588358896 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68f01f39d0 con 0x7f68f0156ef0 2024-04-25T10:34:53.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.419+0000 7f68effff640 1 --2- 172.21.15.106:0/3588358896 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f68f00c7e30 0x7f68f0151a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:34:53.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.419+0000 7f68f502f640 1 --2- 172.21.15.106:0/3588358896 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f68f0156ef0 0x7f68f0151fb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f68e8009f40 tx=0x7f68e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:53.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.419+0000 7f68ed7fa640 1 -- 172.21.15.106:0/3588358896 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68e8036050 con 0x7f68f0156ef0 2024-04-25T10:34:53.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.419+0000 7f68ed7fa640 1 -- 172.21.15.106:0/3588358896 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68e8042070 con 0x7f68f0156ef0 2024-04-25T10:34:53.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.419+0000 7f68ed7fa640 1 -- 172.21.15.106:0/3588358896 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68e8009510 con 0x7f68f0156ef0 2024-04-25T10:34:53.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.419+0000 7f68f6a92640 1 -- 172.21.15.106:0/3588358896 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68f01f3f10 con 0x7f68f0156ef0 2024-04-25T10:34:53.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.419+0000 7f68f6a92640 1 -- 172.21.15.106:0/3588358896 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68f01f6640 con 0x7f68f0156ef0 2024-04-25T10:34:53.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.420+0000 7f68f6a92640 1 -- 172.21.15.106:0/3588358896 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68f00b79f0 con 0x7f68f0156ef0 2024-04-25T10:34:53.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.421+0000 7f68ed7fa640 1 -- 172.21.15.106:0/3588358896 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f68e8007bd0 con 0x7f68f0156ef0 2024-04-25T10:34:53.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.421+0000 7f68ed7fa640 1 --2- 172.21.15.106:0/3588358896 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f68d40744b0 0x7f68d40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:53.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.421+0000 7f68f482e640 1 --2- 172.21.15.106:0/3588358896 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f68d40744b0 0x7f68d40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:53.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.421+0000 7f68ed7fa640 1 -- 172.21.15.106:0/3588358896 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f68e80b6040 con 0x7f68f0156ef0 2024-04-25T10:34:53.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.422+0000 7f68f482e640 1 --2- 172.21.15.106:0/3588358896 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f68d40744b0 0x7f68d40769c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f68e4003610 tx=0x7f68e4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:53.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.426+0000 7f68ed7fa640 1 -- 172.21.15.106:0/3588358896 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f68f00b79f0 con 0x7f68f0156ef0 2024-04-25T10:34:53.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.608+0000 7f68f6a92640 1 -- 172.21.15.106:0/3588358896 --> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f68f00b5350 con 0x7f68d40744b0 2024-04-25T10:34:53.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.608+0000 7f68ed7fa640 1 -- 172.21.15.106:0/3588358896 <== mgr.4107 v2:172.21.15.106:6832/1744210167 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f68f00b5350 con 0x7f68d40744b0 2024-04-25T10:34:53.611 INFO:teuthology.orchestra.run.smithi106.stderr:instructed osd(s) 1 to deep-scrub 2024-04-25T10:34:53.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.610+0000 7f68f6a92640 1 -- 172.21.15.106:0/3588358896 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f68d40744b0 msgr2=0x7f68d40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:53.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.610+0000 7f68f6a92640 1 --2- 172.21.15.106:0/3588358896 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f68d40744b0 0x7f68d40769c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f68e4003610 tx=0x7f68e4008040 comp rx=0 tx=0).stop 2024-04-25T10:34:53.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.610+0000 7f68f6a92640 1 -- 172.21.15.106:0/3588358896 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f68f0156ef0 msgr2=0x7f68f0151fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:53.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.610+0000 7f68f6a92640 1 --2- 172.21.15.106:0/3588358896 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f68f0156ef0 0x7f68f0151fb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f68e8009f40 tx=0x7f68e8004a40 comp rx=0 tx=0).stop 2024-04-25T10:34:53.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.610+0000 7f68f6a92640 1 -- 172.21.15.106:0/3588358896 shutdown_connections 2024-04-25T10:34:53.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.610+0000 7f68f6a92640 1 --2- 172.21.15.106:0/3588358896 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f68d40744b0 0x7f68d40769c0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:53.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.610+0000 7f68f6a92640 1 --2- 172.21.15.106:0/3588358896 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f68f0156ef0 0x7f68f0151fb0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:53.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.610+0000 7f68f6a92640 1 --2- 172.21.15.106:0/3588358896 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f68f00c7e30 0x7f68f0151a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:53.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.610+0000 7f68f6a92640 1 --2- 172.21.15.106:0/3588358896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68f006d000 0x7f68f0159b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:53.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.610+0000 7f68f6a92640 1 -- 172.21.15.106:0/3588358896 >> 172.21.15.106:0/3588358896 conn(0x7f68f0068b10 msgr2=0x7f68f0154930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:53.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.611+0000 7f68f6a92640 1 -- 172.21.15.106:0/3588358896 shutdown_connections 2024-04-25T10:34:53.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.611+0000 7f68f6a92640 1 -- 172.21.15.106:0/3588358896 wait complete. 2024-04-25T10:34:53.624 INFO:tasks.ceph:Scrubbing osd.2 2024-04-25T10:34:53.624 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:34:53.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.735+0000 7f535323c640 1 Processor -- start 2024-04-25T10:34:53.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.735+0000 7f535323c640 1 -- start start 2024-04-25T10:34:53.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.736+0000 7f535323c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f534c0caa90 0x7f534c0ccb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:53.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.736+0000 7f535323c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f534c0c56f0 con 0x7f534c0c52e0 2024-04-25T10:34:53.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.736+0000 7f535323c640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f534c0cc050 con 0x7f534c0caea0 2024-04-25T10:34:53.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.736+0000 7f535323c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f534c0cc8f0 con 0x7f534c0caa90 2024-04-25T10:34:53.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.736+0000 7f5350fd8640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f534c0c52e0 0x7f534c15b2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:49438/0 (socket says 172.21.15.106:49438) 2024-04-25T10:34:53.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.737+0000 7f5350fd8640 1 -- 172.21.15.106:0/3189510851 learned_addr learned my addr 172.21.15.106:0/3189510851 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:34:53.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.737+0000 7f534bfff640 1 --2- 172.21.15.106:0/3189510851 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f534c0caa90 0x7f534c0ccb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:53.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.737+0000 7f534b7fe640 1 -- 172.21.15.106:0/3189510851 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 502080064 0 0) 0x7f534c0c56f0 con 0x7f534c0c52e0 2024-04-25T10:34:53.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.737+0000 7f534b7fe640 1 -- 172.21.15.106:0/3189510851 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f533c003520 con 0x7f534c0c52e0 2024-04-25T10:34:53.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.737+0000 7f534b7fe640 1 -- 172.21.15.106:0/3189510851 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2844725552 0 0) 0x7f534c0cc050 con 0x7f534c0caea0 2024-04-25T10:34:53.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.737+0000 7f534b7fe640 1 -- 172.21.15.106:0/3189510851 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f534c0c56f0 con 0x7f534c0caea0 2024-04-25T10:34:53.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.737+0000 7f534b7fe640 1 -- 172.21.15.106:0/3189510851 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2801661745 0 0) 0x7f533c003520 con 0x7f534c0c52e0 2024-04-25T10:34:53.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.737+0000 7f534b7fe640 1 -- 172.21.15.106:0/3189510851 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f534c0caa90 msgr2=0x7f534c0ccb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:53.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.737+0000 7f534b7fe640 1 --2- 172.21.15.106:0/3189510851 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f534c0caa90 0x7f534c0ccb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:53.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.737+0000 7f534b7fe640 1 -- 172.21.15.106:0/3189510851 >> v1:172.21.15.134:6789/0 conn(0x7f534c0caea0 legacy=0x7f534c1652a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:53.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.737+0000 7f534b7fe640 1 -- 172.21.15.106:0/3189510851 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f534c0cc330 con 0x7f534c0c52e0 2024-04-25T10:34:53.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.737+0000 7f534b7fe640 1 -- 172.21.15.106:0/3189510851 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3776459350 0 0) 0x7f5340001670 con 0x7f534c0c52e0 2024-04-25T10:34:53.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.737+0000 7f534b7fe640 1 -- 172.21.15.106:0/3189510851 >> v1:172.21.15.106:6789/0 conn(0x7f534c0c52e0 legacy=0x7f534c15b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:53.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.737+0000 7f534b7fe640 1 --2- 172.21.15.106:0/3189510851 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f533c003b70 0x7f533c003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:53.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.737+0000 7f534b7fe640 1 --2- 172.21.15.106:0/3189510851 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f533c004800 0x7f533c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:53.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.737+0000 7f534b7fe640 1 --2- 172.21.15.106:0/3189510851 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f533c0051a0 0x7f533c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:53.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.737+0000 7f534b7fe640 1 -- 172.21.15.106:0/3189510851 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f534c0cc8f0 con 0x7f533c003b70 2024-04-25T10:34:53.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.737+0000 7f534b7fe640 1 -- 172.21.15.106:0/3189510851 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f533c00c090 con 0x7f533c004800 2024-04-25T10:34:53.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.737+0000 7f534b7fe640 1 -- 172.21.15.106:0/3189510851 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f533c00c290 con 0x7f533c0051a0 2024-04-25T10:34:53.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.737+0000 7f534bfff640 1 --2- 172.21.15.106:0/3189510851 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f534c0caa90 0x7f534c0ccb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:34:53.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.737+0000 7f5350fd8640 1 --2- 172.21.15.106:0/3189510851 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f533c003b70 0x7f533c003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:53.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.737+0000 7f53517d9640 1 --2- 172.21.15.106:0/3189510851 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f533c0051a0 0x7f533c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:53.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.738+0000 7f534bfff640 1 --2- 172.21.15.106:0/3189510851 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f533c004800 0x7f533c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:53.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.738+0000 7f5350fd8640 1 -- 172.21.15.106:0/3189510851 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f533c0051a0 msgr2=0x7f533c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:53.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.738+0000 7f5350fd8640 1 --2- 172.21.15.106:0/3189510851 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f533c0051a0 0x7f533c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:53.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.738+0000 7f5350fd8640 1 -- 172.21.15.106:0/3189510851 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f533c004800 msgr2=0x7f533c004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:53.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.738+0000 7f5350fd8640 1 --2- 172.21.15.106:0/3189510851 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f533c004800 0x7f533c004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:53.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.738+0000 7f5350fd8640 1 -- 172.21.15.106:0/3189510851 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f533c003520 con 0x7f533c003b70 2024-04-25T10:34:53.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.738+0000 7f534bfff640 1 --2- 172.21.15.106:0/3189510851 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f533c004800 0x7f533c004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:34:53.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.738+0000 7f5350fd8640 1 --2- 172.21.15.106:0/3189510851 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f533c003b70 0x7f533c003fc0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f534000c190 tx=0x7f533c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7e56ce00f0be92a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:53.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.738+0000 7f534b7fe640 1 -- 172.21.15.106:0/3189510851 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5340007ce0 con 0x7f533c003b70 2024-04-25T10:34:53.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.738+0000 7f534b7fe640 1 -- 172.21.15.106:0/3189510851 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5340005ad0 con 0x7f533c003b70 2024-04-25T10:34:53.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.738+0000 7f534b7fe640 1 -- 172.21.15.106:0/3189510851 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5340007070 con 0x7f533c003b70 2024-04-25T10:34:53.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.739+0000 7f535323c640 1 -- 172.21.15.106:0/3189510851 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f533c003b70 msgr2=0x7f533c003fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:53.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.739+0000 7f535323c640 1 --2- 172.21.15.106:0/3189510851 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f533c003b70 0x7f533c003fc0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f534000c190 tx=0x7f533c005830 comp rx=0 tx=0).stop 2024-04-25T10:34:53.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.739+0000 7f53517d9640 1 -- 172.21.15.106:0/3189510851 reap_dead start 2024-04-25T10:34:53.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.739+0000 7f535323c640 1 -- 172.21.15.106:0/3189510851 shutdown_connections 2024-04-25T10:34:53.744 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.739+0000 7f535323c640 1 -- 172.21.15.106:0/3189510851 >> 172.21.15.106:0/3189510851 conn(0x7f534c068cd0 msgr2=0x7f534c072800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:53.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.739+0000 7f535323c640 1 -- 172.21.15.106:0/3189510851 shutdown_connections 2024-04-25T10:34:53.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.739+0000 7f535323c640 1 -- 172.21.15.106:0/3189510851 wait complete. 2024-04-25T10:34:53.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.742+0000 7f535323c640 1 Processor -- start 2024-04-25T10:34:53.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.742+0000 7f535323c640 1 -- start start 2024-04-25T10:34:53.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.744+0000 7f535323c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f533c003b70 0x7f534c1ef3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:53.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.744+0000 7f535323c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f534c0cf350 0x7f534c153f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:53.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.744+0000 7f535323c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f534c14b2b0 0x7f534c1498d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:53.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.744+0000 7f535323c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f534c15ada0 con 0x7f533c003b70 2024-04-25T10:34:53.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.744+0000 7f535323c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f534c15aac0 con 0x7f534c14b2b0 2024-04-25T10:34:53.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.744+0000 7f535323c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f534c0cbd30 con 0x7f534c0cf350 2024-04-25T10:34:53.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.744+0000 7f5350fd8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f533c003b70 0x7f534c1ef3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:53.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.744+0000 7f5350fd8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f533c003b70 0x7f534c1ef3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53278/0 (socket says 172.21.15.106:53278) 2024-04-25T10:34:53.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.744+0000 7f5350fd8640 1 -- 172.21.15.106:0/4134276175 learned_addr learned my addr 172.21.15.106:0/4134276175 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:34:53.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.744+0000 7f53517d9640 1 --2- 172.21.15.106:0/4134276175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f534c14b2b0 0x7f534c1498d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:53.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.744+0000 7f534bfff640 1 --2- 172.21.15.106:0/4134276175 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f534c0cf350 0x7f534c153f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:53.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.745+0000 7f5350fd8640 1 -- 172.21.15.106:0/4134276175 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f534c0cf350 msgr2=0x7f534c153f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:53.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.745+0000 7f5350fd8640 1 --2- 172.21.15.106:0/4134276175 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f534c0cf350 0x7f534c153f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:53.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.745+0000 7f5350fd8640 1 -- 172.21.15.106:0/4134276175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f534c14b2b0 msgr2=0x7f534c1498d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:53.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.745+0000 7f5350fd8640 1 --2- 172.21.15.106:0/4134276175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f534c14b2b0 0x7f534c1498d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:53.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.745+0000 7f5350fd8640 1 -- 172.21.15.106:0/4134276175 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f534c14fa80 con 0x7f533c003b70 2024-04-25T10:34:53.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.745+0000 7f5350fd8640 1 --2- 172.21.15.106:0/4134276175 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f533c003b70 0x7f534c1ef3b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f534002adb0 tx=0x7f53400019c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:53.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.745+0000 7f53497fa640 1 -- 172.21.15.106:0/4134276175 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5340041030 con 0x7f533c003b70 2024-04-25T10:34:53.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.745+0000 7f53497fa640 1 -- 172.21.15.106:0/4134276175 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5340038040 con 0x7f533c003b70 2024-04-25T10:34:53.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.745+0000 7f53497fa640 1 -- 172.21.15.106:0/4134276175 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5340007070 con 0x7f533c003b70 2024-04-25T10:34:53.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.745+0000 7f535323c640 1 -- 172.21.15.106:0/4134276175 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f534c1ee930 con 0x7f533c003b70 2024-04-25T10:34:53.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.745+0000 7f535323c640 1 -- 172.21.15.106:0/4134276175 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f534c14fcf0 con 0x7f533c003b70 2024-04-25T10:34:53.749 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.746+0000 7f535323c640 1 -- 172.21.15.106:0/4134276175 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f534c0cec00 con 0x7f533c003b70 2024-04-25T10:34:53.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.747+0000 7f53497fa640 1 -- 172.21.15.106:0/4134276175 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5340044070 con 0x7f533c003b70 2024-04-25T10:34:53.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.747+0000 7f53497fa640 1 --2- 172.21.15.106:0/4134276175 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f53300744e0 0x7f53300769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:53.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.747+0000 7f53497fa640 1 -- 172.21.15.106:0/4134276175 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f53400b4c30 con 0x7f533c003b70 2024-04-25T10:34:53.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.747+0000 7f534bfff640 1 --2- 172.21.15.106:0/4134276175 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f53300744e0 0x7f53300769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:53.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.747+0000 7f53497fa640 1 --2- 172.21.15.106:0/4134276175 >> [v2:172.21.15.106:6808/1441018239,v1:172.21.15.106:6809/1441018239] conn(0x7f533007c8b0 0x7f533007ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:53.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.747+0000 7f53497fa640 1 -- 172.21.15.106:0/4134276175 --> [v2:172.21.15.106:6808/1441018239,v1:172.21.15.106:6809/1441018239] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f534003fe50 con 0x7f533007c8b0 2024-04-25T10:34:53.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.747+0000 7f53497fa640 1 -- 172.21.15.106:0/4134276175 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f53400b7330 con 0x7f533c003b70 2024-04-25T10:34:53.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.747+0000 7f53517d9640 1 --2- 172.21.15.106:0/4134276175 >> [v2:172.21.15.106:6808/1441018239,v1:172.21.15.106:6809/1441018239] conn(0x7f533007c8b0 0x7f533007ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:53.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.748+0000 7f534bfff640 1 --2- 172.21.15.106:0/4134276175 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f53300744e0 0x7f53300769f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f534c0cd6b0 tx=0x7f5338008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:53.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.748+0000 7f53517d9640 1 --2- 172.21.15.106:0/4134276175 >> [v2:172.21.15.106:6808/1441018239,v1:172.21.15.106:6809/1441018239] conn(0x7f533007c8b0 0x7f533007ed80 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:53.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.749+0000 7f53497fa640 1 -- 172.21.15.106:0/4134276175 <== osd.2 v2:172.21.15.106:6808/1441018239 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f534003fe50 con 0x7f533007c8b0 2024-04-25T10:34:53.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.762+0000 7f535323c640 1 -- 172.21.15.106:0/4134276175 --> [v2:172.21.15.106:6808/1441018239,v1:172.21.15.106:6809/1441018239] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f534c06a3d0 con 0x7f533007c8b0 2024-04-25T10:34:53.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.762+0000 7f53497fa640 1 -- 172.21.15.106:0/4134276175 <== osd.2 v2:172.21.15.106:6808/1441018239 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f534c06a3d0 con 0x7f533007c8b0 2024-04-25T10:34:53.764 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-25T10:34:53.764 INFO:teuthology.orchestra.run.smithi106.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:34:53.764 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-25T10:34:53.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.762+0000 7f535323c640 1 -- 172.21.15.106:0/4134276175 >> [v2:172.21.15.106:6808/1441018239,v1:172.21.15.106:6809/1441018239] conn(0x7f533007c8b0 msgr2=0x7f533007ed80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:53.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.762+0000 7f535323c640 1 --2- 172.21.15.106:0/4134276175 >> [v2:172.21.15.106:6808/1441018239,v1:172.21.15.106:6809/1441018239] conn(0x7f533007c8b0 0x7f533007ed80 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:53.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.762+0000 7f535323c640 1 -- 172.21.15.106:0/4134276175 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f53300744e0 msgr2=0x7f53300769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:53.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.762+0000 7f535323c640 1 --2- 172.21.15.106:0/4134276175 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f53300744e0 0x7f53300769f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f534c0cd6b0 tx=0x7f5338008040 comp rx=0 tx=0).stop 2024-04-25T10:34:53.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.762+0000 7f535323c640 1 -- 172.21.15.106:0/4134276175 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f533c003b70 msgr2=0x7f534c1ef3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:53.765 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.762+0000 7f535323c640 1 --2- 172.21.15.106:0/4134276175 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f533c003b70 0x7f534c1ef3b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f534002adb0 tx=0x7f53400019c0 comp rx=0 tx=0).stop 2024-04-25T10:34:53.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.762+0000 7f53517d9640 1 -- 172.21.15.106:0/4134276175 reap_dead start 2024-04-25T10:34:53.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.762+0000 7f535323c640 1 -- 172.21.15.106:0/4134276175 shutdown_connections 2024-04-25T10:34:53.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.762+0000 7f535323c640 1 -- 172.21.15.106:0/4134276175 >> 172.21.15.106:0/4134276175 conn(0x7f534c068cd0 msgr2=0x7f534c153590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:53.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.762+0000 7f535323c640 1 -- 172.21.15.106:0/4134276175 shutdown_connections 2024-04-25T10:34:53.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.762+0000 7f535323c640 1 -- 172.21.15.106:0/4134276175 wait complete. 2024-04-25T10:34:53.776 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-25T10:34:53.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.883+0000 7fdc699d5640 1 Processor -- start 2024-04-25T10:34:53.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.883+0000 7fdc699d5640 1 -- start start 2024-04-25T10:34:53.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.884+0000 7fdc699d5640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fdc6415d510 0x7fdc64168690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:53.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.884+0000 7fdc699d5640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc64073980 con 0x7fdc640ca170 2024-04-25T10:34:53.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.884+0000 7fdc699d5640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc64160ea0 con 0x7fdc64156f70 2024-04-25T10:34:53.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.884+0000 7fdc699d5640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7fdc64073f00 con 0x7fdc6415d510 2024-04-25T10:34:53.887 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.884+0000 7fdc637fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fdc6415d510 0x7fdc64168690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:53.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.884+0000 7fdc637fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fdc6415d510 0x7fdc64168690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53294/0 (socket says 172.21.15.106:53294) 2024-04-25T10:34:53.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.884+0000 7fdc62ffd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fdc640ca170 0x7fdc6406c5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:49448/0 (socket says 172.21.15.106:49448) 2024-04-25T10:34:53.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.884+0000 7fdc637fe640 1 -- 172.21.15.106:0/2576149091 learned_addr learned my addr 172.21.15.106:0/2576149091 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:34:53.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.885+0000 7fdc637fe640 1 -- 172.21.15.106:0/2576149091 >> v1:172.21.15.134:6789/0 conn(0x7fdc64156f70 legacy=0x7fdc6415ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:53.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.885+0000 7fdc637fe640 1 -- 172.21.15.106:0/2576149091 >> v1:172.21.15.106:6789/0 conn(0x7fdc640ca170 legacy=0x7fdc6406c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:53.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.885+0000 7fdc637fe640 1 -- 172.21.15.106:0/2576149091 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc64074640 con 0x7fdc6415d510 2024-04-25T10:34:53.888 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.885+0000 7fdc61ffb640 1 -- 172.21.15.106:0/2576149091 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1352226330 0 0) 0x7fdc64073980 con 0x7fdc640ca170 2024-04-25T10:34:53.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.886+0000 7fdc637fe640 1 --2- 172.21.15.106:0/2576149091 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdc6415d510 0x7fdc64168690 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c009e50 tx=0x7fdc5c02a840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7952b142629e096a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:53.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.886+0000 7fdc61ffb640 1 -- 172.21.15.106:0/2576149091 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c037070 con 0x7fdc6415d510 2024-04-25T10:34:53.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.886+0000 7fdc61ffb640 1 -- 172.21.15.106:0/2576149091 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc5c032630 con 0x7fdc6415d510 2024-04-25T10:34:53.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.886+0000 7fdc61ffb640 1 -- 172.21.15.106:0/2576149091 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c02e050 con 0x7fdc6415d510 2024-04-25T10:34:53.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.887+0000 7fdc699d5640 1 -- 172.21.15.106:0/2576149091 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdc6415d510 msgr2=0x7fdc64168690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:53.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.887+0000 7fdc699d5640 1 --2- 172.21.15.106:0/2576149091 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdc6415d510 0x7fdc64168690 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c009e50 tx=0x7fdc5c02a840 comp rx=0 tx=0).stop 2024-04-25T10:34:53.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.887+0000 7fdc699d5640 1 -- 172.21.15.106:0/2576149091 shutdown_connections 2024-04-25T10:34:53.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.887+0000 7fdc699d5640 1 --2- 172.21.15.106:0/2576149091 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdc6415d510 0x7fdc64168690 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:53.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.887+0000 7fdc699d5640 1 -- 172.21.15.106:0/2576149091 >> 172.21.15.106:0/2576149091 conn(0x7fdc640cef20 msgr2=0x7fdc640cf330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:53.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.887+0000 7fdc699d5640 1 -- 172.21.15.106:0/2576149091 shutdown_connections 2024-04-25T10:34:53.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.887+0000 7fdc699d5640 1 -- 172.21.15.106:0/2576149091 wait complete. 2024-04-25T10:34:53.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.888+0000 7fdc699d5640 1 Processor -- start 2024-04-25T10:34:53.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.889+0000 7fdc699d5640 1 -- start start 2024-04-25T10:34:53.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.890+0000 7fdc699d5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc640ca170 0x7fdc641f3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:53.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.890+0000 7fdc699d5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdc64156f70 0x7fdc641543d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:53.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.890+0000 7fdc699d5640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc6415d510 0x7fdc64142690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:53.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.890+0000 7fdc699d5640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fdc64161140 con 0x7fdc64156f70 2024-04-25T10:34:53.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.890+0000 7fdc699d5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdc64074990 con 0x7fdc640ca170 2024-04-25T10:34:53.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.890+0000 7fdc699d5640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fdc64157380 con 0x7fdc6415d510 2024-04-25T10:34:53.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.890+0000 7fdc627fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdc64156f70 0x7fdc641543d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:53.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.890+0000 7fdc627fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdc64156f70 0x7fdc641543d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53308/0 (socket says 172.21.15.106:53308) 2024-04-25T10:34:53.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.890+0000 7fdc627fc640 1 -- 172.21.15.106:0/3469986604 learned_addr learned my addr 172.21.15.106:0/3469986604 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:34:53.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.890+0000 7fdc62ffd640 1 --2- 172.21.15.106:0/3469986604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc640ca170 0x7fdc641f3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:53.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.890+0000 7fdc637fe640 1 --2- 172.21.15.106:0/3469986604 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc6415d510 0x7fdc64142690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:53.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.890+0000 7fdc627fc640 1 -- 172.21.15.106:0/3469986604 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc6415d510 msgr2=0x7fdc64142690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:53.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.890+0000 7fdc627fc640 1 --2- 172.21.15.106:0/3469986604 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc6415d510 0x7fdc64142690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:53.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.890+0000 7fdc627fc640 1 -- 172.21.15.106:0/3469986604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc640ca170 msgr2=0x7fdc641f3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:53.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.890+0000 7fdc627fc640 1 --2- 172.21.15.106:0/3469986604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc640ca170 0x7fdc641f3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:53.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.890+0000 7fdc627fc640 1 -- 172.21.15.106:0/3469986604 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc641555e0 con 0x7fdc64156f70 2024-04-25T10:34:53.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.890+0000 7fdc627fc640 1 --2- 172.21.15.106:0/3469986604 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdc64156f70 0x7fdc641543d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fdc5400c3c0 tx=0x7fdc5400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:53.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.891+0000 7fdc4bfff640 1 -- 172.21.15.106:0/3469986604 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc54017070 con 0x7fdc64156f70 2024-04-25T10:34:53.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.891+0000 7fdc4bfff640 1 -- 172.21.15.106:0/3469986604 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc540052d0 con 0x7fdc64156f70 2024-04-25T10:34:53.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.891+0000 7fdc4bfff640 1 -- 172.21.15.106:0/3469986604 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc54003830 con 0x7fdc64156f70 2024-04-25T10:34:53.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.891+0000 7fdc699d5640 1 -- 172.21.15.106:0/3469986604 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc641f37b0 con 0x7fdc64156f70 2024-04-25T10:34:53.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.891+0000 7fdc699d5640 1 -- 172.21.15.106:0/3469986604 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc64157e00 con 0x7fdc64156f70 2024-04-25T10:34:53.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.892+0000 7fdc699d5640 1 -- 172.21.15.106:0/3469986604 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc6406a3d0 con 0x7fdc64156f70 2024-04-25T10:34:53.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.892+0000 7fdc4bfff640 1 -- 172.21.15.106:0/3469986604 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fdc5401a070 con 0x7fdc64156f70 2024-04-25T10:34:53.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.893+0000 7fdc4bfff640 1 --2- 172.21.15.106:0/3469986604 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fdc500744b0 0x7fdc500769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:53.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.893+0000 7fdc4bfff640 1 -- 172.21.15.106:0/3469986604 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fdc54094470 con 0x7fdc64156f70 2024-04-25T10:34:53.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.893+0000 7fdc62ffd640 1 --2- 172.21.15.106:0/3469986604 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fdc500744b0 0x7fdc500769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:53.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.893+0000 7fdc62ffd640 1 --2- 172.21.15.106:0/3469986604 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fdc500744b0 0x7fdc500769c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fdc58000f60 tx=0x7fdc58007e40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:53.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:53.897+0000 7fdc4bfff640 1 -- 172.21.15.106:0/3469986604 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdc5405e5c0 con 0x7fdc64156f70 2024-04-25T10:34:54.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.075+0000 7fdc699d5640 1 -- 172.21.15.106:0/3469986604 --> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7fdc640c89b0 con 0x7fdc500744b0 2024-04-25T10:34:54.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.075+0000 7fdc4bfff640 1 -- 172.21.15.106:0/3469986604 <== mgr.4107 v2:172.21.15.106:6832/1744210167 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fdc640c89b0 con 0x7fdc500744b0 2024-04-25T10:34:54.077 INFO:teuthology.orchestra.run.smithi106.stderr:instructed osd(s) 2 to deep-scrub 2024-04-25T10:34:54.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.077+0000 7fdc699d5640 1 -- 172.21.15.106:0/3469986604 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fdc500744b0 msgr2=0x7fdc500769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:54.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.077+0000 7fdc699d5640 1 --2- 172.21.15.106:0/3469986604 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fdc500744b0 0x7fdc500769c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fdc58000f60 tx=0x7fdc58007e40 comp rx=0 tx=0).stop 2024-04-25T10:34:54.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.077+0000 7fdc699d5640 1 -- 172.21.15.106:0/3469986604 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdc64156f70 msgr2=0x7fdc641543d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:54.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.077+0000 7fdc699d5640 1 --2- 172.21.15.106:0/3469986604 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdc64156f70 0x7fdc641543d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fdc5400c3c0 tx=0x7fdc5400c880 comp rx=0 tx=0).stop 2024-04-25T10:34:54.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.077+0000 7fdc699d5640 1 -- 172.21.15.106:0/3469986604 shutdown_connections 2024-04-25T10:34:54.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.077+0000 7fdc699d5640 1 --2- 172.21.15.106:0/3469986604 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fdc500744b0 0x7fdc500769c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:54.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.077+0000 7fdc699d5640 1 --2- 172.21.15.106:0/3469986604 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc6415d510 0x7fdc64142690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:54.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.077+0000 7fdc699d5640 1 --2- 172.21.15.106:0/3469986604 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdc64156f70 0x7fdc641543d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:54.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.077+0000 7fdc699d5640 1 --2- 172.21.15.106:0/3469986604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc640ca170 0x7fdc641f3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:54.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.077+0000 7fdc699d5640 1 -- 172.21.15.106:0/3469986604 >> 172.21.15.106:0/3469986604 conn(0x7fdc640cef20 msgr2=0x7fdc640c6010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:54.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.077+0000 7fdc699d5640 1 -- 172.21.15.106:0/3469986604 shutdown_connections 2024-04-25T10:34:54.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.077+0000 7fdc699d5640 1 -- 172.21.15.106:0/3469986604 wait complete. 2024-04-25T10:34:54.091 INFO:tasks.ceph:Scrubbing osd.3 2024-04-25T10:34:54.091 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:34:54.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.203+0000 7f782a44c640 1 Processor -- start 2024-04-25T10:34:54.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.203+0000 7f782a44c640 1 -- start start 2024-04-25T10:34:54.207 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.205+0000 7f782a44c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f7824157060 0x7f7824071630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:54.207 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.205+0000 7f782a44c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f782415cbf0 0x7f782406dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:54.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.205+0000 7f782a44c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f782415d580 con 0x7f78240cb520 2024-04-25T10:34:54.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.205+0000 7f782a44c640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f7824071b70 con 0x7f7824157060 2024-04-25T10:34:54.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.205+0000 7f782a44c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f782415db40 con 0x7f782415cbf0 2024-04-25T10:34:54.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.205+0000 7f78237fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f7824157060 0x7f7824071630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:54.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.205+0000 7f7823fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f78240cb520 0x7f78240cabe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:49450/0 (socket says 172.21.15.106:49450) 2024-04-25T10:34:54.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.205+0000 7f78237fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f7824157060 0x7f7824071630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53312/0 (socket says 172.21.15.106:53312) 2024-04-25T10:34:54.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.205+0000 7f7823fff640 1 -- 172.21.15.106:0/2481615542 learned_addr learned my addr 172.21.15.106:0/2481615542 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:34:54.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.205+0000 7f78289e9640 1 --2- 172.21.15.106:0/2481615542 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f782415cbf0 0x7f782406dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:54.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.205+0000 7f78237fe640 1 -- 172.21.15.106:0/2481615542 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f782415cbf0 msgr2=0x7f782406dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:54.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.205+0000 7f78237fe640 1 --2- 172.21.15.106:0/2481615542 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f782415cbf0 0x7f782406dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:54.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.205+0000 7f78237fe640 1 -- 172.21.15.106:0/2481615542 >> v1:172.21.15.106:6789/0 conn(0x7f78240cb520 legacy=0x7f78240cabe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:54.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.205+0000 7f7822ffd640 1 -- 172.21.15.106:0/2481615542 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3452240333 0 0) 0x7f782415d580 con 0x7f78240cb520 2024-04-25T10:34:54.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.205+0000 7f78237fe640 1 -- 172.21.15.106:0/2481615542 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f782415d860 con 0x7f7824157060 2024-04-25T10:34:54.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.206+0000 7f78237fe640 1 --2- 172.21.15.106:0/2481615542 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7824157060 0x7f7824071630 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f78180097a0 tx=0x7f781802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b73741f03cb9ea25 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:54.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.206+0000 7f7822ffd640 1 -- 172.21.15.106:0/2481615542 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7818037070 con 0x7f7824157060 2024-04-25T10:34:54.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.206+0000 7f7822ffd640 1 -- 172.21.15.106:0/2481615542 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f781802ae50 con 0x7f7824157060 2024-04-25T10:34:54.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.206+0000 7f7822ffd640 1 -- 172.21.15.106:0/2481615542 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7818032660 con 0x7f7824157060 2024-04-25T10:34:54.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.207+0000 7f782a44c640 1 -- 172.21.15.106:0/2481615542 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7824157060 msgr2=0x7f7824071630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:54.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.207+0000 7f782a44c640 1 --2- 172.21.15.106:0/2481615542 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7824157060 0x7f7824071630 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f78180097a0 tx=0x7f781802a100 comp rx=0 tx=0).stop 2024-04-25T10:34:54.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.207+0000 7f782a44c640 1 -- 172.21.15.106:0/2481615542 shutdown_connections 2024-04-25T10:34:54.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.207+0000 7f782a44c640 1 --2- 172.21.15.106:0/2481615542 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f782415cbf0 0x7f782406dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:54.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.207+0000 7f782a44c640 1 --2- 172.21.15.106:0/2481615542 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7824157060 0x7f7824071630 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:54.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.207+0000 7f782a44c640 1 -- 172.21.15.106:0/2481615542 >> 172.21.15.106:0/2481615542 conn(0x7f7824068cd0 msgr2=0x7f7824072800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:54.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.207+0000 7f782a44c640 1 -- 172.21.15.106:0/2481615542 shutdown_connections 2024-04-25T10:34:54.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.207+0000 7f782a44c640 1 -- 172.21.15.106:0/2481615542 wait complete. 2024-04-25T10:34:54.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.208+0000 7f782a44c640 1 Processor -- start 2024-04-25T10:34:54.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.209+0000 7f782a44c640 1 -- start start 2024-04-25T10:34:54.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.210+0000 7f782a44c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f78240cb520 0x7f78241e98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:54.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.210+0000 7f782a44c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7824157060 0x7f78241519e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:54.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.210+0000 7f782a44c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f782415cbf0 0x7f7824151f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:54.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.210+0000 7f782a44c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7824071110 con 0x7f782415cbf0 2024-04-25T10:34:54.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.210+0000 7f782a44c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7824070e30 con 0x7f7824157060 2024-04-25T10:34:54.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.210+0000 7f782a44c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f782415d000 con 0x7f78240cb520 2024-04-25T10:34:54.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.210+0000 7f78289e9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f782415cbf0 0x7f7824151f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:54.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.210+0000 7f78289e9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f782415cbf0 0x7f7824151f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53322/0 (socket says 172.21.15.106:53322) 2024-04-25T10:34:54.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.210+0000 7f78289e9640 1 -- 172.21.15.106:0/1285564257 learned_addr learned my addr 172.21.15.106:0/1285564257 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:34:54.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.210+0000 7f7823fff640 1 --2- 172.21.15.106:0/1285564257 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f78240cb520 0x7f78241e98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:54.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.210+0000 7f78237fe640 1 --2- 172.21.15.106:0/1285564257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7824157060 0x7f78241519e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:54.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.210+0000 7f78289e9640 1 -- 172.21.15.106:0/1285564257 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f78240cb520 msgr2=0x7f78241e98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:54.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.210+0000 7f78289e9640 1 --2- 172.21.15.106:0/1285564257 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f78240cb520 0x7f78241e98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:54.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.210+0000 7f78289e9640 1 -- 172.21.15.106:0/1285564257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7824157060 msgr2=0x7f78241519e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:54.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.210+0000 7f78289e9640 1 --2- 172.21.15.106:0/1285564257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7824157060 0x7f78241519e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:54.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.210+0000 7f78289e9640 1 -- 172.21.15.106:0/1285564257 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7824159af0 con 0x7f782415cbf0 2024-04-25T10:34:54.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.210+0000 7f78237fe640 1 --2- 172.21.15.106:0/1285564257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7824157060 0x7f78241519e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:34:54.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.210+0000 7f78289e9640 1 --2- 172.21.15.106:0/1285564257 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f782415cbf0 0x7f7824151f20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f781c009d30 tx=0x7f781c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:54.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.210+0000 7f7820ff9640 1 -- 172.21.15.106:0/1285564257 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f781c018020 con 0x7f782415cbf0 2024-04-25T10:34:54.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.210+0000 7f7820ff9640 1 -- 172.21.15.106:0/1285564257 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f781c014480 con 0x7f782415cbf0 2024-04-25T10:34:54.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.210+0000 7f782a44c640 1 -- 172.21.15.106:0/1285564257 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7824158d70 con 0x7f782415cbf0 2024-04-25T10:34:54.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.211+0000 7f7820ff9640 1 -- 172.21.15.106:0/1285564257 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f781c014870 con 0x7f782415cbf0 2024-04-25T10:34:54.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.211+0000 7f782a44c640 1 -- 172.21.15.106:0/1285564257 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78241592f0 con 0x7f782415cbf0 2024-04-25T10:34:54.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.211+0000 7f782a44c640 1 -- 172.21.15.106:0/1285564257 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f78240d0e90 con 0x7f782415cbf0 2024-04-25T10:34:54.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.212+0000 7f7820ff9640 1 -- 172.21.15.106:0/1285564257 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f781c005d10 con 0x7f782415cbf0 2024-04-25T10:34:54.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.213+0000 7f7820ff9640 1 --2- 172.21.15.106:0/1285564257 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f78100744b0 0x7f78100769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:54.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.213+0000 7f7823fff640 1 --2- 172.21.15.106:0/1285564257 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f78100744b0 0x7f78100769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:54.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.213+0000 7f7820ff9640 1 -- 172.21.15.106:0/1285564257 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f781c0964a0 con 0x7f782415cbf0 2024-04-25T10:34:54.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.213+0000 7f7820ff9640 1 --2- 172.21.15.106:0/1285564257 >> [v2:172.21.15.106:6816/1846968711,v1:172.21.15.106:6817/1846968711] conn(0x7f781007c880 0x7f781007ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:54.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.213+0000 7f7820ff9640 1 -- 172.21.15.106:0/1285564257 --> [v2:172.21.15.106:6816/1846968711,v1:172.21.15.106:6817/1846968711] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f781c01ae50 con 0x7f781007c880 2024-04-25T10:34:54.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.213+0000 7f7820ff9640 1 -- 172.21.15.106:0/1285564257 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f781c096930 con 0x7f782415cbf0 2024-04-25T10:34:54.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.213+0000 7f78237fe640 1 --2- 172.21.15.106:0/1285564257 >> [v2:172.21.15.106:6816/1846968711,v1:172.21.15.106:6817/1846968711] conn(0x7f781007c880 0x7f781007ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:54.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.213+0000 7f7823fff640 1 --2- 172.21.15.106:0/1285564257 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f78100744b0 0x7f78100769c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7814000f60 tx=0x7f7814007c00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:54.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.213+0000 7f78237fe640 1 --2- 172.21.15.106:0/1285564257 >> [v2:172.21.15.106:6816/1846968711,v1:172.21.15.106:6817/1846968711] conn(0x7f781007c880 0x7f781007ed50 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:54.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.214+0000 7f7820ff9640 1 -- 172.21.15.106:0/1285564257 <== osd.3 v2:172.21.15.106:6816/1846968711 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f781c01ae50 con 0x7f781007c880 2024-04-25T10:34:54.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.227+0000 7f782a44c640 1 -- 172.21.15.106:0/1285564257 --> [v2:172.21.15.106:6816/1846968711,v1:172.21.15.106:6817/1846968711] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f782406a3d0 con 0x7f781007c880 2024-04-25T10:34:54.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.227+0000 7f7820ff9640 1 -- 172.21.15.106:0/1285564257 <== osd.3 v2:172.21.15.106:6816/1846968711 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f782406a3d0 con 0x7f781007c880 2024-04-25T10:34:54.229 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-25T10:34:54.229 INFO:teuthology.orchestra.run.smithi106.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:34:54.230 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-25T10:34:54.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.227+0000 7f782a44c640 1 -- 172.21.15.106:0/1285564257 >> [v2:172.21.15.106:6816/1846968711,v1:172.21.15.106:6817/1846968711] conn(0x7f781007c880 msgr2=0x7f781007ed50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:54.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.227+0000 7f782a44c640 1 --2- 172.21.15.106:0/1285564257 >> [v2:172.21.15.106:6816/1846968711,v1:172.21.15.106:6817/1846968711] conn(0x7f781007c880 0x7f781007ed50 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:54.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.228+0000 7f782a44c640 1 -- 172.21.15.106:0/1285564257 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f78100744b0 msgr2=0x7f78100769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:54.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.228+0000 7f782a44c640 1 --2- 172.21.15.106:0/1285564257 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f78100744b0 0x7f78100769c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7814000f60 tx=0x7f7814007c00 comp rx=0 tx=0).stop 2024-04-25T10:34:54.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.228+0000 7f782a44c640 1 -- 172.21.15.106:0/1285564257 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f782415cbf0 msgr2=0x7f7824151f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:54.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.228+0000 7f782a44c640 1 --2- 172.21.15.106:0/1285564257 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f782415cbf0 0x7f7824151f20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f781c009d30 tx=0x7f781c00c490 comp rx=0 tx=0).stop 2024-04-25T10:34:54.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.228+0000 7f78289e9640 1 -- 172.21.15.106:0/1285564257 reap_dead start 2024-04-25T10:34:54.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.228+0000 7f782a44c640 1 -- 172.21.15.106:0/1285564257 shutdown_connections 2024-04-25T10:34:54.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.228+0000 7f782a44c640 1 -- 172.21.15.106:0/1285564257 >> 172.21.15.106:0/1285564257 conn(0x7f7824068cd0 msgr2=0x7f78241531e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:54.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.228+0000 7f782a44c640 1 -- 172.21.15.106:0/1285564257 shutdown_connections 2024-04-25T10:34:54.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.228+0000 7f782a44c640 1 -- 172.21.15.106:0/1285564257 wait complete. 2024-04-25T10:34:54.241 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-25T10:34:54.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.347+0000 7f3cb3ab6640 1 Processor -- start 2024-04-25T10:34:54.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.347+0000 7f3cb3ab6640 1 -- start start 2024-04-25T10:34:54.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.348+0000 7f3cb3ab6640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f3cac06dc00 0x7f3cac15d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:54.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.348+0000 7f3cb3ab6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3cac0c61a0 0x7f3cac156ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:54.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.348+0000 7f3cb3ab6640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3cac1601c0 0x7f3cac0ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:54.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.348+0000 7f3cb3ab6640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f3cac0c65b0 con 0x7f3cac1601c0 2024-04-25T10:34:54.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.348+0000 7f3cb3ab6640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f3cac160ae0 con 0x7f3cac06dc00 2024-04-25T10:34:54.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.348+0000 7f3cb3ab6640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f3cac160dc0 con 0x7f3cac0c61a0 2024-04-25T10:34:54.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.349+0000 7f3cb2053640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3cac1601c0 0x7f3cac0ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:54.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.349+0000 7f3cb2053640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3cac1601c0 0x7f3cac0ccde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53334/0 (socket says 172.21.15.106:53334) 2024-04-25T10:34:54.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.349+0000 7f3cb2053640 1 -- 172.21.15.106:0/372867040 learned_addr learned my addr 172.21.15.106:0/372867040 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:34:54.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.349+0000 7f3cb1852640 1 --2- 172.21.15.106:0/372867040 >> v2:172.21.15.134:3300/0 conn(0x7f3cac06dc00 0x7f3cac15d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:54.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.349+0000 7f3cb1051640 1 --2- 172.21.15.106:0/372867040 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3cac0c61a0 0x7f3cac156ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:54.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.349+0000 7f3cb2053640 1 -- 172.21.15.106:0/372867040 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3cac0c61a0 msgr2=0x7f3cac156ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:54.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.349+0000 7f3cb2053640 1 --2- 172.21.15.106:0/372867040 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3cac0c61a0 0x7f3cac156ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:54.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.349+0000 7f3cb2053640 1 -- 172.21.15.106:0/372867040 >> v2:172.21.15.134:3300/0 conn(0x7f3cac06dc00 msgr2=0x7f3cac15d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:54.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.349+0000 7f3cb2053640 1 --2- 172.21.15.106:0/372867040 >> v2:172.21.15.134:3300/0 conn(0x7f3cac06dc00 0x7f3cac15d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:54.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.349+0000 7f3cb2053640 1 -- 172.21.15.106:0/372867040 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cac1610a0 con 0x7f3cac1601c0 2024-04-25T10:34:54.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.350+0000 7f3cb2053640 1 --2- 172.21.15.106:0/372867040 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3cac1601c0 0x7f3cac0ccde0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3cac0744e0 tx=0x7f3ca402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed0b46114a217e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:54.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.350+0000 7f3cb0850640 1 -- 172.21.15.106:0/372867040 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca4037070 con 0x7f3cac1601c0 2024-04-25T10:34:54.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.350+0000 7f3cb0850640 1 -- 172.21.15.106:0/372867040 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ca4032440 con 0x7f3cac1601c0 2024-04-25T10:34:54.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.350+0000 7f3cb0850640 1 -- 172.21.15.106:0/372867040 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca4032810 con 0x7f3cac1601c0 2024-04-25T10:34:54.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.350+0000 7f3cb3ab6640 1 -- 172.21.15.106:0/372867040 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3cac1601c0 msgr2=0x7f3cac0ccde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:54.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.350+0000 7f3cb3ab6640 1 --2- 172.21.15.106:0/372867040 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3cac1601c0 0x7f3cac0ccde0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3cac0744e0 tx=0x7f3ca402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:34:54.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.351+0000 7f3cb3ab6640 1 -- 172.21.15.106:0/372867040 shutdown_connections 2024-04-25T10:34:54.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.351+0000 7f3cb3ab6640 1 --2- 172.21.15.106:0/372867040 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3cac1601c0 0x7f3cac0ccde0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:54.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.351+0000 7f3cb3ab6640 1 --2- 172.21.15.106:0/372867040 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3cac0c61a0 0x7f3cac156ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:54.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.351+0000 7f3cb3ab6640 1 --2- 172.21.15.106:0/372867040 >> v2:172.21.15.134:3300/0 conn(0x7f3cac06dc00 0x7f3cac15d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:54.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.351+0000 7f3cb3ab6640 1 -- 172.21.15.106:0/372867040 >> 172.21.15.106:0/372867040 conn(0x7f3cac0727d0 msgr2=0x7f3cac072c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:54.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.351+0000 7f3cb3ab6640 1 -- 172.21.15.106:0/372867040 shutdown_connections 2024-04-25T10:34:54.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.351+0000 7f3cb3ab6640 1 -- 172.21.15.106:0/372867040 wait complete. 2024-04-25T10:34:54.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.352+0000 7f3cb3ab6640 1 Processor -- start 2024-04-25T10:34:54.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.352+0000 7f3cb3ab6640 1 -- start start 2024-04-25T10:34:54.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.354+0000 7f3cb3ab6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3cac06dc00 0x7f3cac1f3150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:54.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.354+0000 7f3cb3ab6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3cac0c61a0 0x7f3cac15f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:54.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.354+0000 7f3cb3ab6640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3cac1601c0 0x7f3cac150960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:54.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.354+0000 7f3cb3ab6640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3cac15d2a0 con 0x7f3cac1601c0 2024-04-25T10:34:54.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.354+0000 7f3cb3ab6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3cac15cfc0 con 0x7f3cac0c61a0 2024-04-25T10:34:54.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.354+0000 7f3cb3ab6640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f3cac06d740 con 0x7f3cac06dc00 2024-04-25T10:34:54.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.354+0000 7f3cb2053640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3cac1601c0 0x7f3cac150960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:54.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.354+0000 7f3cb2053640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3cac1601c0 0x7f3cac150960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53350/0 (socket says 172.21.15.106:53350) 2024-04-25T10:34:54.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.354+0000 7f3cb2053640 1 -- 172.21.15.106:0/2447064112 learned_addr learned my addr 172.21.15.106:0/2447064112 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:34:54.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.354+0000 7f3cb2053640 1 -- 172.21.15.106:0/2447064112 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3cac06dc00 msgr2=0x7f3cac1f3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:54.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.354+0000 7f3cb2053640 1 --2- 172.21.15.106:0/2447064112 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3cac06dc00 0x7f3cac1f3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:54.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.354+0000 7f3cb2053640 1 -- 172.21.15.106:0/2447064112 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3cac0c61a0 msgr2=0x7f3cac15f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:54.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.354+0000 7f3cb2053640 1 --2- 172.21.15.106:0/2447064112 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3cac0c61a0 0x7f3cac15f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:54.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.354+0000 7f3cb2053640 1 -- 172.21.15.106:0/2447064112 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cac15f980 con 0x7f3cac1601c0 2024-04-25T10:34:54.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.354+0000 7f3cb2053640 1 --2- 172.21.15.106:0/2447064112 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3cac1601c0 0x7f3cac150960 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3ca40090b0 tx=0x7f3ca40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:54.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.354+0000 7f3ca27fc640 1 -- 172.21.15.106:0/2447064112 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca4037040 con 0x7f3cac1601c0 2024-04-25T10:34:54.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.354+0000 7f3ca27fc640 1 -- 172.21.15.106:0/2447064112 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ca4042070 con 0x7f3cac1601c0 2024-04-25T10:34:54.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.354+0000 7f3cb3ab6640 1 -- 172.21.15.106:0/2447064112 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cac15ec80 con 0x7f3cac1601c0 2024-04-25T10:34:54.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.354+0000 7f3ca27fc640 1 -- 172.21.15.106:0/2447064112 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca4030920 con 0x7f3cac1601c0 2024-04-25T10:34:54.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.354+0000 7f3cb3ab6640 1 -- 172.21.15.106:0/2447064112 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cac1f3d50 con 0x7f3cac1601c0 2024-04-25T10:34:54.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.355+0000 7f3cb3ab6640 1 -- 172.21.15.106:0/2447064112 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cac06f1c0 con 0x7f3cac1601c0 2024-04-25T10:34:54.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.356+0000 7f3ca27fc640 1 -- 172.21.15.106:0/2447064112 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3ca4030ac0 con 0x7f3cac1601c0 2024-04-25T10:34:54.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.356+0000 7f3ca27fc640 1 --2- 172.21.15.106:0/2447064112 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f3c900744e0 0x7f3c900769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:54.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.356+0000 7f3ca27fc640 1 -- 172.21.15.106:0/2447064112 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f3ca40b5c10 con 0x7f3cac1601c0 2024-04-25T10:34:54.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.356+0000 7f3cb1852640 1 --2- 172.21.15.106:0/2447064112 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f3c900744e0 0x7f3c900769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:54.360 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.357+0000 7f3cb1852640 1 --2- 172.21.15.106:0/2447064112 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f3c900744e0 0x7f3c900769f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3ca80070b0 tx=0x7f3ca8007040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:54.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.361+0000 7f3ca27fc640 1 -- 172.21.15.106:0/2447064112 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3cac06f1c0 con 0x7f3cac1601c0 2024-04-25T10:34:54.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.539+0000 7f3cb3ab6640 1 -- 172.21.15.106:0/2447064112 --> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f3cac076860 con 0x7f3c900744e0 2024-04-25T10:34:54.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.539+0000 7f3ca27fc640 1 -- 172.21.15.106:0/2447064112 <== mgr.4107 v2:172.21.15.106:6832/1744210167 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f3cac076860 con 0x7f3c900744e0 2024-04-25T10:34:54.542 INFO:teuthology.orchestra.run.smithi106.stderr:instructed osd(s) 3 to deep-scrub 2024-04-25T10:34:54.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.541+0000 7f3cb3ab6640 1 -- 172.21.15.106:0/2447064112 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f3c900744e0 msgr2=0x7f3c900769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:54.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.541+0000 7f3cb3ab6640 1 --2- 172.21.15.106:0/2447064112 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f3c900744e0 0x7f3c900769f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3ca80070b0 tx=0x7f3ca8007040 comp rx=0 tx=0).stop 2024-04-25T10:34:54.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.541+0000 7f3cb3ab6640 1 -- 172.21.15.106:0/2447064112 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3cac1601c0 msgr2=0x7f3cac150960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:54.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.541+0000 7f3cb3ab6640 1 --2- 172.21.15.106:0/2447064112 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3cac1601c0 0x7f3cac150960 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3ca40090b0 tx=0x7f3ca40046c0 comp rx=0 tx=0).stop 2024-04-25T10:34:54.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.541+0000 7f3cb3ab6640 1 -- 172.21.15.106:0/2447064112 shutdown_connections 2024-04-25T10:34:54.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.541+0000 7f3cb3ab6640 1 --2- 172.21.15.106:0/2447064112 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f3c900744e0 0x7f3c900769f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:54.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.541+0000 7f3cb3ab6640 1 --2- 172.21.15.106:0/2447064112 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3cac1601c0 0x7f3cac150960 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:54.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.541+0000 7f3cb3ab6640 1 --2- 172.21.15.106:0/2447064112 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3cac0c61a0 0x7f3cac15f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:54.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.541+0000 7f3cb3ab6640 1 --2- 172.21.15.106:0/2447064112 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3cac06dc00 0x7f3cac1f3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:54.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.541+0000 7f3cb3ab6640 1 -- 172.21.15.106:0/2447064112 >> 172.21.15.106:0/2447064112 conn(0x7f3cac0727d0 msgr2=0x7f3cac06eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:54.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.542+0000 7f3cb3ab6640 1 -- 172.21.15.106:0/2447064112 shutdown_connections 2024-04-25T10:34:54.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.542+0000 7f3cb3ab6640 1 -- 172.21.15.106:0/2447064112 wait complete. 2024-04-25T10:34:54.556 INFO:tasks.ceph:Scrubbing osd.4 2024-04-25T10:34:54.556 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:34:54.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.665+0000 7f8636b1f640 1 Processor -- start 2024-04-25T10:34:54.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.665+0000 7f8636b1f640 1 -- start start 2024-04-25T10:34:54.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.666+0000 7f8636b1f640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f8630154dc0 0x7f8630071750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:54.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.666+0000 7f8636b1f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f863015b040 0x7f863015b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:54.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.666+0000 7f8636b1f640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8630072170 con 0x7f8630071170 2024-04-25T10:34:54.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.666+0000 7f8636b1f640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f86300cb630 con 0x7f8630154dc0 2024-04-25T10:34:54.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.666+0000 7f8636b1f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f863006c5c0 con 0x7f863015b040 2024-04-25T10:34:54.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.667+0000 7f86348bb640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f8630071170 0x7f86300cf280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:49462/0 (socket says 172.21.15.106:49462) 2024-04-25T10:34:54.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.667+0000 7f86348bb640 1 -- 172.21.15.106:0/1818375326 learned_addr learned my addr 172.21.15.106:0/1818375326 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:34:54.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.667+0000 7f862ffff640 1 --2- 172.21.15.106:0/1818375326 >> v2:172.21.15.134:3300/0 conn(0x7f8630154dc0 0x7f8630071750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:54.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.667+0000 7f86350bc640 1 --2- 172.21.15.106:0/1818375326 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f863015b040 0x7f863015b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:54.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.667+0000 7f862f7fe640 1 -- 172.21.15.106:0/1818375326 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1464741933 0 0) 0x7f8630072170 con 0x7f8630071170 2024-04-25T10:34:54.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.667+0000 7f862f7fe640 1 -- 172.21.15.106:0/1818375326 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8618003520 con 0x7f8630071170 2024-04-25T10:34:54.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.667+0000 7f862f7fe640 1 -- 172.21.15.106:0/1818375326 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 929459238 0 0) 0x7f8618003520 con 0x7f8630071170 2024-04-25T10:34:54.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.667+0000 7f862f7fe640 1 -- 172.21.15.106:0/1818375326 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f863015b040 msgr2=0x7f863015b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:54.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.667+0000 7f862f7fe640 1 --2- 172.21.15.106:0/1818375326 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f863015b040 0x7f863015b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:54.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.667+0000 7f862f7fe640 1 -- 172.21.15.106:0/1818375326 >> v2:172.21.15.134:3300/0 conn(0x7f8630154dc0 msgr2=0x7f8630071750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:54.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.667+0000 7f862f7fe640 1 --2- 172.21.15.106:0/1818375326 >> v2:172.21.15.134:3300/0 conn(0x7f8630154dc0 0x7f8630071750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:54.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.667+0000 7f862f7fe640 1 -- 172.21.15.106:0/1818375326 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f863006c810 con 0x7f8630071170 2024-04-25T10:34:54.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.667+0000 7f862f7fe640 1 -- 172.21.15.106:0/1818375326 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3776459350 0 0) 0x7f8620001690 con 0x7f8630071170 2024-04-25T10:34:54.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.667+0000 7f862f7fe640 1 -- 172.21.15.106:0/1818375326 >> v1:172.21.15.106:6789/0 conn(0x7f8630071170 legacy=0x7f86300cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:54.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.667+0000 7f862f7fe640 1 --2- 172.21.15.106:0/1818375326 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f863015b040 0x7f86180039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:54.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.667+0000 7f862f7fe640 1 --2- 172.21.15.106:0/1818375326 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8618003f30 0x7f8618004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:54.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.667+0000 7f862f7fe640 1 --2- 172.21.15.106:0/1818375326 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8618004d40 0x7f8618005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:54.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.667+0000 7f862f7fe640 1 -- 172.21.15.106:0/1818375326 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f86300cb630 con 0x7f8618003f30 2024-04-25T10:34:54.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.667+0000 7f862f7fe640 1 -- 172.21.15.106:0/1818375326 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f863006c5c0 con 0x7f8618004d40 2024-04-25T10:34:54.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.667+0000 7f862f7fe640 1 -- 172.21.15.106:0/1818375326 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f861800ba10 con 0x7f863015b040 2024-04-25T10:34:54.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.667+0000 7f862ffff640 1 --2- 172.21.15.106:0/1818375326 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8618003f30 0x7f8618004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:54.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.667+0000 7f86348bb640 1 --2- 172.21.15.106:0/1818375326 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f863015b040 0x7f86180039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:54.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.667+0000 7f86350bc640 1 --2- 172.21.15.106:0/1818375326 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8618004d40 0x7f8618005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:54.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.668+0000 7f862ffff640 1 -- 172.21.15.106:0/1818375326 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f863015b040 msgr2=0x7f86180039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:54.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.668+0000 7f862ffff640 1 --2- 172.21.15.106:0/1818375326 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f863015b040 0x7f86180039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:54.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.668+0000 7f862ffff640 1 -- 172.21.15.106:0/1818375326 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8618004d40 msgr2=0x7f8618005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:54.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.668+0000 7f862ffff640 1 --2- 172.21.15.106:0/1818375326 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8618004d40 0x7f8618005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:54.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.668+0000 7f862ffff640 1 -- 172.21.15.106:0/1818375326 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8618003520 con 0x7f8618003f30 2024-04-25T10:34:54.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.668+0000 7f862ffff640 1 --2- 172.21.15.106:0/1818375326 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8618003f30 0x7f8618004800 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8624009a90 tx=0x7f8618005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef4d3386adb27ffa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:54.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.668+0000 7f862f7fe640 1 -- 172.21.15.106:0/1818375326 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8624036070 con 0x7f8618003f30 2024-04-25T10:34:54.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.668+0000 7f862f7fe640 1 -- 172.21.15.106:0/1818375326 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8624031400 con 0x7f8618003f30 2024-04-25T10:34:54.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.668+0000 7f862f7fe640 1 -- 172.21.15.106:0/1818375326 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8624009690 con 0x7f8618003f30 2024-04-25T10:34:54.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.669+0000 7f8636b1f640 1 -- 172.21.15.106:0/1818375326 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8618003f30 msgr2=0x7f8618004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:54.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.669+0000 7f8636b1f640 1 --2- 172.21.15.106:0/1818375326 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8618003f30 0x7f8618004800 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8624009a90 tx=0x7f8618005830 comp rx=0 tx=0).stop 2024-04-25T10:34:54.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.669+0000 7f86350bc640 1 -- 172.21.15.106:0/1818375326 reap_dead start 2024-04-25T10:34:54.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.669+0000 7f8636b1f640 1 -- 172.21.15.106:0/1818375326 shutdown_connections 2024-04-25T10:34:54.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.669+0000 7f8636b1f640 1 -- 172.21.15.106:0/1818375326 >> 172.21.15.106:0/1818375326 conn(0x7f8630073910 msgr2=0x7f8630073d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:54.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.669+0000 7f8636b1f640 1 -- 172.21.15.106:0/1818375326 shutdown_connections 2024-04-25T10:34:54.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.669+0000 7f8636b1f640 1 -- 172.21.15.106:0/1818375326 wait complete. 2024-04-25T10:34:54.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.671+0000 7f8636b1f640 1 Processor -- start 2024-04-25T10:34:54.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.671+0000 7f8636b1f640 1 -- start start 2024-04-25T10:34:54.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.672+0000 7f8636b1f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8618003f30 0x7f86300c7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:54.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.672+0000 7f8636b1f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f863014c9b0 0x7f863014cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:54.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.672+0000 7f8636b1f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f863015de20 0x7f863015e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:54.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.672+0000 7f8636b1f640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f863015ab20 con 0x7f8618003f30 2024-04-25T10:34:54.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.672+0000 7f8636b1f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f863006cdd0 con 0x7f863015de20 2024-04-25T10:34:54.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.672+0000 7f8636b1f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f863006d460 con 0x7f863014c9b0 2024-04-25T10:34:54.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.672+0000 7f86348bb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8618003f30 0x7f86300c7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:54.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.672+0000 7f86348bb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8618003f30 0x7f86300c7290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53374/0 (socket says 172.21.15.106:53374) 2024-04-25T10:34:54.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.672+0000 7f86348bb640 1 -- 172.21.15.106:0/1588383592 learned_addr learned my addr 172.21.15.106:0/1588383592 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:34:54.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.673+0000 7f862ffff640 1 --2- 172.21.15.106:0/1588383592 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f863014c9b0 0x7f863014cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:54.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.673+0000 7f86350bc640 1 --2- 172.21.15.106:0/1588383592 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f863015de20 0x7f863015e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:54.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.673+0000 7f86348bb640 1 -- 172.21.15.106:0/1588383592 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f863014c9b0 msgr2=0x7f863014cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:54.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.673+0000 7f86348bb640 1 --2- 172.21.15.106:0/1588383592 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f863014c9b0 0x7f863014cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:54.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.673+0000 7f86348bb640 1 -- 172.21.15.106:0/1588383592 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f863015de20 msgr2=0x7f863015e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:54.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.673+0000 7f86348bb640 1 --2- 172.21.15.106:0/1588383592 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f863015de20 0x7f863015e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:54.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.673+0000 7f86348bb640 1 -- 172.21.15.106:0/1588383592 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f863014ec60 con 0x7f8618003f30 2024-04-25T10:34:54.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.673+0000 7f86348bb640 1 --2- 172.21.15.106:0/1588383592 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8618003f30 0x7f86300c7290 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8620007340 tx=0x7f8620007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:54.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.673+0000 7f862d7fa640 1 -- 172.21.15.106:0/1588383592 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f862000f8c0 con 0x7f8618003f30 2024-04-25T10:34:54.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.673+0000 7f862d7fa640 1 -- 172.21.15.106:0/1588383592 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8620018070 con 0x7f8618003f30 2024-04-25T10:34:54.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.673+0000 7f8636b1f640 1 -- 172.21.15.106:0/1588383592 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86300c77d0 con 0x7f8618003f30 2024-04-25T10:34:54.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.673+0000 7f862d7fa640 1 -- 172.21.15.106:0/1588383592 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f862000fca0 con 0x7f8618003f30 2024-04-25T10:34:54.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.673+0000 7f8636b1f640 1 -- 172.21.15.106:0/1588383592 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8630151e30 con 0x7f8618003f30 2024-04-25T10:34:54.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.674+0000 7f8636b1f640 1 -- 172.21.15.106:0/1588383592 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f863015e770 con 0x7f8618003f30 2024-04-25T10:34:54.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.674+0000 7f862d7fa640 1 -- 172.21.15.106:0/1588383592 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f862001d460 con 0x7f8618003f30 2024-04-25T10:34:54.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.675+0000 7f862d7fa640 1 --2- 172.21.15.106:0/1588383592 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f861c0744b0 0x7f861c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:54.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.675+0000 7f862d7fa640 1 -- 172.21.15.106:0/1588383592 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f86200953c0 con 0x7f8618003f30 2024-04-25T10:34:54.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.675+0000 7f862ffff640 1 --2- 172.21.15.106:0/1588383592 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f861c0744b0 0x7f861c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:54.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.675+0000 7f862d7fa640 1 --2- 172.21.15.106:0/1588383592 >> [v2:172.21.15.134:6824/729224739,v1:172.21.15.134:6825/729224739] conn(0x7f861c07c880 0x7f861c07ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:54.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.675+0000 7f862d7fa640 1 -- 172.21.15.106:0/1588383592 --> [v2:172.21.15.134:6824/729224739,v1:172.21.15.134:6825/729224739] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f861c07f420 con 0x7f861c07c880 2024-04-25T10:34:54.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.675+0000 7f862d7fa640 1 -- 172.21.15.106:0/1588383592 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f862009c050 con 0x7f8618003f30 2024-04-25T10:34:54.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.675+0000 7f86350bc640 1 --2- 172.21.15.106:0/1588383592 >> [v2:172.21.15.134:6824/729224739,v1:172.21.15.134:6825/729224739] conn(0x7f861c07c880 0x7f861c07ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:54.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.675+0000 7f862ffff640 1 --2- 172.21.15.106:0/1588383592 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f861c0744b0 0x7f861c0769c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8624033000 tx=0x7f8624008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:54.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.676+0000 7f86350bc640 1 --2- 172.21.15.106:0/1588383592 >> [v2:172.21.15.134:6824/729224739,v1:172.21.15.134:6825/729224739] conn(0x7f861c07c880 0x7f861c07ed50 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:54.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.677+0000 7f862d7fa640 1 -- 172.21.15.106:0/1588383592 <== osd.4 v2:172.21.15.134:6824/729224739 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f861c07f420 con 0x7f861c07c880 2024-04-25T10:34:54.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.689+0000 7f8636b1f640 1 -- 172.21.15.106:0/1588383592 --> [v2:172.21.15.134:6824/729224739,v1:172.21.15.134:6825/729224739] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f863006a3d0 con 0x7f861c07c880 2024-04-25T10:34:54.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.690+0000 7f862d7fa640 1 -- 172.21.15.106:0/1588383592 <== osd.4 v2:172.21.15.134:6824/729224739 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f863006a3d0 con 0x7f861c07c880 2024-04-25T10:34:54.692 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-25T10:34:54.693 INFO:teuthology.orchestra.run.smithi106.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:34:54.693 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-25T10:34:54.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.690+0000 7f8636b1f640 1 -- 172.21.15.106:0/1588383592 >> [v2:172.21.15.134:6824/729224739,v1:172.21.15.134:6825/729224739] conn(0x7f861c07c880 msgr2=0x7f861c07ed50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:54.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.690+0000 7f8636b1f640 1 --2- 172.21.15.106:0/1588383592 >> [v2:172.21.15.134:6824/729224739,v1:172.21.15.134:6825/729224739] conn(0x7f861c07c880 0x7f861c07ed50 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:54.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.690+0000 7f8636b1f640 1 -- 172.21.15.106:0/1588383592 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f861c0744b0 msgr2=0x7f861c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:54.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.690+0000 7f8636b1f640 1 --2- 172.21.15.106:0/1588383592 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f861c0744b0 0x7f861c0769c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8624033000 tx=0x7f8624008030 comp rx=0 tx=0).stop 2024-04-25T10:34:54.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.690+0000 7f8636b1f640 1 -- 172.21.15.106:0/1588383592 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8618003f30 msgr2=0x7f86300c7290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:54.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.690+0000 7f8636b1f640 1 --2- 172.21.15.106:0/1588383592 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8618003f30 0x7f86300c7290 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8620007340 tx=0x7f8620007800 comp rx=0 tx=0).stop 2024-04-25T10:34:54.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.690+0000 7f86350bc640 1 -- 172.21.15.106:0/1588383592 reap_dead start 2024-04-25T10:34:54.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.691+0000 7f8636b1f640 1 -- 172.21.15.106:0/1588383592 shutdown_connections 2024-04-25T10:34:54.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.691+0000 7f8636b1f640 1 -- 172.21.15.106:0/1588383592 >> 172.21.15.106:0/1588383592 conn(0x7f8630073910 msgr2=0x7f86300cf250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:54.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.691+0000 7f8636b1f640 1 -- 172.21.15.106:0/1588383592 shutdown_connections 2024-04-25T10:34:54.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.691+0000 7f8636b1f640 1 -- 172.21.15.106:0/1588383592 wait complete. 2024-04-25T10:34:54.704 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-25T10:34:54.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.809+0000 7f59e42a1640 1 Processor -- start 2024-04-25T10:34:54.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.809+0000 7f59e42a1640 1 -- start start 2024-04-25T10:34:54.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.810+0000 7f59e42a1640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f59dc0c8f70 0x7f59dc0c4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:54.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.811+0000 7f59e42a1640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59dc070150 con 0x7f59dc153ca0 2024-04-25T10:34:54.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.811+0000 7f59e42a1640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59dc0ca170 con 0x7f59dc0c3590 2024-04-25T10:34:54.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.811+0000 7f59e42a1640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f59dc06ff00 con 0x7f59dc0c8f70 2024-04-25T10:34:54.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.811+0000 7f59e283e640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f59dc153ca0 0x7f59dc1641c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:49466/0 (socket says 172.21.15.106:49466) 2024-04-25T10:34:54.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.811+0000 7f59e283e640 1 -- 172.21.15.106:0/2491287111 learned_addr learned my addr 172.21.15.106:0/2491287111 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:34:54.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.811+0000 7f59e203d640 1 --2- 172.21.15.106:0/2491287111 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f59dc0c8f70 0x7f59dc0c4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:54.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.811+0000 7f59e103b640 1 -- 172.21.15.106:0/2491287111 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2829028136 0 0) 0x7f59dc070150 con 0x7f59dc153ca0 2024-04-25T10:34:54.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.811+0000 7f59e103b640 1 -- 172.21.15.106:0/2491287111 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f59c8003520 con 0x7f59dc153ca0 2024-04-25T10:34:54.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.811+0000 7f59e103b640 1 -- 172.21.15.106:0/2491287111 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3063408941 0 0) 0x7f59dc0ca170 con 0x7f59dc0c3590 2024-04-25T10:34:54.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.811+0000 7f59e103b640 1 -- 172.21.15.106:0/2491287111 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f59dc070150 con 0x7f59dc0c3590 2024-04-25T10:34:54.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.811+0000 7f59e203d640 1 -- 172.21.15.106:0/2491287111 >> v1:172.21.15.134:6789/0 conn(0x7f59dc0c3590 legacy=0x7f59dc159840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:54.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.811+0000 7f59e203d640 1 -- 172.21.15.106:0/2491287111 >> v1:172.21.15.106:6789/0 conn(0x7f59dc153ca0 legacy=0x7f59dc1641c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:54.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.811+0000 7f59e203d640 1 -- 172.21.15.106:0/2491287111 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59dc0ca450 con 0x7f59dc0c8f70 2024-04-25T10:34:54.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.812+0000 7f59e203d640 1 --2- 172.21.15.106:0/2491287111 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f59dc0c8f70 0x7f59dc0c4de0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f59dc070080 tx=0x7f59d802a6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7bde2cc9ff2617b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:54.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.812+0000 7f59e103b640 1 -- 172.21.15.106:0/2491287111 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d8007740 con 0x7f59dc0c8f70 2024-04-25T10:34:54.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.812+0000 7f59e103b640 1 -- 172.21.15.106:0/2491287111 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59d80078e0 con 0x7f59dc0c8f70 2024-04-25T10:34:54.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.812+0000 7f59e103b640 1 -- 172.21.15.106:0/2491287111 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d8007c90 con 0x7f59dc0c8f70 2024-04-25T10:34:54.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.813+0000 7f59e42a1640 1 -- 172.21.15.106:0/2491287111 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f59dc0c8f70 msgr2=0x7f59dc0c4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:54.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.813+0000 7f59e42a1640 1 --2- 172.21.15.106:0/2491287111 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f59dc0c8f70 0x7f59dc0c4de0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f59dc070080 tx=0x7f59d802a6c0 comp rx=0 tx=0).stop 2024-04-25T10:34:54.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.813+0000 7f59e42a1640 1 -- 172.21.15.106:0/2491287111 shutdown_connections 2024-04-25T10:34:54.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.813+0000 7f59e42a1640 1 --2- 172.21.15.106:0/2491287111 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f59dc0c8f70 0x7f59dc0c4de0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:54.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.813+0000 7f59e42a1640 1 -- 172.21.15.106:0/2491287111 >> 172.21.15.106:0/2491287111 conn(0x7f59dc0ce570 msgr2=0x7f59dc0ce980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:54.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.813+0000 7f59e42a1640 1 -- 172.21.15.106:0/2491287111 shutdown_connections 2024-04-25T10:34:54.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.813+0000 7f59e42a1640 1 -- 172.21.15.106:0/2491287111 wait complete. 2024-04-25T10:34:54.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.815+0000 7f59e42a1640 1 Processor -- start 2024-04-25T10:34:54.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.815+0000 7f59e42a1640 1 -- start start 2024-04-25T10:34:54.819 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.816+0000 7f59e42a1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f59dc0c3590 0x7f59dc1481c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:54.820 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.816+0000 7f59e42a1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59dc0c8f70 0x7f59dc14b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:54.820 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.816+0000 7f59e42a1640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f59dc153ca0 0x7f59dc145ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:54.820 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.816+0000 7f59e42a1640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f59dc0748d0 con 0x7f59dc0c3590 2024-04-25T10:34:54.820 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.816+0000 7f59e42a1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f59dc0caa10 con 0x7f59dc0c8f70 2024-04-25T10:34:54.820 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.816+0000 7f59e42a1640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f59dc074a50 con 0x7f59dc153ca0 2024-04-25T10:34:54.820 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.816+0000 7f59e203d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f59dc0c3590 0x7f59dc1481c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:54.820 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.816+0000 7f59e203d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f59dc0c3590 0x7f59dc1481c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53390/0 (socket says 172.21.15.106:53390) 2024-04-25T10:34:54.820 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.816+0000 7f59e203d640 1 -- 172.21.15.106:0/1307847608 learned_addr learned my addr 172.21.15.106:0/1307847608 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:34:54.820 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.816+0000 7f59e283e640 1 --2- 172.21.15.106:0/1307847608 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f59dc153ca0 0x7f59dc145ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:54.820 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.816+0000 7f59e183c640 1 --2- 172.21.15.106:0/1307847608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59dc0c8f70 0x7f59dc14b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:54.820 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.816+0000 7f59e203d640 1 -- 172.21.15.106:0/1307847608 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f59dc153ca0 msgr2=0x7f59dc145ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:54.820 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.816+0000 7f59e203d640 1 --2- 172.21.15.106:0/1307847608 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f59dc153ca0 0x7f59dc145ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:54.820 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.816+0000 7f59e203d640 1 -- 172.21.15.106:0/1307847608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59dc0c8f70 msgr2=0x7f59dc14b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:54.820 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.816+0000 7f59e203d640 1 --2- 172.21.15.106:0/1307847608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59dc0c8f70 0x7f59dc14b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:54.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.816+0000 7f59e203d640 1 -- 172.21.15.106:0/1307847608 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59dc1460e0 con 0x7f59dc0c3590 2024-04-25T10:34:54.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.816+0000 7f59e283e640 1 --2- 172.21.15.106:0/1307847608 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f59dc153ca0 0x7f59dc145ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:34:54.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.816+0000 7f59e183c640 1 --2- 172.21.15.106:0/1307847608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59dc0c8f70 0x7f59dc14b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:34:54.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.816+0000 7f59e203d640 1 --2- 172.21.15.106:0/1307847608 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f59dc0c3590 0x7f59dc1481c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f59d8009eb0 tx=0x7f59d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:54.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.817+0000 7f59d2ffd640 1 -- 172.21.15.106:0/1307847608 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d8009450 con 0x7f59dc0c3590 2024-04-25T10:34:54.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.817+0000 7f59d2ffd640 1 -- 172.21.15.106:0/1307847608 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59d8004240 con 0x7f59dc0c3590 2024-04-25T10:34:54.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.817+0000 7f59d2ffd640 1 -- 172.21.15.106:0/1307847608 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d8004610 con 0x7f59dc0c3590 2024-04-25T10:34:54.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.817+0000 7f59e42a1640 1 -- 172.21.15.106:0/1307847608 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59dc148c30 con 0x7f59dc0c3590 2024-04-25T10:34:54.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.817+0000 7f59e42a1640 1 -- 172.21.15.106:0/1307847608 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59dc14cbc0 con 0x7f59dc0c3590 2024-04-25T10:34:54.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.818+0000 7f59e42a1640 1 -- 172.21.15.106:0/1307847608 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59dc06a3d0 con 0x7f59dc0c3590 2024-04-25T10:34:54.822 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.818+0000 7f59d2ffd640 1 -- 172.21.15.106:0/1307847608 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f59d8039040 con 0x7f59dc0c3590 2024-04-25T10:34:54.822 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.819+0000 7f59d2ffd640 1 --2- 172.21.15.106:0/1307847608 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f59c00744e0 0x7f59c00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:54.822 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.819+0000 7f59e183c640 1 --2- 172.21.15.106:0/1307847608 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f59c00744e0 0x7f59c00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:54.822 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.819+0000 7f59d2ffd640 1 -- 172.21.15.106:0/1307847608 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f59d80b6680 con 0x7f59dc0c3590 2024-04-25T10:34:54.822 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.819+0000 7f59e183c640 1 --2- 172.21.15.106:0/1307847608 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f59c00744e0 0x7f59c00769f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f59dc148e50 tx=0x7f59cc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:54.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.823+0000 7f59d2ffd640 1 -- 172.21.15.106:0/1307847608 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f59d8040030 con 0x7f59dc0c3590 2024-04-25T10:34:54.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.996+0000 7f59e42a1640 1 -- 172.21.15.106:0/1307847608 --> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f59dc0c9ca0 con 0x7f59c00744e0 2024-04-25T10:34:54.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.997+0000 7f59d2ffd640 1 -- 172.21.15.106:0/1307847608 <== mgr.4107 v2:172.21.15.106:6832/1744210167 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f59dc0c9ca0 con 0x7f59c00744e0 2024-04-25T10:34:54.999 INFO:teuthology.orchestra.run.smithi106.stderr:instructed osd(s) 4 to deep-scrub 2024-04-25T10:34:55.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.999+0000 7f59e42a1640 1 -- 172.21.15.106:0/1307847608 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f59c00744e0 msgr2=0x7f59c00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:55.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.999+0000 7f59e42a1640 1 --2- 172.21.15.106:0/1307847608 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f59c00744e0 0x7f59c00769f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f59dc148e50 tx=0x7f59cc009040 comp rx=0 tx=0).stop 2024-04-25T10:34:55.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.999+0000 7f59e42a1640 1 -- 172.21.15.106:0/1307847608 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f59dc0c3590 msgr2=0x7f59dc1481c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:55.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.999+0000 7f59e42a1640 1 --2- 172.21.15.106:0/1307847608 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f59dc0c3590 0x7f59dc1481c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f59d8009eb0 tx=0x7f59d8004a40 comp rx=0 tx=0).stop 2024-04-25T10:34:55.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.999+0000 7f59e42a1640 1 -- 172.21.15.106:0/1307847608 shutdown_connections 2024-04-25T10:34:55.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.999+0000 7f59e42a1640 1 --2- 172.21.15.106:0/1307847608 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f59c00744e0 0x7f59c00769f0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.999+0000 7f59e42a1640 1 --2- 172.21.15.106:0/1307847608 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f59dc153ca0 0x7f59dc145ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.999+0000 7f59e42a1640 1 --2- 172.21.15.106:0/1307847608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59dc0c8f70 0x7f59dc14b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.999+0000 7f59e42a1640 1 --2- 172.21.15.106:0/1307847608 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f59dc0c3590 0x7f59dc1481c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.999+0000 7f59e42a1640 1 -- 172.21.15.106:0/1307847608 >> 172.21.15.106:0/1307847608 conn(0x7f59dc0ce570 msgr2=0x7f59dc144fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:55.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.999+0000 7f59e42a1640 1 -- 172.21.15.106:0/1307847608 shutdown_connections 2024-04-25T10:34:55.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:54.999+0000 7f59e42a1640 1 -- 172.21.15.106:0/1307847608 wait complete. 2024-04-25T10:34:55.013 INFO:tasks.ceph:Scrubbing osd.5 2024-04-25T10:34:55.013 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:34:55.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.123+0000 7f21b60e7640 1 Processor -- start 2024-04-25T10:34:55.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.123+0000 7f21b60e7640 1 -- start start 2024-04-25T10:34:55.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.124+0000 7f21b60e7640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f21b0153dd0 0x7f21b015a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:55.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.124+0000 7f21b60e7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21b00c8b50 0x7f21b00c8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:55.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.124+0000 7f21b60e7640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21b00c7000 con 0x7f21b015ae60 2024-04-25T10:34:55.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.124+0000 7f21b60e7640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f21b01541e0 con 0x7f21b0153dd0 2024-04-25T10:34:55.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.124+0000 7f21b60e7640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f21b015a9b0 con 0x7f21b00c8b50 2024-04-25T10:34:55.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.124+0000 7f21af7fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f21b015ae60 0x7f21b0074550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:49472/0 (socket says 172.21.15.106:49472) 2024-04-25T10:34:55.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.124+0000 7f21af7fe640 1 -- 172.21.15.106:0/1992793666 learned_addr learned my addr 172.21.15.106:0/1992793666 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:34:55.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.124+0000 7f21ae7fc640 1 -- 172.21.15.106:0/1992793666 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1990708691 0 0) 0x7f21b00c7000 con 0x7f21b015ae60 2024-04-25T10:34:55.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.124+0000 7f21ae7fc640 1 -- 172.21.15.106:0/1992793666 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2198003520 con 0x7f21b015ae60 2024-04-25T10:34:55.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.124+0000 7f21affff640 1 --2- 172.21.15.106:0/1992793666 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21b00c8b50 0x7f21b00c8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:55.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.124+0000 7f21aeffd640 1 --2- 172.21.15.106:0/1992793666 >> v2:172.21.15.134:3300/0 conn(0x7f21b0153dd0 0x7f21b015a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:55.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.125+0000 7f21ae7fc640 1 -- 172.21.15.106:0/1992793666 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3256718662 0 0) 0x7f2198003520 con 0x7f21b015ae60 2024-04-25T10:34:55.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.125+0000 7f21ae7fc640 1 -- 172.21.15.106:0/1992793666 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21b00c8b50 msgr2=0x7f21b00c8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:55.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.125+0000 7f21ae7fc640 1 --2- 172.21.15.106:0/1992793666 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21b00c8b50 0x7f21b00c8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.125+0000 7f21ae7fc640 1 -- 172.21.15.106:0/1992793666 >> v2:172.21.15.134:3300/0 conn(0x7f21b0153dd0 msgr2=0x7f21b015a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:55.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.125+0000 7f21ae7fc640 1 --2- 172.21.15.106:0/1992793666 >> v2:172.21.15.134:3300/0 conn(0x7f21b0153dd0 0x7f21b015a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.125+0000 7f21ae7fc640 1 -- 172.21.15.106:0/1992793666 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21b00c72e0 con 0x7f21b015ae60 2024-04-25T10:34:55.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.125+0000 7f21ae7fc640 1 -- 172.21.15.106:0/1992793666 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3776459350 0 0) 0x7f21a0001670 con 0x7f21b015ae60 2024-04-25T10:34:55.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.125+0000 7f21ae7fc640 1 -- 172.21.15.106:0/1992793666 >> v1:172.21.15.106:6789/0 conn(0x7f21b015ae60 legacy=0x7f21b0074550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:55.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.125+0000 7f21ae7fc640 1 --2- 172.21.15.106:0/1992793666 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21b00c8b50 0x7f21980039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:55.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.125+0000 7f21ae7fc640 1 --2- 172.21.15.106:0/1992793666 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2198003f30 0x7f2198004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:55.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.125+0000 7f21ae7fc640 1 --2- 172.21.15.106:0/1992793666 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2198004d40 0x7f2198005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:55.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.125+0000 7f21ae7fc640 1 -- 172.21.15.106:0/1992793666 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f21b01541e0 con 0x7f2198004d40 2024-04-25T10:34:55.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.125+0000 7f21ae7fc640 1 -- 172.21.15.106:0/1992793666 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f21b015a9b0 con 0x7f2198003f30 2024-04-25T10:34:55.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.125+0000 7f21ae7fc640 1 -- 172.21.15.106:0/1992793666 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f219800ba10 con 0x7f21b00c8b50 2024-04-25T10:34:55.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.125+0000 7f21affff640 1 --2- 172.21.15.106:0/1992793666 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2198004d40 0x7f2198005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:55.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.125+0000 7f21af7fe640 1 --2- 172.21.15.106:0/1992793666 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21b00c8b50 0x7f21980039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:55.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.125+0000 7f21aeffd640 1 --2- 172.21.15.106:0/1992793666 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2198003f30 0x7f2198004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:55.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.125+0000 7f21affff640 1 -- 172.21.15.106:0/1992793666 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21b00c8b50 msgr2=0x7f21980039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:55.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.125+0000 7f21affff640 1 --2- 172.21.15.106:0/1992793666 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21b00c8b50 0x7f21980039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.125+0000 7f21affff640 1 -- 172.21.15.106:0/1992793666 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2198003f30 msgr2=0x7f2198004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:55.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.125+0000 7f21affff640 1 --2- 172.21.15.106:0/1992793666 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2198003f30 0x7f2198004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.125+0000 7f21affff640 1 -- 172.21.15.106:0/1992793666 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2198003520 con 0x7f2198004d40 2024-04-25T10:34:55.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.126+0000 7f21affff640 1 --2- 172.21.15.106:0/1992793666 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2198004d40 0x7f2198005150 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f21b00c62f0 tx=0x7f2198005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cbcd7f01116ec90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:55.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.126+0000 7f21ae7fc640 1 -- 172.21.15.106:0/1992793666 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a8009c10 con 0x7f2198004d40 2024-04-25T10:34:55.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.126+0000 7f21ae7fc640 1 -- 172.21.15.106:0/1992793666 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21a8036070 con 0x7f2198004d40 2024-04-25T10:34:55.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.126+0000 7f21ae7fc640 1 -- 172.21.15.106:0/1992793666 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a8031830 con 0x7f2198004d40 2024-04-25T10:34:55.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.127+0000 7f21b60e7640 1 -- 172.21.15.106:0/1992793666 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2198004d40 msgr2=0x7f2198005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:55.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.127+0000 7f21b60e7640 1 --2- 172.21.15.106:0/1992793666 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2198004d40 0x7f2198005150 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f21b00c62f0 tx=0x7f2198005830 comp rx=0 tx=0).stop 2024-04-25T10:34:55.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.127+0000 7f21affff640 1 -- 172.21.15.106:0/1992793666 reap_dead start 2024-04-25T10:34:55.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.127+0000 7f21b60e7640 1 -- 172.21.15.106:0/1992793666 shutdown_connections 2024-04-25T10:34:55.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.127+0000 7f21b60e7640 1 -- 172.21.15.106:0/1992793666 >> 172.21.15.106:0/1992793666 conn(0x7f21b00cba20 msgr2=0x7f21b00cbe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:55.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.127+0000 7f21b60e7640 1 -- 172.21.15.106:0/1992793666 shutdown_connections 2024-04-25T10:34:55.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.127+0000 7f21b60e7640 1 -- 172.21.15.106:0/1992793666 wait complete. 2024-04-25T10:34:55.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.129+0000 7f21b60e7640 1 Processor -- start 2024-04-25T10:34:55.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.129+0000 7f21b60e7640 1 -- start start 2024-04-25T10:34:55.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.130+0000 7f21b60e7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2198004d40 0x7f21b00cb200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:55.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.130+0000 7f21b60e7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21b01f4960 0x7f21b01e65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:55.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.130+0000 7f21b60e7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21b0154360 0x7f21b0154770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:55.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.130+0000 7f21b60e7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f21b0073d50 con 0x7f21b0154360 2024-04-25T10:34:55.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.130+0000 7f21b60e7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f21b00740c0 con 0x7f2198004d40 2024-04-25T10:34:55.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.130+0000 7f21b60e7640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f21b00c6d10 con 0x7f21b01f4960 2024-04-25T10:34:55.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.130+0000 7f21affff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21b0154360 0x7f21b0154770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:55.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.130+0000 7f21affff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21b0154360 0x7f21b0154770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53420/0 (socket says 172.21.15.106:53420) 2024-04-25T10:34:55.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.130+0000 7f21affff640 1 -- 172.21.15.106:0/3815894143 learned_addr learned my addr 172.21.15.106:0/3815894143 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:34:55.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.130+0000 7f21aeffd640 1 --2- 172.21.15.106:0/3815894143 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21b01f4960 0x7f21b01e65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:55.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.130+0000 7f21af7fe640 1 --2- 172.21.15.106:0/3815894143 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2198004d40 0x7f21b00cb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:55.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.130+0000 7f21affff640 1 -- 172.21.15.106:0/3815894143 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21b01f4960 msgr2=0x7f21b01e65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:55.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.130+0000 7f21affff640 1 --2- 172.21.15.106:0/3815894143 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21b01f4960 0x7f21b01e65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.130+0000 7f21affff640 1 -- 172.21.15.106:0/3815894143 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2198004d40 msgr2=0x7f21b00cb200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:55.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.130+0000 7f21affff640 1 --2- 172.21.15.106:0/3815894143 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2198004d40 0x7f21b00cb200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.130+0000 7f21affff640 1 -- 172.21.15.106:0/3815894143 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21b00c9820 con 0x7f21b0154360 2024-04-25T10:34:55.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.131+0000 7f21affff640 1 --2- 172.21.15.106:0/3815894143 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21b0154360 0x7f21b0154770 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f21a8033040 tx=0x7f21a8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:55.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.131+0000 7f2197fff640 1 -- 172.21.15.106:0/3815894143 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a8009c10 con 0x7f21b0154360 2024-04-25T10:34:55.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.131+0000 7f2197fff640 1 -- 172.21.15.106:0/3815894143 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21a8003990 con 0x7f21b0154360 2024-04-25T10:34:55.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.131+0000 7f21b60e7640 1 -- 172.21.15.106:0/3815894143 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21b00c9b00 con 0x7f21b0154360 2024-04-25T10:34:55.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.131+0000 7f2197fff640 1 -- 172.21.15.106:0/3815894143 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a8003d80 con 0x7f21b0154360 2024-04-25T10:34:55.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.131+0000 7f21b60e7640 1 -- 172.21.15.106:0/3815894143 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21b01f4440 con 0x7f21b0154360 2024-04-25T10:34:55.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.132+0000 7f21b60e7640 1 -- 172.21.15.106:0/3815894143 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f21b0073440 con 0x7f21b0154360 2024-04-25T10:34:55.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.132+0000 7f2197fff640 1 -- 172.21.15.106:0/3815894143 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f21a8005d10 con 0x7f21b0154360 2024-04-25T10:34:55.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.133+0000 7f2197fff640 1 --2- 172.21.15.106:0/3815894143 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f219c0744b0 0x7f219c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:55.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.133+0000 7f2197fff640 1 -- 172.21.15.106:0/3815894143 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f21a80b67d0 con 0x7f21b0154360 2024-04-25T10:34:55.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.133+0000 7f21af7fe640 1 --2- 172.21.15.106:0/3815894143 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f219c0744b0 0x7f219c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:55.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.133+0000 7f2197fff640 1 --2- 172.21.15.106:0/3815894143 >> [v2:172.21.15.134:6808/3132275652,v1:172.21.15.134:6809/3132275652] conn(0x7f219c07c880 0x7f219c07ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:55.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.133+0000 7f2197fff640 1 -- 172.21.15.106:0/3815894143 --> [v2:172.21.15.134:6808/3132275652,v1:172.21.15.134:6809/3132275652] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f21a804de50 con 0x7f219c07c880 2024-04-25T10:34:55.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.133+0000 7f2197fff640 1 -- 172.21.15.106:0/3815894143 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f21a80bd050 con 0x7f21b0154360 2024-04-25T10:34:55.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.133+0000 7f21aeffd640 1 --2- 172.21.15.106:0/3815894143 >> [v2:172.21.15.134:6808/3132275652,v1:172.21.15.134:6809/3132275652] conn(0x7f219c07c880 0x7f219c07ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:55.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.133+0000 7f21af7fe640 1 --2- 172.21.15.106:0/3815894143 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f219c0744b0 0x7f219c0769c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f21b0157620 tx=0x7f21a00078f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:55.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.134+0000 7f21aeffd640 1 --2- 172.21.15.106:0/3815894143 >> [v2:172.21.15.134:6808/3132275652,v1:172.21.15.134:6809/3132275652] conn(0x7f219c07c880 0x7f219c07ed50 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:55.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.134+0000 7f2197fff640 1 -- 172.21.15.106:0/3815894143 <== osd.5 v2:172.21.15.134:6808/3132275652 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f21a804de50 con 0x7f219c07c880 2024-04-25T10:34:55.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.148+0000 7f21b60e7640 1 -- 172.21.15.106:0/3815894143 --> [v2:172.21.15.134:6808/3132275652,v1:172.21.15.134:6809/3132275652] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f21b01f0db0 con 0x7f219c07c880 2024-04-25T10:34:55.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.148+0000 7f2197fff640 1 -- 172.21.15.106:0/3815894143 <== osd.5 v2:172.21.15.134:6808/3132275652 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f21b01f0db0 con 0x7f219c07c880 2024-04-25T10:34:55.150 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-25T10:34:55.150 INFO:teuthology.orchestra.run.smithi106.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:34:55.150 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-25T10:34:55.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.148+0000 7f21b60e7640 1 -- 172.21.15.106:0/3815894143 >> [v2:172.21.15.134:6808/3132275652,v1:172.21.15.134:6809/3132275652] conn(0x7f219c07c880 msgr2=0x7f219c07ed50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:55.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.148+0000 7f21b60e7640 1 --2- 172.21.15.106:0/3815894143 >> [v2:172.21.15.134:6808/3132275652,v1:172.21.15.134:6809/3132275652] conn(0x7f219c07c880 0x7f219c07ed50 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.148+0000 7f21b60e7640 1 -- 172.21.15.106:0/3815894143 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f219c0744b0 msgr2=0x7f219c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:55.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.148+0000 7f21b60e7640 1 --2- 172.21.15.106:0/3815894143 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f219c0744b0 0x7f219c0769c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f21b0157620 tx=0x7f21a00078f0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.148+0000 7f21b60e7640 1 -- 172.21.15.106:0/3815894143 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21b0154360 msgr2=0x7f21b0154770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:55.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.148+0000 7f21b60e7640 1 --2- 172.21.15.106:0/3815894143 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21b0154360 0x7f21b0154770 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f21a8033040 tx=0x7f21a8009750 comp rx=0 tx=0).stop 2024-04-25T10:34:55.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.148+0000 7f21affff640 1 -- 172.21.15.106:0/3815894143 reap_dead start 2024-04-25T10:34:55.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.149+0000 7f21b60e7640 1 -- 172.21.15.106:0/3815894143 shutdown_connections 2024-04-25T10:34:55.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.149+0000 7f21b60e7640 1 -- 172.21.15.106:0/3815894143 >> 172.21.15.106:0/3815894143 conn(0x7f21b00cba20 msgr2=0x7f21b0152070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:55.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.149+0000 7f21b60e7640 1 -- 172.21.15.106:0/3815894143 shutdown_connections 2024-04-25T10:34:55.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.149+0000 7f21b60e7640 1 -- 172.21.15.106:0/3815894143 wait complete. 2024-04-25T10:34:55.164 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-25T10:34:55.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.271+0000 7f6c022f2640 1 Processor -- start 2024-04-25T10:34:55.274 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.271+0000 7f6c022f2640 1 -- start start 2024-04-25T10:34:55.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.273+0000 7f6c022f2640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6bfc154c60 0x7f6bfc070ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:55.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.273+0000 7f6c022f2640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f6bfc15e020 0x7f6bfc15e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:55.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.273+0000 7f6c022f2640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6bfc0caa00 con 0x7f6bfc071220 2024-04-25T10:34:55.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.273+0000 7f6c022f2640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f6bfc0cb880 con 0x7f6bfc154c60 2024-04-25T10:34:55.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.273+0000 7f6c022f2640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f6bfc0cac70 con 0x7f6bfc15e020 2024-04-25T10:34:55.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.273+0000 7f6bfbfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6bfc154c60 0x7f6bfc070ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:55.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.273+0000 7f6bfbfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6bfc154c60 0x7f6bfc070ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53428/0 (socket says 172.21.15.106:53428) 2024-04-25T10:34:55.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.273+0000 7f6bfbfff640 1 -- 172.21.15.106:0/3020588809 learned_addr learned my addr 172.21.15.106:0/3020588809 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:34:55.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.273+0000 7f6c0088f640 1 --2- 172.21.15.106:0/3020588809 >> v2:172.21.15.134:3300/0 conn(0x7f6bfc15e020 0x7f6bfc15e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:55.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.273+0000 7f6bfbfff640 1 -- 172.21.15.106:0/3020588809 >> v2:172.21.15.134:3300/0 conn(0x7f6bfc15e020 msgr2=0x7f6bfc15e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:55.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.273+0000 7f6bfbfff640 1 --2- 172.21.15.106:0/3020588809 >> v2:172.21.15.134:3300/0 conn(0x7f6bfc15e020 0x7f6bfc15e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.273+0000 7f6bfbfff640 1 -- 172.21.15.106:0/3020588809 >> v1:172.21.15.134:6789/0 conn(0x7f6bfc071220 legacy=0x7f6bfc0c5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:55.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.273+0000 7f6bfbfff640 1 -- 172.21.15.106:0/3020588809 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bfc0caf50 con 0x7f6bfc154c60 2024-04-25T10:34:55.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.273+0000 7f6c0088f640 1 --2- 172.21.15.106:0/3020588809 >> v2:172.21.15.134:3300/0 conn(0x7f6bfc15e020 0x7f6bfc15e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:34:55.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.274+0000 7f6bfbfff640 1 --2- 172.21.15.106:0/3020588809 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6bfc154c60 0x7f6bfc070ce0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc074a40 tx=0x7f6bec02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ce7f03a2afad6b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:55.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.274+0000 7f6bfaffd640 1 -- 172.21.15.106:0/3020588809 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bec037070 con 0x7f6bfc154c60 2024-04-25T10:34:55.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.274+0000 7f6bfaffd640 1 -- 172.21.15.106:0/3020588809 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bec032440 con 0x7f6bfc154c60 2024-04-25T10:34:55.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.274+0000 7f6bfaffd640 1 -- 172.21.15.106:0/3020588809 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bec032810 con 0x7f6bfc154c60 2024-04-25T10:34:55.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.275+0000 7f6c022f2640 1 -- 172.21.15.106:0/3020588809 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6bfc154c60 msgr2=0x7f6bfc070ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:55.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.275+0000 7f6c022f2640 1 --2- 172.21.15.106:0/3020588809 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6bfc154c60 0x7f6bfc070ce0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc074a40 tx=0x7f6bec02a270 comp rx=0 tx=0).stop 2024-04-25T10:34:55.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.275+0000 7f6c022f2640 1 -- 172.21.15.106:0/3020588809 shutdown_connections 2024-04-25T10:34:55.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.275+0000 7f6c022f2640 1 --2- 172.21.15.106:0/3020588809 >> v2:172.21.15.134:3300/0 conn(0x7f6bfc15e020 0x7f6bfc15e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.275+0000 7f6c022f2640 1 --2- 172.21.15.106:0/3020588809 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6bfc154c60 0x7f6bfc070ce0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.275+0000 7f6c022f2640 1 -- 172.21.15.106:0/3020588809 >> 172.21.15.106:0/3020588809 conn(0x7f6bfc0cff60 msgr2=0x7f6bfc0d0370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:55.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.275+0000 7f6c022f2640 1 -- 172.21.15.106:0/3020588809 shutdown_connections 2024-04-25T10:34:55.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.275+0000 7f6c022f2640 1 -- 172.21.15.106:0/3020588809 wait complete. 2024-04-25T10:34:55.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.277+0000 7f6c022f2640 1 Processor -- start 2024-04-25T10:34:55.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.277+0000 7f6c022f2640 1 -- start start 2024-04-25T10:34:55.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.278+0000 7f6c022f2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6bfc071220 0x7f6bfc0ce870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:55.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.278+0000 7f6c022f2640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6bfc154c60 0x7f6bfc14c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:55.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.278+0000 7f6c022f2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bfc15e020 0x7f6bfc1e6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:55.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.278+0000 7f6c022f2640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6bfc0c5b10 con 0x7f6bfc071220 2024-04-25T10:34:55.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.278+0000 7f6c022f2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6bfc0cb510 con 0x7f6bfc15e020 2024-04-25T10:34:55.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.278+0000 7f6c022f2640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6bfc0c5c90 con 0x7f6bfc154c60 2024-04-25T10:34:55.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.278+0000 7f6bfbfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6bfc071220 0x7f6bfc0ce870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:55.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.278+0000 7f6bfbfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6bfc071220 0x7f6bfc0ce870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53440/0 (socket says 172.21.15.106:53440) 2024-04-25T10:34:55.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.278+0000 7f6bfbfff640 1 -- 172.21.15.106:0/2096675730 learned_addr learned my addr 172.21.15.106:0/2096675730 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:34:55.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.278+0000 7f6c0088f640 1 --2- 172.21.15.106:0/2096675730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bfc15e020 0x7f6bfc1e6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:55.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.278+0000 7f6bfb7fe640 1 --2- 172.21.15.106:0/2096675730 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6bfc154c60 0x7f6bfc14c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:55.282 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.278+0000 7f6bfbfff640 1 -- 172.21.15.106:0/2096675730 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6bfc154c60 msgr2=0x7f6bfc14c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:55.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.278+0000 7f6bfbfff640 1 --2- 172.21.15.106:0/2096675730 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6bfc154c60 0x7f6bfc14c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.278+0000 7f6bfbfff640 1 -- 172.21.15.106:0/2096675730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bfc15e020 msgr2=0x7f6bfc1e6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:55.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.278+0000 7f6bfbfff640 1 --2- 172.21.15.106:0/2096675730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bfc15e020 0x7f6bfc1e6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.278+0000 7f6bfbfff640 1 -- 172.21.15.106:0/2096675730 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bfc15de00 con 0x7f6bfc071220 2024-04-25T10:34:55.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.278+0000 7f6c0088f640 1 --2- 172.21.15.106:0/2096675730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bfc15e020 0x7f6bfc1e6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:34:55.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.278+0000 7f6bfbfff640 1 --2- 172.21.15.106:0/2096675730 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6bfc071220 0x7f6bfc0ce870 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6bec02a2b0 tx=0x7f6bec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:55.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.279+0000 7f6bf8ff9640 1 -- 172.21.15.106:0/2096675730 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bec041020 con 0x7f6bfc071220 2024-04-25T10:34:55.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.279+0000 7f6bf8ff9640 1 -- 172.21.15.106:0/2096675730 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bec0306c0 con 0x7f6bfc071220 2024-04-25T10:34:55.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.279+0000 7f6bf8ff9640 1 -- 172.21.15.106:0/2096675730 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bec037040 con 0x7f6bfc071220 2024-04-25T10:34:55.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.279+0000 7f6c022f2640 1 -- 172.21.15.106:0/2096675730 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bfc14ce60 con 0x7f6bfc071220 2024-04-25T10:34:55.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.279+0000 7f6c022f2640 1 -- 172.21.15.106:0/2096675730 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bfc0cf4c0 con 0x7f6bfc071220 2024-04-25T10:34:55.283 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.280+0000 7f6c022f2640 1 -- 172.21.15.106:0/2096675730 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bfc06a3d0 con 0x7f6bfc071220 2024-04-25T10:34:55.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.280+0000 7f6bf8ff9640 1 -- 172.21.15.106:0/2096675730 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6bec044020 con 0x7f6bfc071220 2024-04-25T10:34:55.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.281+0000 7f6bf8ff9640 1 --2- 172.21.15.106:0/2096675730 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f6be80744b0 0x7f6be80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:55.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.281+0000 7f6bf8ff9640 1 -- 172.21.15.106:0/2096675730 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f6bec0b5a80 con 0x7f6bfc071220 2024-04-25T10:34:55.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.281+0000 7f6bfb7fe640 1 --2- 172.21.15.106:0/2096675730 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f6be80744b0 0x7f6be80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:55.284 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.281+0000 7f6bfb7fe640 1 --2- 172.21.15.106:0/2096675730 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f6be80744b0 0x7f6be80769c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6bf0007fd0 tx=0x7f6bf0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:55.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.285+0000 7f6bf8ff9640 1 -- 172.21.15.106:0/2096675730 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6bec04b030 con 0x7f6bfc071220 2024-04-25T10:34:55.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.464+0000 7f6c022f2640 1 -- 172.21.15.106:0/2096675730 --> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f6bfc0cbe80 con 0x7f6be80744b0 2024-04-25T10:34:55.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.464+0000 7f6bf8ff9640 1 -- 172.21.15.106:0/2096675730 <== mgr.4107 v2:172.21.15.106:6832/1744210167 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f6bfc0cbe80 con 0x7f6be80744b0 2024-04-25T10:34:55.467 INFO:teuthology.orchestra.run.smithi106.stderr:instructed osd(s) 5 to deep-scrub 2024-04-25T10:34:55.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.467+0000 7f6c022f2640 1 -- 172.21.15.106:0/2096675730 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f6be80744b0 msgr2=0x7f6be80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:55.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.467+0000 7f6c022f2640 1 --2- 172.21.15.106:0/2096675730 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f6be80744b0 0x7f6be80769c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6bf0007fd0 tx=0x7f6bf0009040 comp rx=0 tx=0).stop 2024-04-25T10:34:55.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.467+0000 7f6c022f2640 1 -- 172.21.15.106:0/2096675730 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6bfc071220 msgr2=0x7f6bfc0ce870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:55.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.467+0000 7f6c022f2640 1 --2- 172.21.15.106:0/2096675730 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6bfc071220 0x7f6bfc0ce870 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6bec02a2b0 tx=0x7f6bec0046c0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.467+0000 7f6c022f2640 1 -- 172.21.15.106:0/2096675730 shutdown_connections 2024-04-25T10:34:55.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.467+0000 7f6c022f2640 1 --2- 172.21.15.106:0/2096675730 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f6be80744b0 0x7f6be80769c0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.467+0000 7f6c022f2640 1 --2- 172.21.15.106:0/2096675730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bfc15e020 0x7f6bfc1e6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.467+0000 7f6c022f2640 1 --2- 172.21.15.106:0/2096675730 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6bfc154c60 0x7f6bfc14c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.467+0000 7f6c022f2640 1 --2- 172.21.15.106:0/2096675730 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6bfc071220 0x7f6bfc0ce870 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.467+0000 7f6c022f2640 1 -- 172.21.15.106:0/2096675730 >> 172.21.15.106:0/2096675730 conn(0x7f6bfc0cff60 msgr2=0x7f6bfc1512b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:55.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.467+0000 7f6c022f2640 1 -- 172.21.15.106:0/2096675730 shutdown_connections 2024-04-25T10:34:55.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.467+0000 7f6c022f2640 1 -- 172.21.15.106:0/2096675730 wait complete. 2024-04-25T10:34:55.480 INFO:tasks.ceph:Scrubbing osd.6 2024-04-25T10:34:55.480 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:34:55.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.589+0000 7fa7545d7640 1 Processor -- start 2024-04-25T10:34:55.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.589+0000 7fa7545d7640 1 -- start start 2024-04-25T10:34:55.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.591+0000 7fa7545d7640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fa74c0c52e0 0x7fa74c15b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:55.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.591+0000 7fa7545d7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa74c0caea0 0x7fa74c1652a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:55.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.591+0000 7fa7545d7640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa74c0c56f0 con 0x7fa74c0caa90 2024-04-25T10:34:55.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.591+0000 7fa7545d7640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fa74c0cc8f0 con 0x7fa74c0c52e0 2024-04-25T10:34:55.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.591+0000 7fa7545d7640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa74c0cc050 con 0x7fa74c0caea0 2024-04-25T10:34:55.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.591+0000 7fa751b72640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fa74c0caa90 0x7fa74c0ccb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:49484/0 (socket says 172.21.15.106:49484) 2024-04-25T10:34:55.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.591+0000 7fa751b72640 1 -- 172.21.15.106:0/2742250620 learned_addr learned my addr 172.21.15.106:0/2742250620 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:34:55.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.591+0000 7fa752b74640 1 --2- 172.21.15.106:0/2742250620 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa74c0caea0 0x7fa74c1652a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:55.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.591+0000 7fa752373640 1 --2- 172.21.15.106:0/2742250620 >> v2:172.21.15.134:3300/0 conn(0x7fa74c0c52e0 0x7fa74c15b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:55.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.591+0000 7fa751371640 1 -- 172.21.15.106:0/2742250620 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1352841910 0 0) 0x7fa74c0c56f0 con 0x7fa74c0caa90 2024-04-25T10:34:55.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.591+0000 7fa751371640 1 -- 172.21.15.106:0/2742250620 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa738003520 con 0x7fa74c0caa90 2024-04-25T10:34:55.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.591+0000 7fa751371640 1 -- 172.21.15.106:0/2742250620 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1194973148 0 0) 0x7fa738003520 con 0x7fa74c0caa90 2024-04-25T10:34:55.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.591+0000 7fa751371640 1 -- 172.21.15.106:0/2742250620 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa74c0caea0 msgr2=0x7fa74c1652a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:55.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.592+0000 7fa751371640 1 --2- 172.21.15.106:0/2742250620 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa74c0caea0 0x7fa74c1652a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.592+0000 7fa751371640 1 -- 172.21.15.106:0/2742250620 >> v2:172.21.15.134:3300/0 conn(0x7fa74c0c52e0 msgr2=0x7fa74c15b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:55.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.592+0000 7fa751371640 1 --2- 172.21.15.106:0/2742250620 >> v2:172.21.15.134:3300/0 conn(0x7fa74c0c52e0 0x7fa74c15b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.592+0000 7fa751371640 1 -- 172.21.15.106:0/2742250620 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa74c0cc330 con 0x7fa74c0caa90 2024-04-25T10:34:55.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.592+0000 7fa751371640 1 -- 172.21.15.106:0/2742250620 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3776459350 0 0) 0x7fa73c001670 con 0x7fa74c0caa90 2024-04-25T10:34:55.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.592+0000 7fa752b74640 1 --2- 172.21.15.106:0/2742250620 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa74c0caea0 0x7fa74c1652a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:34:55.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.592+0000 7fa751371640 1 -- 172.21.15.106:0/2742250620 >> v1:172.21.15.106:6789/0 conn(0x7fa74c0caa90 legacy=0x7fa74c0ccb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:55.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.592+0000 7fa751371640 1 --2- 172.21.15.106:0/2742250620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7380039b0 0x7fa738003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:55.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.592+0000 7fa751371640 1 --2- 172.21.15.106:0/2742250620 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa74c0caea0 0x7fa738004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:55.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.592+0000 7fa751371640 1 --2- 172.21.15.106:0/2742250620 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa738004d40 0x7fa738005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:55.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.592+0000 7fa751371640 1 -- 172.21.15.106:0/2742250620 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa74c0cc8f0 con 0x7fa738004d40 2024-04-25T10:34:55.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.592+0000 7fa751371640 1 -- 172.21.15.106:0/2742250620 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa74c0cc050 con 0x7fa7380039b0 2024-04-25T10:34:55.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.592+0000 7fa751371640 1 -- 172.21.15.106:0/2742250620 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa73800b9c0 con 0x7fa74c0caea0 2024-04-25T10:34:55.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.592+0000 7fa752373640 1 --2- 172.21.15.106:0/2742250620 >> v2:172.21.15.134:3300/0 conn(0x7fa74c0c52e0 0x7fa74c15b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:34:55.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.592+0000 7fa752b74640 1 --2- 172.21.15.106:0/2742250620 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa738004d40 0x7fa738005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:55.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.592+0000 7fa751b72640 1 --2- 172.21.15.106:0/2742250620 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa74c0caea0 0x7fa738004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:55.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.592+0000 7fa752373640 1 --2- 172.21.15.106:0/2742250620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7380039b0 0x7fa738003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:55.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.592+0000 7fa752b74640 1 -- 172.21.15.106:0/2742250620 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa74c0caea0 msgr2=0x7fa738004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:55.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.592+0000 7fa752b74640 1 --2- 172.21.15.106:0/2742250620 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa74c0caea0 0x7fa738004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.592+0000 7fa752b74640 1 -- 172.21.15.106:0/2742250620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7380039b0 msgr2=0x7fa738003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:55.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.592+0000 7fa752b74640 1 --2- 172.21.15.106:0/2742250620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7380039b0 0x7fa738003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.592+0000 7fa752b74640 1 -- 172.21.15.106:0/2742250620 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa738003520 con 0x7fa738004d40 2024-04-25T10:34:55.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.592+0000 7fa752373640 1 --2- 172.21.15.106:0/2742250620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7380039b0 0x7fa738003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:34:55.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.593+0000 7fa752b74640 1 --2- 172.21.15.106:0/2742250620 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa738004d40 0x7fa738005150 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa744009c90 tx=0x7fa738005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2c0fb984c8b3caa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:55.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.593+0000 7fa751371640 1 -- 172.21.15.106:0/2742250620 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa744036070 con 0x7fa738004d40 2024-04-25T10:34:55.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.593+0000 7fa751371640 1 -- 172.21.15.106:0/2742250620 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa744031400 con 0x7fa738004d40 2024-04-25T10:34:55.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.593+0000 7fa751371640 1 -- 172.21.15.106:0/2742250620 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa744009860 con 0x7fa738004d40 2024-04-25T10:34:55.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.594+0000 7fa7545d7640 1 -- 172.21.15.106:0/2742250620 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa738004d40 msgr2=0x7fa738005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:55.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.594+0000 7fa7545d7640 1 --2- 172.21.15.106:0/2742250620 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa738004d40 0x7fa738005150 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa744009c90 tx=0x7fa738005830 comp rx=0 tx=0).stop 2024-04-25T10:34:55.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.594+0000 7fa752b74640 1 -- 172.21.15.106:0/2742250620 reap_dead start 2024-04-25T10:34:55.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.594+0000 7fa7545d7640 1 -- 172.21.15.106:0/2742250620 shutdown_connections 2024-04-25T10:34:55.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.594+0000 7fa7545d7640 1 -- 172.21.15.106:0/2742250620 >> 172.21.15.106:0/2742250620 conn(0x7fa74c068cd0 msgr2=0x7fa74c072800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:55.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.594+0000 7fa7545d7640 1 -- 172.21.15.106:0/2742250620 shutdown_connections 2024-04-25T10:34:55.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.594+0000 7fa7545d7640 1 -- 172.21.15.106:0/2742250620 wait complete. 2024-04-25T10:34:55.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.595+0000 7fa7545d7640 1 Processor -- start 2024-04-25T10:34:55.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.595+0000 7fa7545d7640 1 -- start start 2024-04-25T10:34:55.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.597+0000 7fa7545d7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa738004d40 0x7fa74c15bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:55.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.597+0000 7fa7545d7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa74c0b50d0 0x7fa74c0d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:55.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.597+0000 7fa7545d7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa74c151150 0x7fa74c151560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:55.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.597+0000 7fa7545d7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa74c15ada0 con 0x7fa74c0b50d0 2024-04-25T10:34:55.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.597+0000 7fa7545d7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa74c15aac0 con 0x7fa74c151150 2024-04-25T10:34:55.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.597+0000 7fa7545d7640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa74c0cbd30 con 0x7fa738004d40 2024-04-25T10:34:55.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.597+0000 7fa751b72640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa74c0b50d0 0x7fa74c0d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:55.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.597+0000 7fa751b72640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa74c0b50d0 0x7fa74c0d01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53464/0 (socket says 172.21.15.106:53464) 2024-04-25T10:34:55.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.597+0000 7fa751b72640 1 -- 172.21.15.106:0/1097163429 learned_addr learned my addr 172.21.15.106:0/1097163429 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:34:55.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.597+0000 7fa752373640 1 --2- 172.21.15.106:0/1097163429 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa738004d40 0x7fa74c15bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:55.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.597+0000 7fa752b74640 1 --2- 172.21.15.106:0/1097163429 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa74c151150 0x7fa74c151560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:55.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.597+0000 7fa751b72640 1 -- 172.21.15.106:0/1097163429 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa738004d40 msgr2=0x7fa74c15bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:55.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.597+0000 7fa751b72640 1 --2- 172.21.15.106:0/1097163429 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa738004d40 0x7fa74c15bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.597+0000 7fa751b72640 1 -- 172.21.15.106:0/1097163429 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa74c151150 msgr2=0x7fa74c151560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:55.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.597+0000 7fa751b72640 1 --2- 172.21.15.106:0/1097163429 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa74c151150 0x7fa74c151560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.597+0000 7fa751b72640 1 -- 172.21.15.106:0/1097163429 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa74c15e9a0 con 0x7fa74c0b50d0 2024-04-25T10:34:55.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.597+0000 7fa751b72640 1 --2- 172.21.15.106:0/1097163429 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa74c0b50d0 0x7fa74c0d01e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa73c007310 tx=0x7fa73c0077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:55.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.597+0000 7fa742ffd640 1 -- 172.21.15.106:0/1097163429 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa73c00f9a0 con 0x7fa74c0b50d0 2024-04-25T10:34:55.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.597+0000 7fa742ffd640 1 -- 172.21.15.106:0/1097163429 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa73c018070 con 0x7fa74c0b50d0 2024-04-25T10:34:55.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.597+0000 7fa742ffd640 1 -- 172.21.15.106:0/1097163429 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa73c00fdc0 con 0x7fa74c0b50d0 2024-04-25T10:34:55.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.597+0000 7fa7545d7640 1 -- 172.21.15.106:0/1097163429 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa74c15c120 con 0x7fa74c0b50d0 2024-04-25T10:34:55.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.597+0000 7fa7545d7640 1 -- 172.21.15.106:0/1097163429 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa74c1f1790 con 0x7fa74c0b50d0 2024-04-25T10:34:55.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.598+0000 7fa7545d7640 1 -- 172.21.15.106:0/1097163429 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa74c06a3d0 con 0x7fa74c0b50d0 2024-04-25T10:34:55.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.599+0000 7fa742ffd640 1 -- 172.21.15.106:0/1097163429 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa73c01d460 con 0x7fa74c0b50d0 2024-04-25T10:34:55.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.599+0000 7fa742ffd640 1 --2- 172.21.15.106:0/1097163429 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fa7300744b0 0x7fa7300769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:55.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.600+0000 7fa742ffd640 1 -- 172.21.15.106:0/1097163429 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fa73c0954d0 con 0x7fa74c0b50d0 2024-04-25T10:34:55.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.600+0000 7fa752373640 1 --2- 172.21.15.106:0/1097163429 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fa7300744b0 0x7fa7300769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:55.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.600+0000 7fa742ffd640 1 --2- 172.21.15.106:0/1097163429 >> [v2:172.21.15.134:6800/992702668,v1:172.21.15.134:6801/992702668] conn(0x7fa73007c880 0x7fa73007ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:55.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.600+0000 7fa742ffd640 1 -- 172.21.15.106:0/1097163429 --> [v2:172.21.15.134:6800/992702668,v1:172.21.15.134:6801/992702668] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa73007f420 con 0x7fa73007c880 2024-04-25T10:34:55.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.600+0000 7fa742ffd640 1 -- 172.21.15.106:0/1097163429 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fa73c09c050 con 0x7fa74c0b50d0 2024-04-25T10:34:55.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.600+0000 7fa752b74640 1 --2- 172.21.15.106:0/1097163429 >> [v2:172.21.15.134:6800/992702668,v1:172.21.15.134:6801/992702668] conn(0x7fa73007c880 0x7fa73007ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:55.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.600+0000 7fa752373640 1 --2- 172.21.15.106:0/1097163429 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fa7300744b0 0x7fa7300769c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa748003be0 tx=0x7fa74800b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:55.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.600+0000 7fa752b74640 1 --2- 172.21.15.106:0/1097163429 >> [v2:172.21.15.134:6800/992702668,v1:172.21.15.134:6801/992702668] conn(0x7fa73007c880 0x7fa73007ed50 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:55.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.601+0000 7fa742ffd640 1 -- 172.21.15.106:0/1097163429 <== osd.6 v2:172.21.15.134:6800/992702668 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa73007f420 con 0x7fa73007c880 2024-04-25T10:34:55.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.614+0000 7fa7545d7640 1 -- 172.21.15.106:0/1097163429 --> [v2:172.21.15.134:6800/992702668,v1:172.21.15.134:6801/992702668] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fa74c0725f0 con 0x7fa73007c880 2024-04-25T10:34:55.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.614+0000 7fa742ffd640 1 -- 172.21.15.106:0/1097163429 <== osd.6 v2:172.21.15.134:6800/992702668 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fa74c0725f0 con 0x7fa73007c880 2024-04-25T10:34:55.617 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-25T10:34:55.617 INFO:teuthology.orchestra.run.smithi106.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:34:55.617 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-25T10:34:55.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.615+0000 7fa7545d7640 1 -- 172.21.15.106:0/1097163429 >> [v2:172.21.15.134:6800/992702668,v1:172.21.15.134:6801/992702668] conn(0x7fa73007c880 msgr2=0x7fa73007ed50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:55.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.615+0000 7fa7545d7640 1 --2- 172.21.15.106:0/1097163429 >> [v2:172.21.15.134:6800/992702668,v1:172.21.15.134:6801/992702668] conn(0x7fa73007c880 0x7fa73007ed50 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.615+0000 7fa7545d7640 1 -- 172.21.15.106:0/1097163429 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fa7300744b0 msgr2=0x7fa7300769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:55.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.615+0000 7fa7545d7640 1 --2- 172.21.15.106:0/1097163429 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fa7300744b0 0x7fa7300769c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa748003be0 tx=0x7fa74800b040 comp rx=0 tx=0).stop 2024-04-25T10:34:55.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.615+0000 7fa7545d7640 1 -- 172.21.15.106:0/1097163429 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa74c0b50d0 msgr2=0x7fa74c0d01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:55.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.615+0000 7fa7545d7640 1 --2- 172.21.15.106:0/1097163429 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa74c0b50d0 0x7fa74c0d01e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa73c007310 tx=0x7fa73c0077d0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.615+0000 7fa752b74640 1 -- 172.21.15.106:0/1097163429 reap_dead start 2024-04-25T10:34:55.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.615+0000 7fa7545d7640 1 -- 172.21.15.106:0/1097163429 shutdown_connections 2024-04-25T10:34:55.618 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.615+0000 7fa7545d7640 1 -- 172.21.15.106:0/1097163429 >> 172.21.15.106:0/1097163429 conn(0x7fa74c068cd0 msgr2=0x7fa74c1536d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:55.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.615+0000 7fa7545d7640 1 -- 172.21.15.106:0/1097163429 shutdown_connections 2024-04-25T10:34:55.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.615+0000 7fa7545d7640 1 -- 172.21.15.106:0/1097163429 wait complete. 2024-04-25T10:34:55.629 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-25T10:34:55.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.735+0000 7f5210405640 1 Processor -- start 2024-04-25T10:34:55.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.735+0000 7f5210405640 1 -- start start 2024-04-25T10:34:55.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.736+0000 7f5210405640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f520815a590 0x7f52080c5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:55.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.736+0000 7f5210405640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f520815d9a0 0x7f5208153710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:55.739 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.736+0000 7f5210405640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52080c7c40 con 0x7f5208072870 2024-04-25T10:34:55.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.736+0000 7f5210405640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f52080cb590 con 0x7f520815a590 2024-04-25T10:34:55.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.736+0000 7f5210405640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f520815a9a0 con 0x7f520815d9a0 2024-04-25T10:34:55.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.737+0000 7f520e9a2640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f520815d9a0 0x7f5208153710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:55.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.737+0000 7f520e9a2640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f520815d9a0 0x7f5208153710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.106:55834/0 (socket says 172.21.15.106:55834) 2024-04-25T10:34:55.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.737+0000 7f520e9a2640 1 -- 172.21.15.106:0/1416951454 learned_addr learned my addr 172.21.15.106:0/1416951454 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:34:55.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.737+0000 7f520d9a0640 1 --2- 172.21.15.106:0/1416951454 >> v2:172.21.15.134:3300/0 conn(0x7f520815a590 0x7f52080c5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:55.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.737+0000 7f520e9a2640 1 -- 172.21.15.106:0/1416951454 >> v2:172.21.15.134:3300/0 conn(0x7f520815a590 msgr2=0x7f52080c5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:55.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.737+0000 7f520e9a2640 1 --2- 172.21.15.106:0/1416951454 >> v2:172.21.15.134:3300/0 conn(0x7f520815a590 0x7f52080c5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.737+0000 7f520e9a2640 1 -- 172.21.15.106:0/1416951454 >> v1:172.21.15.134:6789/0 conn(0x7f5208072870 legacy=0x7f52080cae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:55.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.737+0000 7f520e9a2640 1 -- 172.21.15.106:0/1416951454 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52080c7f20 con 0x7f520815d9a0 2024-04-25T10:34:55.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.737+0000 7f520d9a0640 1 --2- 172.21.15.106:0/1416951454 >> v2:172.21.15.134:3300/0 conn(0x7f520815a590 0x7f52080c5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:34:55.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.738+0000 7f520e9a2640 1 --2- 172.21.15.106:0/1416951454 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f520815d9a0 0x7f5208153710 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5200009870 tx=0x7f520002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=345a2a50c5b8676a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:55.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.738+0000 7f520d19f640 1 -- 172.21.15.106:0/1416951454 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5200037070 con 0x7f520815d9a0 2024-04-25T10:34:55.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.738+0000 7f520d19f640 1 -- 172.21.15.106:0/1416951454 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5200032440 con 0x7f520815d9a0 2024-04-25T10:34:55.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.738+0000 7f520d19f640 1 -- 172.21.15.106:0/1416951454 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5200032810 con 0x7f520815d9a0 2024-04-25T10:34:55.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.739+0000 7f5210405640 1 -- 172.21.15.106:0/1416951454 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f520815d9a0 msgr2=0x7f5208153710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:55.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.739+0000 7f5210405640 1 --2- 172.21.15.106:0/1416951454 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f520815d9a0 0x7f5208153710 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5200009870 tx=0x7f520002a270 comp rx=0 tx=0).stop 2024-04-25T10:34:55.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.739+0000 7f5210405640 1 -- 172.21.15.106:0/1416951454 shutdown_connections 2024-04-25T10:34:55.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.739+0000 7f5210405640 1 --2- 172.21.15.106:0/1416951454 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f520815d9a0 0x7f5208153710 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.739+0000 7f5210405640 1 --2- 172.21.15.106:0/1416951454 >> v2:172.21.15.134:3300/0 conn(0x7f520815a590 0x7f52080c5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.739+0000 7f5210405640 1 -- 172.21.15.106:0/1416951454 >> 172.21.15.106:0/1416951454 conn(0x7f520806c9b0 msgr2=0x7f520806cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:55.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.739+0000 7f5210405640 1 -- 172.21.15.106:0/1416951454 shutdown_connections 2024-04-25T10:34:55.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.739+0000 7f5210405640 1 -- 172.21.15.106:0/1416951454 wait complete. 2024-04-25T10:34:55.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.741+0000 7f5210405640 1 Processor -- start 2024-04-25T10:34:55.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.741+0000 7f5210405640 1 -- start start 2024-04-25T10:34:55.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.742+0000 7f5210405640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5208072870 0x7f52080cded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:55.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.742+0000 7f5210405640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f520815a590 0x7f52081e34a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:55.745 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.742+0000 7f5210405640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f520815d9a0 0x7f52081e7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:55.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.742+0000 7f5210405640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5208159d90 con 0x7f520815d9a0 2024-04-25T10:34:55.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.742+0000 7f5210405640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5208159ab0 con 0x7f520815a590 2024-04-25T10:34:55.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.742+0000 7f5210405640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f520815a070 con 0x7f5208072870 2024-04-25T10:34:55.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.742+0000 7f520e9a2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f520815d9a0 0x7f52081e7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:55.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.742+0000 7f520e9a2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f520815d9a0 0x7f52081e7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53478/0 (socket says 172.21.15.106:53478) 2024-04-25T10:34:55.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.742+0000 7f520e9a2640 1 -- 172.21.15.106:0/860623129 learned_addr learned my addr 172.21.15.106:0/860623129 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:34:55.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.742+0000 7f520e1a1640 1 --2- 172.21.15.106:0/860623129 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5208072870 0x7f52080cded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:55.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.742+0000 7f520d9a0640 1 --2- 172.21.15.106:0/860623129 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f520815a590 0x7f52081e34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:55.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.742+0000 7f520e9a2640 1 -- 172.21.15.106:0/860623129 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5208072870 msgr2=0x7f52080cded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:55.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.742+0000 7f520e9a2640 1 --2- 172.21.15.106:0/860623129 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5208072870 0x7f52080cded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.742+0000 7f520e9a2640 1 -- 172.21.15.106:0/860623129 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f520815a590 msgr2=0x7f52081e34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:55.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.742+0000 7f520e9a2640 1 --2- 172.21.15.106:0/860623129 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f520815a590 0x7f52081e34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.746 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.742+0000 7f520e9a2640 1 -- 172.21.15.106:0/860623129 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5208154870 con 0x7f520815d9a0 2024-04-25T10:34:55.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.742+0000 7f520e9a2640 1 --2- 172.21.15.106:0/860623129 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f520815d9a0 0x7f52081e7600 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f520002a2b0 tx=0x7f52000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:55.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.743+0000 7f51feffd640 1 -- 172.21.15.106:0/860623129 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5200041020 con 0x7f520815d9a0 2024-04-25T10:34:55.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.743+0000 7f51feffd640 1 -- 172.21.15.106:0/860623129 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f520003d640 con 0x7f520815d9a0 2024-04-25T10:34:55.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.743+0000 7f5210405640 1 -- 172.21.15.106:0/860623129 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52080ce410 con 0x7f520815d9a0 2024-04-25T10:34:55.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.743+0000 7f51feffd640 1 -- 172.21.15.106:0/860623129 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5200037040 con 0x7f520815d9a0 2024-04-25T10:34:55.747 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.743+0000 7f5210405640 1 -- 172.21.15.106:0/860623129 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52080cd710 con 0x7f520815d9a0 2024-04-25T10:34:55.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.744+0000 7f5210405640 1 -- 172.21.15.106:0/860623129 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5208159110 con 0x7f520815d9a0 2024-04-25T10:34:55.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.744+0000 7f51feffd640 1 -- 172.21.15.106:0/860623129 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5200044020 con 0x7f520815d9a0 2024-04-25T10:34:55.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.745+0000 7f51feffd640 1 --2- 172.21.15.106:0/860623129 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f51ec0744b0 0x7f51ec0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:55.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.745+0000 7f51feffd640 1 -- 172.21.15.106:0/860623129 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f52000b5850 con 0x7f520815d9a0 2024-04-25T10:34:55.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.745+0000 7f520e1a1640 1 --2- 172.21.15.106:0/860623129 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f51ec0744b0 0x7f51ec0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:55.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.745+0000 7f520e1a1640 1 --2- 172.21.15.106:0/860623129 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f51ec0744b0 0x7f51ec0769c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5204001e70 tx=0x7f5204009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:55.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.749+0000 7f51feffd640 1 -- 172.21.15.106:0/860623129 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f520004b030 con 0x7f520815d9a0 2024-04-25T10:34:55.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.924+0000 7f5210405640 1 -- 172.21.15.106:0/860623129 --> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f520806a3d0 con 0x7f51ec0744b0 2024-04-25T10:34:55.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.925+0000 7f51feffd640 1 -- 172.21.15.106:0/860623129 <== mgr.4107 v2:172.21.15.106:6832/1744210167 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f520806a3d0 con 0x7f51ec0744b0 2024-04-25T10:34:55.927 INFO:teuthology.orchestra.run.smithi106.stderr:instructed osd(s) 6 to deep-scrub 2024-04-25T10:34:55.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.927+0000 7f5210405640 1 -- 172.21.15.106:0/860623129 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f51ec0744b0 msgr2=0x7f51ec0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:55.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.927+0000 7f5210405640 1 --2- 172.21.15.106:0/860623129 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f51ec0744b0 0x7f51ec0769c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5204001e70 tx=0x7f5204009040 comp rx=0 tx=0).stop 2024-04-25T10:34:55.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.927+0000 7f5210405640 1 -- 172.21.15.106:0/860623129 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f520815d9a0 msgr2=0x7f52081e7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:55.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.927+0000 7f5210405640 1 --2- 172.21.15.106:0/860623129 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f520815d9a0 0x7f52081e7600 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f520002a2b0 tx=0x7f52000046c0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.927+0000 7f5210405640 1 -- 172.21.15.106:0/860623129 shutdown_connections 2024-04-25T10:34:55.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.927+0000 7f5210405640 1 --2- 172.21.15.106:0/860623129 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f51ec0744b0 0x7f51ec0769c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.927+0000 7f5210405640 1 --2- 172.21.15.106:0/860623129 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f520815d9a0 0x7f52081e7600 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.927+0000 7f5210405640 1 --2- 172.21.15.106:0/860623129 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f520815a590 0x7f52081e34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.927+0000 7f5210405640 1 --2- 172.21.15.106:0/860623129 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5208072870 0x7f52080cded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:55.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.927+0000 7f5210405640 1 -- 172.21.15.106:0/860623129 >> 172.21.15.106:0/860623129 conn(0x7f520806c9b0 msgr2=0x7f5208070980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:55.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.927+0000 7f5210405640 1 -- 172.21.15.106:0/860623129 shutdown_connections 2024-04-25T10:34:55.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:55.927+0000 7f5210405640 1 -- 172.21.15.106:0/860623129 wait complete. 2024-04-25T10:34:55.940 INFO:tasks.ceph:Scrubbing osd.7 2024-04-25T10:34:55.941 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:34:56.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.050+0000 7f2709384640 1 Processor -- start 2024-04-25T10:34:56.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.050+0000 7f2709384640 1 -- start start 2024-04-25T10:34:56.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.051+0000 7f2709384640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27040cb520 0x7f2704071630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:56.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.051+0000 7f2709384640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f270415d580 con 0x7f2704157060 2024-04-25T10:34:56.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.051+0000 7f2709384640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f270415d860 con 0x7f270415cbf0 2024-04-25T10:34:56.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.051+0000 7f2709384640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f2704071b70 con 0x7f27040cb520 2024-04-25T10:34:56.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.051+0000 7f27027fc640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f2704157060 0x7f27040cabe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:49500/0 (socket says 172.21.15.106:49500) 2024-04-25T10:34:56.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.051+0000 7f27027fc640 1 -- 172.21.15.106:0/675702511 learned_addr learned my addr 172.21.15.106:0/675702511 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:34:56.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.051+0000 7f2702ffd640 1 --2- 172.21.15.106:0/675702511 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27040cb520 0x7f2704071630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:56.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.051+0000 7f2701ffb640 1 -- 172.21.15.106:0/675702511 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3527256206 0 0) 0x7f270415d580 con 0x7f2704157060 2024-04-25T10:34:56.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.051+0000 7f2701ffb640 1 -- 172.21.15.106:0/675702511 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f26ec003520 con 0x7f2704157060 2024-04-25T10:34:56.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.052+0000 7f2701ffb640 1 -- 172.21.15.106:0/675702511 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1402251930 0 0) 0x7f26ec003520 con 0x7f2704157060 2024-04-25T10:34:56.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.052+0000 7f2701ffb640 1 -- 172.21.15.106:0/675702511 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27040cb520 msgr2=0x7f2704071630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:56.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.052+0000 7f2701ffb640 1 --2- 172.21.15.106:0/675702511 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27040cb520 0x7f2704071630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:56.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.052+0000 7f2701ffb640 1 -- 172.21.15.106:0/675702511 >> v1:172.21.15.134:6789/0 conn(0x7f270415cbf0 legacy=0x7f270406dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:56.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.052+0000 7f2701ffb640 1 -- 172.21.15.106:0/675702511 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2704070b50 con 0x7f2704157060 2024-04-25T10:34:56.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.052+0000 7f2701ffb640 1 -- 172.21.15.106:0/675702511 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3776459350 0 0) 0x7f26f8001690 con 0x7f2704157060 2024-04-25T10:34:56.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.052+0000 7f2701ffb640 1 -- 172.21.15.106:0/675702511 >> v1:172.21.15.106:6789/0 conn(0x7f2704157060 legacy=0x7f27040cabe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:56.059 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.052+0000 7f2701ffb640 1 --2- 172.21.15.106:0/675702511 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26ec0039e0 0x7f26ec003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:56.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.052+0000 7f2701ffb640 1 --2- 172.21.15.106:0/675702511 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26ec004800 0x7f26ec004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:56.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.052+0000 7f2701ffb640 1 --2- 172.21.15.106:0/675702511 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27040cb520 0x7f26ec0052d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:56.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.052+0000 7f2701ffb640 1 -- 172.21.15.106:0/675702511 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2704071b70 con 0x7f26ec0039e0 2024-04-25T10:34:56.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.052+0000 7f2701ffb640 1 -- 172.21.15.106:0/675702511 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f26ec00bb80 con 0x7f26ec004800 2024-04-25T10:34:56.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.052+0000 7f2701ffb640 1 -- 172.21.15.106:0/675702511 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f26ec00bd80 con 0x7f27040cb520 2024-04-25T10:34:56.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.052+0000 7f2702ffd640 1 --2- 172.21.15.106:0/675702511 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26ec0039e0 0x7f26ec003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:56.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.052+0000 7f27027fc640 1 --2- 172.21.15.106:0/675702511 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26ec004800 0x7f26ec004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:56.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.052+0000 7f27037fe640 1 --2- 172.21.15.106:0/675702511 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27040cb520 0x7f26ec0052d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:56.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.052+0000 7f2702ffd640 1 -- 172.21.15.106:0/675702511 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27040cb520 msgr2=0x7f26ec0052d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:56.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.052+0000 7f2702ffd640 1 --2- 172.21.15.106:0/675702511 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27040cb520 0x7f26ec0052d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:56.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.052+0000 7f2702ffd640 1 -- 172.21.15.106:0/675702511 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26ec004800 msgr2=0x7f26ec004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:56.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.052+0000 7f2702ffd640 1 --2- 172.21.15.106:0/675702511 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26ec004800 0x7f26ec004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:56.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.052+0000 7f2702ffd640 1 -- 172.21.15.106:0/675702511 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26ec003520 con 0x7f26ec0039e0 2024-04-25T10:34:56.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.053+0000 7f2702ffd640 1 --2- 172.21.15.106:0/675702511 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26ec0039e0 0x7f26ec003e50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f26f4009fc0 tx=0x7f26ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a5079f989afb454 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:56.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.053+0000 7f2701ffb640 1 -- 172.21.15.106:0/675702511 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f4034020 con 0x7f26ec0039e0 2024-04-25T10:34:56.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.053+0000 7f2701ffb640 1 -- 172.21.15.106:0/675702511 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26f40305f0 con 0x7f26ec0039e0 2024-04-25T10:34:56.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.053+0000 7f2701ffb640 1 -- 172.21.15.106:0/675702511 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f4009c10 con 0x7f26ec0039e0 2024-04-25T10:34:56.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.054+0000 7f2709384640 1 -- 172.21.15.106:0/675702511 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26ec0039e0 msgr2=0x7f26ec003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:56.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.054+0000 7f2709384640 1 --2- 172.21.15.106:0/675702511 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26ec0039e0 0x7f26ec003e50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f26f4009fc0 tx=0x7f26ec005830 comp rx=0 tx=0).stop 2024-04-25T10:34:56.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.054+0000 7f27037fe640 1 -- 172.21.15.106:0/675702511 reap_dead start 2024-04-25T10:34:56.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.054+0000 7f2709384640 1 -- 172.21.15.106:0/675702511 shutdown_connections 2024-04-25T10:34:56.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.054+0000 7f2709384640 1 -- 172.21.15.106:0/675702511 >> 172.21.15.106:0/675702511 conn(0x7f2704068cd0 msgr2=0x7f2704072800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:56.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.054+0000 7f2709384640 1 -- 172.21.15.106:0/675702511 shutdown_connections 2024-04-25T10:34:56.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.054+0000 7f2709384640 1 -- 172.21.15.106:0/675702511 wait complete. 2024-04-25T10:34:56.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.056+0000 7f2709384640 1 Processor -- start 2024-04-25T10:34:56.062 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.056+0000 7f2709384640 1 -- start start 2024-04-25T10:34:56.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.057+0000 7f2709384640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26ec0039e0 0x7f270414de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:56.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.057+0000 7f2709384640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27040cf8b0 0x7f27040cea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:56.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.057+0000 7f2709384640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f270414bab0 0x7f270414bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:56.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.057+0000 7f2709384640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2704071110 con 0x7f270414bab0 2024-04-25T10:34:56.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.057+0000 7f2709384640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f270415db40 con 0x7f26ec0039e0 2024-04-25T10:34:56.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.057+0000 7f2709384640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f270406e6d0 con 0x7f27040cf8b0 2024-04-25T10:34:56.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.057+0000 7f27037fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f270414bab0 0x7f270414bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:56.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.057+0000 7f27037fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f270414bab0 0x7f270414bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53484/0 (socket says 172.21.15.106:53484) 2024-04-25T10:34:56.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.057+0000 7f27037fe640 1 -- 172.21.15.106:0/3520143516 learned_addr learned my addr 172.21.15.106:0/3520143516 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:34:56.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.057+0000 7f2702ffd640 1 --2- 172.21.15.106:0/3520143516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26ec0039e0 0x7f270414de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:56.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.057+0000 7f27027fc640 1 --2- 172.21.15.106:0/3520143516 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27040cf8b0 0x7f27040cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:56.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.057+0000 7f27037fe640 1 -- 172.21.15.106:0/3520143516 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27040cf8b0 msgr2=0x7f27040cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:56.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.057+0000 7f27037fe640 1 --2- 172.21.15.106:0/3520143516 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27040cf8b0 0x7f27040cea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:56.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.057+0000 7f27037fe640 1 -- 172.21.15.106:0/3520143516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26ec0039e0 msgr2=0x7f270414de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:56.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.057+0000 7f27037fe640 1 --2- 172.21.15.106:0/3520143516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26ec0039e0 0x7f270414de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:56.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.057+0000 7f27037fe640 1 -- 172.21.15.106:0/3520143516 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f270414e3a0 con 0x7f270414bab0 2024-04-25T10:34:56.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.057+0000 7f27037fe640 1 --2- 172.21.15.106:0/3520143516 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f270414bab0 0x7f270414bec0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2704074bf0 tx=0x7f26fc00f920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:56.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.058+0000 7f26eb7fe640 1 -- 172.21.15.106:0/3520143516 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26fc0053a0 con 0x7f270414bab0 2024-04-25T10:34:56.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.058+0000 7f26eb7fe640 1 -- 172.21.15.106:0/3520143516 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26fc018070 con 0x7f270414bab0 2024-04-25T10:34:56.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.058+0000 7f2709384640 1 -- 172.21.15.106:0/3520143516 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27040ce5e0 con 0x7f270414bab0 2024-04-25T10:34:56.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.058+0000 7f26eb7fe640 1 -- 172.21.15.106:0/3520143516 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26fc003830 con 0x7f270414bab0 2024-04-25T10:34:56.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.058+0000 7f2709384640 1 -- 172.21.15.106:0/3520143516 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27041f38c0 con 0x7f270414bab0 2024-04-25T10:34:56.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.059+0000 7f2709384640 1 -- 172.21.15.106:0/3520143516 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f27040cd200 con 0x7f270414bab0 2024-04-25T10:34:56.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.059+0000 7f26eb7fe640 1 -- 172.21.15.106:0/3520143516 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f26fc0039d0 con 0x7f270414bab0 2024-04-25T10:34:56.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.060+0000 7f26eb7fe640 1 --2- 172.21.15.106:0/3520143516 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f26f00744b0 0x7f26f00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:56.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.060+0000 7f2702ffd640 1 --2- 172.21.15.106:0/3520143516 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f26f00744b0 0x7f26f00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:56.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.060+0000 7f26eb7fe640 1 -- 172.21.15.106:0/3520143516 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f26fc094c50 con 0x7f270414bab0 2024-04-25T10:34:56.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.060+0000 7f26eb7fe640 1 --2- 172.21.15.106:0/3520143516 >> [v2:172.21.15.134:6816/3291062284,v1:172.21.15.134:6817/3291062284] conn(0x7f26f007c820 0x7f26f007ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:56.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.060+0000 7f26eb7fe640 1 -- 172.21.15.106:0/3520143516 --> [v2:172.21.15.134:6816/3291062284,v1:172.21.15.134:6817/3291062284] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f26fc010e50 con 0x7f26f007c820 2024-04-25T10:34:56.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.060+0000 7f26eb7fe640 1 -- 172.21.15.106:0/3520143516 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f26fc097350 con 0x7f270414bab0 2024-04-25T10:34:56.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.060+0000 7f2702ffd640 1 --2- 172.21.15.106:0/3520143516 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f26f00744b0 0x7f26f00769c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f26f4035000 tx=0x7f26f402f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:56.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.060+0000 7f27027fc640 1 --2- 172.21.15.106:0/3520143516 >> [v2:172.21.15.134:6816/3291062284,v1:172.21.15.134:6817/3291062284] conn(0x7f26f007c820 0x7f26f007ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:56.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.061+0000 7f27027fc640 1 --2- 172.21.15.106:0/3520143516 >> [v2:172.21.15.134:6816/3291062284,v1:172.21.15.134:6817/3291062284] conn(0x7f26f007c820 0x7f26f007ecf0 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:56.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.061+0000 7f26eb7fe640 1 -- 172.21.15.106:0/3520143516 <== osd.7 v2:172.21.15.134:6816/3291062284 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f26fc010e50 con 0x7f26f007c820 2024-04-25T10:34:56.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.074+0000 7f2709384640 1 -- 172.21.15.106:0/3520143516 --> [v2:172.21.15.134:6816/3291062284,v1:172.21.15.134:6817/3291062284] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f27041f3390 con 0x7f26f007c820 2024-04-25T10:34:56.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.075+0000 7f26eb7fe640 1 -- 172.21.15.106:0/3520143516 <== osd.7 v2:172.21.15.134:6816/3291062284 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f27041f3390 con 0x7f26f007c820 2024-04-25T10:34:56.077 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-25T10:34:56.077 INFO:teuthology.orchestra.run.smithi106.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:34:56.077 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-25T10:34:56.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.075+0000 7f2709384640 1 -- 172.21.15.106:0/3520143516 >> [v2:172.21.15.134:6816/3291062284,v1:172.21.15.134:6817/3291062284] conn(0x7f26f007c820 msgr2=0x7f26f007ecf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:56.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.075+0000 7f2709384640 1 --2- 172.21.15.106:0/3520143516 >> [v2:172.21.15.134:6816/3291062284,v1:172.21.15.134:6817/3291062284] conn(0x7f26f007c820 0x7f26f007ecf0 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:56.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.075+0000 7f2709384640 1 -- 172.21.15.106:0/3520143516 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f26f00744b0 msgr2=0x7f26f00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:56.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.075+0000 7f2709384640 1 --2- 172.21.15.106:0/3520143516 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f26f00744b0 0x7f26f00769c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f26f4035000 tx=0x7f26f402f000 comp rx=0 tx=0).stop 2024-04-25T10:34:56.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.075+0000 7f2709384640 1 -- 172.21.15.106:0/3520143516 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f270414bab0 msgr2=0x7f270414bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:56.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.075+0000 7f2709384640 1 --2- 172.21.15.106:0/3520143516 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f270414bab0 0x7f270414bec0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2704074bf0 tx=0x7f26fc00f920 comp rx=0 tx=0).stop 2024-04-25T10:34:56.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.075+0000 7f27037fe640 1 -- 172.21.15.106:0/3520143516 reap_dead start 2024-04-25T10:34:56.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.075+0000 7f2709384640 1 -- 172.21.15.106:0/3520143516 shutdown_connections 2024-04-25T10:34:56.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.075+0000 7f2709384640 1 -- 172.21.15.106:0/3520143516 >> 172.21.15.106:0/3520143516 conn(0x7f2704068cd0 msgr2=0x7f27040c5700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:56.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.075+0000 7f2709384640 1 -- 172.21.15.106:0/3520143516 shutdown_connections 2024-04-25T10:34:56.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.075+0000 7f2709384640 1 -- 172.21.15.106:0/3520143516 wait complete. 2024-04-25T10:34:56.089 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-25T10:34:56.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.198+0000 7f30094ca640 1 Processor -- start 2024-04-25T10:34:56.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.198+0000 7f30094ca640 1 -- start start 2024-04-25T10:34:56.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.199+0000 7f30094ca640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f30040c8d90 0x7f3004156ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:56.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.199+0000 7f30094ca640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f30040b4ce0 0x7f300415ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:56.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.199+0000 7f30094ca640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30040ce630 con 0x7f30040c7e30 2024-04-25T10:34:56.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.199+0000 7f30094ca640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f30040b50f0 con 0x7f30040c8d90 2024-04-25T10:34:56.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.199+0000 7f30094ca640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f30040c7c60 con 0x7f30040b4ce0 2024-04-25T10:34:56.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.200+0000 7f30027fc640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f30040c8d90 0x7f3004156ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:56.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.200+0000 7f30027fc640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f30040c8d90 0x7f3004156ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53486/0 (socket says 172.21.15.106:53486) 2024-04-25T10:34:56.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.200+0000 7f30027fc640 1 -- 172.21.15.106:0/2852195704 learned_addr learned my addr 172.21.15.106:0/2852195704 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:34:56.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.200+0000 7f30037fe640 1 --2- 172.21.15.106:0/2852195704 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f30040b4ce0 0x7f300415ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:56.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.200+0000 7f30027fc640 1 -- 172.21.15.106:0/2852195704 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f30040b4ce0 msgr2=0x7f300415ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:56.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.200+0000 7f30027fc640 1 --2- 172.21.15.106:0/2852195704 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f30040b4ce0 0x7f300415ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:56.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.200+0000 7f30027fc640 1 -- 172.21.15.106:0/2852195704 >> v1:172.21.15.134:6789/0 conn(0x7f30040c7e30 legacy=0x7f300406d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:56.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.200+0000 7f30027fc640 1 -- 172.21.15.106:0/2852195704 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30040c7980 con 0x7f30040c8d90 2024-04-25T10:34:56.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.201+0000 7f30027fc640 1 --2- 172.21.15.106:0/2852195704 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f30040c8d90 0x7f3004156ef0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f300408ffe0 tx=0x7f2ff802a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ad7dbf7f7f63d58 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:56.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.201+0000 7f3001ffb640 1 -- 172.21.15.106:0/2852195704 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff802ae40 con 0x7f30040c8d90 2024-04-25T10:34:56.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.201+0000 7f3001ffb640 1 -- 172.21.15.106:0/2852195704 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ff8004030 con 0x7f30040c8d90 2024-04-25T10:34:56.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.201+0000 7f3001ffb640 1 -- 172.21.15.106:0/2852195704 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff8004460 con 0x7f30040c8d90 2024-04-25T10:34:56.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.201+0000 7f30094ca640 1 -- 172.21.15.106:0/2852195704 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f30040c8d90 msgr2=0x7f3004156ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:56.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.201+0000 7f30094ca640 1 --2- 172.21.15.106:0/2852195704 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f30040c8d90 0x7f3004156ef0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f300408ffe0 tx=0x7f2ff802a130 comp rx=0 tx=0).stop 2024-04-25T10:34:56.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.201+0000 7f30094ca640 1 -- 172.21.15.106:0/2852195704 shutdown_connections 2024-04-25T10:34:56.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.201+0000 7f30094ca640 1 --2- 172.21.15.106:0/2852195704 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f30040b4ce0 0x7f300415ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:56.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.201+0000 7f30094ca640 1 --2- 172.21.15.106:0/2852195704 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f30040c8d90 0x7f3004156ef0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:56.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.201+0000 7f30094ca640 1 -- 172.21.15.106:0/2852195704 >> 172.21.15.106:0/2852195704 conn(0x7f3004068b10 msgr2=0x7f30040d1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:56.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.201+0000 7f30094ca640 1 -- 172.21.15.106:0/2852195704 shutdown_connections 2024-04-25T10:34:56.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.201+0000 7f30094ca640 1 -- 172.21.15.106:0/2852195704 wait complete. 2024-04-25T10:34:56.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.203+0000 7f30094ca640 1 Processor -- start 2024-04-25T10:34:56.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.203+0000 7f30094ca640 1 -- start start 2024-04-25T10:34:56.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.205+0000 7f30094ca640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f30040b4ce0 0x7f300414dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:56.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.205+0000 7f30094ca640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f30040c7e30 0x7f300414eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:56.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.205+0000 7f30094ca640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f30040c8d90 0x7f300414b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:56.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.205+0000 7f30094ca640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f300406d710 con 0x7f30040c7e30 2024-04-25T10:34:56.208 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.205+0000 7f30094ca640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f30040c72f0 con 0x7f30040c8d90 2024-04-25T10:34:56.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.205+0000 7f30094ca640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f300406d890 con 0x7f30040b4ce0 2024-04-25T10:34:56.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.205+0000 7f30027fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f30040c7e30 0x7f300414eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:56.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.205+0000 7f30027fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f30040c7e30 0x7f300414eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53494/0 (socket says 172.21.15.106:53494) 2024-04-25T10:34:56.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.205+0000 7f30027fc640 1 -- 172.21.15.106:0/3425177080 learned_addr learned my addr 172.21.15.106:0/3425177080 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:34:56.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.205+0000 7f30037fe640 1 --2- 172.21.15.106:0/3425177080 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f30040c8d90 0x7f300414b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:56.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.205+0000 7f3002ffd640 1 --2- 172.21.15.106:0/3425177080 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f30040b4ce0 0x7f300414dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:56.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.205+0000 7f30027fc640 1 -- 172.21.15.106:0/3425177080 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f30040b4ce0 msgr2=0x7f300414dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:56.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.205+0000 7f30027fc640 1 --2- 172.21.15.106:0/3425177080 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f30040b4ce0 0x7f300414dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:56.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.205+0000 7f30027fc640 1 -- 172.21.15.106:0/3425177080 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f30040c8d90 msgr2=0x7f300414b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:56.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.205+0000 7f30027fc640 1 --2- 172.21.15.106:0/3425177080 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f30040c8d90 0x7f300414b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:56.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.205+0000 7f30027fc640 1 -- 172.21.15.106:0/3425177080 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30041f6660 con 0x7f30040c7e30 2024-04-25T10:34:56.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.205+0000 7f30027fc640 1 --2- 172.21.15.106:0/3425177080 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f30040c7e30 0x7f300414eb70 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2ff802a170 tx=0x7f2ff80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:56.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.205+0000 7f2feb7fe640 1 -- 172.21.15.106:0/3425177080 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff8041020 con 0x7f30040c7e30 2024-04-25T10:34:56.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.205+0000 7f2feb7fe640 1 -- 172.21.15.106:0/3425177080 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ff8005e50 con 0x7f30040c7e30 2024-04-25T10:34:56.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.205+0000 7f30094ca640 1 -- 172.21.15.106:0/3425177080 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30040cd0f0 con 0x7f30040c7e30 2024-04-25T10:34:56.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.205+0000 7f2feb7fe640 1 -- 172.21.15.106:0/3425177080 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff8006040 con 0x7f30040c7e30 2024-04-25T10:34:56.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.205+0000 7f30094ca640 1 -- 172.21.15.106:0/3425177080 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30040cd3b0 con 0x7f30040c7e30 2024-04-25T10:34:56.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.206+0000 7f30094ca640 1 -- 172.21.15.106:0/3425177080 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f300406a3d0 con 0x7f30040c7e30 2024-04-25T10:34:56.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.207+0000 7f2feb7fe640 1 -- 172.21.15.106:0/3425177080 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2ff8044020 con 0x7f30040c7e30 2024-04-25T10:34:56.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.208+0000 7f2feb7fe640 1 --2- 172.21.15.106:0/3425177080 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f2ff00744b0 0x7f2ff00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:56.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.208+0000 7f3002ffd640 1 --2- 172.21.15.106:0/3425177080 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f2ff00744b0 0x7f2ff00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:56.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.208+0000 7f2feb7fe640 1 -- 172.21.15.106:0/3425177080 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f2ff80b5fc0 con 0x7f30040c7e30 2024-04-25T10:34:56.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.208+0000 7f3002ffd640 1 --2- 172.21.15.106:0/3425177080 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f2ff00744b0 0x7f2ff00769c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4006fd0 tx=0x7f2ff4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:56.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.212+0000 7f2feb7fe640 1 -- 172.21.15.106:0/3425177080 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2ff804b030 con 0x7f30040c7e30 2024-04-25T10:34:56.392 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.390+0000 7f30094ca640 1 -- 172.21.15.106:0/3425177080 --> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f300415ac20 con 0x7f2ff00744b0 2024-04-25T10:34:56.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.390+0000 7f2feb7fe640 1 -- 172.21.15.106:0/3425177080 <== mgr.4107 v2:172.21.15.106:6832/1744210167 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f300415ac20 con 0x7f2ff00744b0 2024-04-25T10:34:56.393 INFO:teuthology.orchestra.run.smithi106.stderr:instructed osd(s) 7 to deep-scrub 2024-04-25T10:34:56.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.392+0000 7f30094ca640 1 -- 172.21.15.106:0/3425177080 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f2ff00744b0 msgr2=0x7f2ff00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:56.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.392+0000 7f30094ca640 1 --2- 172.21.15.106:0/3425177080 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f2ff00744b0 0x7f2ff00769c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4006fd0 tx=0x7f2ff4008040 comp rx=0 tx=0).stop 2024-04-25T10:34:56.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.392+0000 7f30094ca640 1 -- 172.21.15.106:0/3425177080 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f30040c7e30 msgr2=0x7f300414eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:56.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.392+0000 7f30094ca640 1 --2- 172.21.15.106:0/3425177080 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f30040c7e30 0x7f300414eb70 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2ff802a170 tx=0x7f2ff80046c0 comp rx=0 tx=0).stop 2024-04-25T10:34:56.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.392+0000 7f30094ca640 1 -- 172.21.15.106:0/3425177080 shutdown_connections 2024-04-25T10:34:56.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.392+0000 7f30094ca640 1 --2- 172.21.15.106:0/3425177080 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f2ff00744b0 0x7f2ff00769c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:56.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.392+0000 7f30094ca640 1 --2- 172.21.15.106:0/3425177080 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f30040c8d90 0x7f300414b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:56.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.392+0000 7f30094ca640 1 --2- 172.21.15.106:0/3425177080 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f30040c7e30 0x7f300414eb70 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:56.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.392+0000 7f30094ca640 1 --2- 172.21.15.106:0/3425177080 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f30040b4ce0 0x7f300414dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:56.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.392+0000 7f30094ca640 1 -- 172.21.15.106:0/3425177080 >> 172.21.15.106:0/3425177080 conn(0x7f3004068b10 msgr2=0x7f300415bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:56.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.392+0000 7f30094ca640 1 -- 172.21.15.106:0/3425177080 shutdown_connections 2024-04-25T10:34:56.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.392+0000 7f30094ca640 1 -- 172.21.15.106:0/3425177080 wait complete. 2024-04-25T10:34:56.406 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:34:56.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.513+0000 7f10edba8640 1 Processor -- start 2024-04-25T10:34:56.516 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.513+0000 7f10edba8640 1 -- start start 2024-04-25T10:34:56.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.515+0000 7f10edba8640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f10e80c61a0 0x7f10e8156f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:56.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.515+0000 7f10edba8640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f10e815d690 0x7f10e80cce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:56.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.515+0000 7f10edba8640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10e815cbd0 con 0x7f10e806dc00 2024-04-25T10:34:56.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.515+0000 7f10edba8640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f10e815d200 con 0x7f10e815d690 2024-04-25T10:34:56.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.515+0000 7f10edba8640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f10e8160990 con 0x7f10e80c61a0 2024-04-25T10:34:56.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.515+0000 7f10e7fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f10e815d690 0x7f10e80cce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:56.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.515+0000 7f10e6ffd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f10e806dc00 0x7f10e81601c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:49508/0 (socket says 172.21.15.106:49508) 2024-04-25T10:34:56.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.515+0000 7f10e6ffd640 1 -- 172.21.15.106:0/2497834621 learned_addr learned my addr 172.21.15.106:0/2497834621 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:34:56.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.515+0000 7f10e7fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f10e815d690 0x7f10e80cce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53508/0 (socket says 172.21.15.106:53508) 2024-04-25T10:34:56.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.515+0000 7f10e77fe640 1 --2- 172.21.15.106:0/2497834621 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f10e80c61a0 0x7f10e8156f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:56.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.515+0000 7f10e7fff640 1 -- 172.21.15.106:0/2497834621 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f10e80c61a0 msgr2=0x7f10e8156f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:34:56.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.515+0000 7f10e7fff640 1 --2- 172.21.15.106:0/2497834621 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f10e80c61a0 0x7f10e8156f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:56.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.515+0000 7f10e7fff640 1 -- 172.21.15.106:0/2497834621 >> v1:172.21.15.106:6789/0 conn(0x7f10e806dc00 legacy=0x7f10e81601c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:56.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.515+0000 7f10e67fc640 1 -- 172.21.15.106:0/2497834621 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3329053865 0 0) 0x7f10e815cbd0 con 0x7f10e806dc00 2024-04-25T10:34:56.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.515+0000 7f10e7fff640 1 -- 172.21.15.106:0/2497834621 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10e8160c70 con 0x7f10e815d690 2024-04-25T10:34:56.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.515+0000 7f10e77fe640 1 --2- 172.21.15.106:0/2497834621 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f10e80c61a0 0x7f10e8156f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:34:56.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.516+0000 7f10e7fff640 1 --2- 172.21.15.106:0/2497834621 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10e815d690 0x7f10e80cce20 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f10e0009d90 tx=0x7f10e002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b792f215426119b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:56.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.516+0000 7f10e67fc640 1 -- 172.21.15.106:0/2497834621 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e0007740 con 0x7f10e815d690 2024-04-25T10:34:56.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.516+0000 7f10e67fc640 1 -- 172.21.15.106:0/2497834621 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10e00078e0 con 0x7f10e815d690 2024-04-25T10:34:56.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.516+0000 7f10e67fc640 1 -- 172.21.15.106:0/2497834621 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e0007cb0 con 0x7f10e815d690 2024-04-25T10:34:56.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.517+0000 7f10edba8640 1 -- 172.21.15.106:0/2497834621 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10e815d690 msgr2=0x7f10e80cce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:56.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.517+0000 7f10edba8640 1 --2- 172.21.15.106:0/2497834621 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10e815d690 0x7f10e80cce20 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f10e0009d90 tx=0x7f10e002a660 comp rx=0 tx=0).stop 2024-04-25T10:34:56.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.517+0000 7f10edba8640 1 -- 172.21.15.106:0/2497834621 shutdown_connections 2024-04-25T10:34:56.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.517+0000 7f10edba8640 1 --2- 172.21.15.106:0/2497834621 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10e815d690 0x7f10e80cce20 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:56.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.517+0000 7f10edba8640 1 --2- 172.21.15.106:0/2497834621 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f10e80c61a0 0x7f10e8156f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:56.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.517+0000 7f10edba8640 1 -- 172.21.15.106:0/2497834621 >> 172.21.15.106:0/2497834621 conn(0x7f10e80727d0 msgr2=0x7f10e8072c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:56.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.517+0000 7f10edba8640 1 -- 172.21.15.106:0/2497834621 shutdown_connections 2024-04-25T10:34:56.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.517+0000 7f10edba8640 1 -- 172.21.15.106:0/2497834621 wait complete. 2024-04-25T10:34:56.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.518+0000 7f10edba8640 1 Processor -- start 2024-04-25T10:34:56.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.519+0000 7f10edba8640 1 -- start start 2024-04-25T10:34:56.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.520+0000 7f10edba8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10e806dc00 0x7f10e815b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:56.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.520+0000 7f10edba8640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f10e80c61a0 0x7f10e81f5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:56.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.520+0000 7f10edba8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10e815d690 0x7f10e81e9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:56.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.520+0000 7f10edba8640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f10e815daa0 con 0x7f10e815d690 2024-04-25T10:34:56.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.520+0000 7f10edba8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f10e815ce90 con 0x7f10e806dc00 2024-04-25T10:34:56.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.520+0000 7f10edba8640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f10e806d6d0 con 0x7f10e80c61a0 2024-04-25T10:34:56.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.520+0000 7f10e7fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10e815d690 0x7f10e81e9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:56.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.520+0000 7f10e7fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10e815d690 0x7f10e81e9ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53512/0 (socket says 172.21.15.106:53512) 2024-04-25T10:34:56.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.520+0000 7f10e7fff640 1 -- 172.21.15.106:0/2318000436 learned_addr learned my addr 172.21.15.106:0/2318000436 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:34:56.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.520+0000 7f10e77fe640 1 --2- 172.21.15.106:0/2318000436 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10e806dc00 0x7f10e815b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:56.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.520+0000 7f10e6ffd640 1 --2- 172.21.15.106:0/2318000436 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f10e80c61a0 0x7f10e81f5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:56.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.520+0000 7f10e7fff640 1 -- 172.21.15.106:0/2318000436 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f10e80c61a0 msgr2=0x7f10e81f5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:56.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.520+0000 7f10e7fff640 1 --2- 172.21.15.106:0/2318000436 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f10e80c61a0 0x7f10e81f5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:56.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.520+0000 7f10e7fff640 1 -- 172.21.15.106:0/2318000436 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10e806dc00 msgr2=0x7f10e815b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:56.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.520+0000 7f10e7fff640 1 --2- 172.21.15.106:0/2318000436 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10e806dc00 0x7f10e815b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:56.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.520+0000 7f10e7fff640 1 -- 172.21.15.106:0/2318000436 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10e81f5870 con 0x7f10e815d690 2024-04-25T10:34:56.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.520+0000 7f10e7fff640 1 --2- 172.21.15.106:0/2318000436 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10e815d690 0x7f10e81e9ae0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f10e0009d90 tx=0x7f10e0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:56.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.521+0000 7f10cffff640 1 -- 172.21.15.106:0/2318000436 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e003f070 con 0x7f10e815d690 2024-04-25T10:34:56.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.521+0000 7f10cffff640 1 -- 172.21.15.106:0/2318000436 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10e0003b90 con 0x7f10e815d690 2024-04-25T10:34:56.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.521+0000 7f10edba8640 1 -- 172.21.15.106:0/2318000436 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10e81f45b0 con 0x7f10e815d690 2024-04-25T10:34:56.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.521+0000 7f10cffff640 1 -- 172.21.15.106:0/2318000436 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e0031940 con 0x7f10e815d690 2024-04-25T10:34:56.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.521+0000 7f10edba8640 1 -- 172.21.15.106:0/2318000436 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10e81f4b30 con 0x7f10e815d690 2024-04-25T10:34:56.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.522+0000 7f10edba8640 1 -- 172.21.15.106:0/2318000436 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10e806a3d0 con 0x7f10e815d690 2024-04-25T10:34:56.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.522+0000 7f10cffff640 1 -- 172.21.15.106:0/2318000436 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f10e0031ae0 con 0x7f10e815d690 2024-04-25T10:34:56.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.523+0000 7f10cffff640 1 --2- 172.21.15.106:0/2318000436 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f10d40744e0 0x7f10d40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:34:56.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.523+0000 7f10cffff640 1 -- 172.21.15.106:0/2318000436 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f10e00b6380 con 0x7f10e815d690 2024-04-25T10:34:56.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.523+0000 7f10e77fe640 1 --2- 172.21.15.106:0/2318000436 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f10d40744e0 0x7f10d40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:34:56.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.523+0000 7f10e77fe640 1 --2- 172.21.15.106:0/2318000436 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f10d40744e0 0x7f10d40769f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f10d8006e80 tx=0x7f10d8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:34:56.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.527+0000 7f10cffff640 1 -- 172.21.15.106:0/2318000436 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f10e00804d0 con 0x7f10e815d690 2024-04-25T10:34:56.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.700+0000 7f10edba8640 1 -- 172.21.15.106:0/2318000436 --> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f10e8154600 con 0x7f10d40744e0 2024-04-25T10:34:56.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.705+0000 7f10cffff640 1 -- 172.21.15.106:0/2318000436 <== mgr.4107 v2:172.21.15.106:6832/1744210167 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251328 (secure 0 0 0) 0x7f10e8154600 con 0x7f10d40744e0 2024-04-25T10:34:56.710 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:34:56.712 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-25T10:34:56.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.708+0000 7f10edba8640 1 -- 172.21.15.106:0/2318000436 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f10d40744e0 msgr2=0x7f10d40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:56.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.708+0000 7f10edba8640 1 --2- 172.21.15.106:0/2318000436 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f10d40744e0 0x7f10d40769f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f10d8006e80 tx=0x7f10d8008040 comp rx=0 tx=0).stop 2024-04-25T10:34:56.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.708+0000 7f10edba8640 1 -- 172.21.15.106:0/2318000436 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10e815d690 msgr2=0x7f10e81e9ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:34:56.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.708+0000 7f10edba8640 1 --2- 172.21.15.106:0/2318000436 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10e815d690 0x7f10e81e9ae0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f10e0009d90 tx=0x7f10e0004a40 comp rx=0 tx=0).stop 2024-04-25T10:34:56.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.708+0000 7f10edba8640 1 -- 172.21.15.106:0/2318000436 shutdown_connections 2024-04-25T10:34:56.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.708+0000 7f10edba8640 1 --2- 172.21.15.106:0/2318000436 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f10d40744e0 0x7f10d40769f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:56.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.708+0000 7f10edba8640 1 --2- 172.21.15.106:0/2318000436 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10e815d690 0x7f10e81e9ae0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:56.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.708+0000 7f10edba8640 1 --2- 172.21.15.106:0/2318000436 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f10e80c61a0 0x7f10e81f5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:56.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.708+0000 7f10edba8640 1 --2- 172.21.15.106:0/2318000436 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10e806dc00 0x7f10e815b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:34:56.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.708+0000 7f10edba8640 1 -- 172.21.15.106:0/2318000436 >> 172.21.15.106:0/2318000436 conn(0x7f10e80727d0 msgr2=0x7f10e8155ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:34:56.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.709+0000 7f10edba8640 1 -- 172.21.15.106:0/2318000436 shutdown_connections 2024-04-25T10:34:56.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:34:56.709+0000 7f10edba8640 1 -- 172.21.15.106:0/2318000436 wait complete. 2024-04-25T10:34:56.722 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":153,"stamp":"2024-04-25T10:34:56.162353+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":800831,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":71,"num_read_kb":60,"num_write":9662,"num_write_kb":25167776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":10000,"num_omap_keys":20,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10117,"ondisk_log_size":10117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":307096,"kb_used_data":3608,"kb_used_omap":12,"kb_used_meta":303411,"kb_avail":754667624,"statfs":{"total":773094113280,"available":772779646976,"internally_reserved":0,"allocated":3694592,"data_stored":2266718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":310693424},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":672,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":10000,"num_omap_keys":20,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003051"},"pg_stats":[{"pgid":"3.3f","version":"22'146","reported_seq":164,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.965117+0000","last_change":"2024-04-25T10:30:32.440983+0000","last_active":"2024-04-25T10:34:43.965117+0000","last_peered":"2024-04-25T10:34:43.965117+0000","last_clean":"2024-04-25T10:34:43.965117+0000","last_became_active":"2024-04-25T10:30:32.440552+0000","last_became_peered":"2024-04-25T10:30:32.440552+0000","last_unstale":"2024-04-25T10:34:43.965117+0000","last_undegraded":"2024-04-25T10:34:43.965117+0000","last_fullsized":"2024-04-25T10:34:43.965117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:47.963061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"22'130","reported_seq":149,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:45.149465+0000","last_change":"2024-04-25T10:30:32.425695+0000","last_active":"2024-04-25T10:34:45.149465+0000","last_peered":"2024-04-25T10:34:45.149465+0000","last_clean":"2024-04-25T10:34:45.149465+0000","last_became_active":"2024-04-25T10:30:32.425368+0000","last_became_peered":"2024-04-25T10:30:32.425368+0000","last_unstale":"2024-04-25T10:34:45.149465+0000","last_undegraded":"2024-04-25T10:34:45.149465+0000","last_fullsized":"2024-04-25T10:34:45.149465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"22'151","reported_seq":170,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:45.150757+0000","last_change":"2024-04-25T10:30:32.430144+0000","last_active":"2024-04-25T10:34:45.150757+0000","last_peered":"2024-04-25T10:34:45.150757+0000","last_clean":"2024-04-25T10:34:45.150757+0000","last_became_active":"2024-04-25T10:30:32.429754+0000","last_became_peered":"2024-04-25T10:30:32.429754+0000","last_unstale":"2024-04-25T10:34:45.150757+0000","last_undegraded":"2024-04-25T10:34:45.150757+0000","last_fullsized":"2024-04-25T10:34:45.150757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"22'173","reported_seq":191,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.964702+0000","last_change":"2024-04-25T10:30:32.442478+0000","last_active":"2024-04-25T10:34:43.964702+0000","last_peered":"2024-04-25T10:34:43.964702+0000","last_clean":"2024-04-25T10:34:43.964702+0000","last_became_active":"2024-04-25T10:30:32.442182+0000","last_became_peered":"2024-04-25T10:30:32.442182+0000","last_unstale":"2024-04-25T10:34:43.964702+0000","last_undegraded":"2024-04-25T10:34:43.964702+0000","last_fullsized":"2024-04-25T10:34:43.964702+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":173,"log_dups_size":0,"ondisk_log_size":173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:22.185978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":450560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"22'148","reported_seq":165,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.934476+0000","last_change":"2024-04-25T10:30:32.438231+0000","last_active":"2024-04-25T10:34:43.934476+0000","last_peered":"2024-04-25T10:34:43.934476+0000","last_clean":"2024-04-25T10:34:43.934476+0000","last_became_active":"2024-04-25T10:30:32.437656+0000","last_became_peered":"2024-04-25T10:30:32.437656+0000","last_unstale":"2024-04-25T10:34:43.934476+0000","last_undegraded":"2024-04-25T10:34:43.934476+0000","last_fullsized":"2024-04-25T10:34:43.934476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:56.915999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"22'135","reported_seq":147,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:44.060334+0000","last_change":"2024-04-25T10:31:13.676329+0000","last_active":"2024-04-25T10:34:44.060334+0000","last_peered":"2024-04-25T10:34:44.060334+0000","last_clean":"2024-04-25T10:34:44.060334+0000","last_became_active":"2024-04-25T10:31:13.676013+0000","last_became_peered":"2024-04-25T10:31:13.676013+0000","last_unstale":"2024-04-25T10:34:44.060334+0000","last_undegraded":"2024-04-25T10:34:44.060334+0000","last_fullsized":"2024-04-25T10:34:44.060334+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"22'164","reported_seq":181,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.941596+0000","last_change":"2024-04-25T10:30:32.429453+0000","last_active":"2024-04-25T10:34:43.941596+0000","last_peered":"2024-04-25T10:34:43.941596+0000","last_clean":"2024-04-25T10:34:43.941596+0000","last_became_active":"2024-04-25T10:30:32.429131+0000","last_became_peered":"2024-04-25T10:30:32.429131+0000","last_unstale":"2024-04-25T10:34:43.941596+0000","last_undegraded":"2024-04-25T10:34:43.941596+0000","last_fullsized":"2024-04-25T10:34:43.941596+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:26.785922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159,"num_write_kb":434176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"22'186","reported_seq":204,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.963620+0000","last_change":"2024-04-25T10:30:32.438860+0000","last_active":"2024-04-25T10:34:43.963620+0000","last_peered":"2024-04-25T10:34:43.963620+0000","last_clean":"2024-04-25T10:34:43.963620+0000","last_became_active":"2024-04-25T10:30:32.438491+0000","last_became_peered":"2024-04-25T10:30:32.438491+0000","last_unstale":"2024-04-25T10:34:43.963620+0000","last_undegraded":"2024-04-25T10:34:43.963620+0000","last_fullsized":"2024-04-25T10:34:43.963620+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":177,"num_write_kb":483328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"22'152","reported_seq":169,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.967681+0000","last_change":"2024-04-25T10:30:32.440331+0000","last_active":"2024-04-25T10:34:43.967681+0000","last_peered":"2024-04-25T10:34:43.967681+0000","last_clean":"2024-04-25T10:34:43.967681+0000","last_became_active":"2024-04-25T10:30:32.440043+0000","last_became_peered":"2024-04-25T10:30:32.440043+0000","last_unstale":"2024-04-25T10:34:43.967681+0000","last_undegraded":"2024-04-25T10:34:43.967681+0000","last_fullsized":"2024-04-25T10:34:43.967681+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:17.684685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"22'156","reported_seq":173,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.968251+0000","last_change":"2024-04-25T10:30:32.435388+0000","last_active":"2024-04-25T10:34:43.968251+0000","last_peered":"2024-04-25T10:34:43.968251+0000","last_clean":"2024-04-25T10:34:43.968251+0000","last_became_active":"2024-04-25T10:30:32.434983+0000","last_became_peered":"2024-04-25T10:30:32.434983+0000","last_unstale":"2024-04-25T10:34:43.968251+0000","last_undegraded":"2024-04-25T10:34:43.968251+0000","last_fullsized":"2024-04-25T10:34:43.968251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:39.256604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"22'189","reported_seq":211,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.953182+0000","last_change":"2024-04-25T10:31:13.640959+0000","last_active":"2024-04-25T10:34:43.953182+0000","last_peered":"2024-04-25T10:34:43.953182+0000","last_clean":"2024-04-25T10:34:43.953182+0000","last_became_active":"2024-04-25T10:31:13.640582+0000","last_became_peered":"2024-04-25T10:31:13.640582+0000","last_unstale":"2024-04-25T10:34:43.953182+0000","last_undegraded":"2024-04-25T10:34:43.953182+0000","last_fullsized":"2024-04-25T10:34:43.953182+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:30.907171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"22'142","reported_seq":160,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.955954+0000","last_change":"2024-04-25T10:30:32.436403+0000","last_active":"2024-04-25T10:34:43.955954+0000","last_peered":"2024-04-25T10:34:43.955954+0000","last_clean":"2024-04-25T10:34:43.955954+0000","last_became_active":"2024-04-25T10:30:32.435908+0000","last_became_peered":"2024-04-25T10:30:32.435908+0000","last_unstale":"2024-04-25T10:34:43.955954+0000","last_undegraded":"2024-04-25T10:34:43.955954+0000","last_fullsized":"2024-04-25T10:34:43.955954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"22'151","reported_seq":169,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.951441+0000","last_change":"2024-04-25T10:30:32.432110+0000","last_active":"2024-04-25T10:34:43.951441+0000","last_peered":"2024-04-25T10:34:43.951441+0000","last_clean":"2024-04-25T10:34:43.951441+0000","last_became_active":"2024-04-25T10:30:32.431792+0000","last_became_peered":"2024-04-25T10:30:32.431792+0000","last_unstale":"2024-04-25T10:34:43.951441+0000","last_undegraded":"2024-04-25T10:34:43.951441+0000","last_fullsized":"2024-04-25T10:34:43.951441+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"22'149","reported_seq":167,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.940387+0000","last_change":"2024-04-25T10:30:32.433676+0000","last_active":"2024-04-25T10:34:43.940387+0000","last_peered":"2024-04-25T10:34:43.940387+0000","last_clean":"2024-04-25T10:34:43.940387+0000","last_became_active":"2024-04-25T10:30:32.433329+0000","last_became_peered":"2024-04-25T10:30:32.433329+0000","last_unstale":"2024-04-25T10:34:43.940387+0000","last_undegraded":"2024-04-25T10:34:43.940387+0000","last_fullsized":"2024-04-25T10:34:43.940387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"22'151","reported_seq":169,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.962858+0000","last_change":"2024-04-25T10:30:32.423582+0000","last_active":"2024-04-25T10:34:43.962858+0000","last_peered":"2024-04-25T10:34:43.962858+0000","last_clean":"2024-04-25T10:34:43.962858+0000","last_became_active":"2024-04-25T10:30:32.422991+0000","last_became_peered":"2024-04-25T10:30:32.422991+0000","last_unstale":"2024-04-25T10:34:43.962858+0000","last_undegraded":"2024-04-25T10:34:43.962858+0000","last_fullsized":"2024-04-25T10:34:43.962858+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"22'147","reported_seq":164,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.965990+0000","last_change":"2024-04-25T10:30:32.440235+0000","last_active":"2024-04-25T10:34:43.965990+0000","last_peered":"2024-04-25T10:34:43.965990+0000","last_clean":"2024-04-25T10:34:43.965990+0000","last_became_active":"2024-04-25T10:30:32.439191+0000","last_became_peered":"2024-04-25T10:30:32.439191+0000","last_unstale":"2024-04-25T10:34:43.965990+0000","last_undegraded":"2024-04-25T10:34:43.965990+0000","last_fullsized":"2024-04-25T10:34:43.965990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":147,"log_dups_size":0,"ondisk_log_size":147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:51.423086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"22'146","reported_seq":163,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.959761+0000","last_change":"2024-04-25T10:30:32.442418+0000","last_active":"2024-04-25T10:34:43.959761+0000","last_peered":"2024-04-25T10:34:43.959761+0000","last_clean":"2024-04-25T10:34:43.959761+0000","last_became_active":"2024-04-25T10:30:32.442125+0000","last_became_peered":"2024-04-25T10:30:32.442125+0000","last_unstale":"2024-04-25T10:34:43.959761+0000","last_undegraded":"2024-04-25T10:34:43.959761+0000","last_fullsized":"2024-04-25T10:34:43.959761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:14.213809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"22'157","reported_seq":175,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.925244+0000","last_change":"2024-04-25T10:30:32.434758+0000","last_active":"2024-04-25T10:34:43.925244+0000","last_peered":"2024-04-25T10:34:43.925244+0000","last_clean":"2024-04-25T10:34:43.925244+0000","last_became_active":"2024-04-25T10:30:32.434492+0000","last_became_peered":"2024-04-25T10:30:32.434492+0000","last_unstale":"2024-04-25T10:34:43.925244+0000","last_undegraded":"2024-04-25T10:34:43.925244+0000","last_fullsized":"2024-04-25T10:34:43.925244+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"22'172","reported_seq":189,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.948403+0000","last_change":"2024-04-25T10:30:32.440399+0000","last_active":"2024-04-25T10:34:43.948403+0000","last_peered":"2024-04-25T10:34:43.948403+0000","last_clean":"2024-04-25T10:34:43.948403+0000","last_became_active":"2024-04-25T10:30:32.438227+0000","last_became_peered":"2024-04-25T10:30:32.438227+0000","last_unstale":"2024-04-25T10:34:43.948403+0000","last_undegraded":"2024-04-25T10:34:43.948403+0000","last_fullsized":"2024-04-25T10:34:43.948403+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:49.571671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":450560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"22'161","reported_seq":178,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.963147+0000","last_change":"2024-04-25T10:30:32.431609+0000","last_active":"2024-04-25T10:34:43.963147+0000","last_peered":"2024-04-25T10:34:43.963147+0000","last_clean":"2024-04-25T10:34:43.963147+0000","last_became_active":"2024-04-25T10:30:32.431314+0000","last_became_peered":"2024-04-25T10:30:32.431314+0000","last_unstale":"2024-04-25T10:34:43.963147+0000","last_undegraded":"2024-04-25T10:34:43.963147+0000","last_fullsized":"2024-04-25T10:34:43.963147+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:33.421989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":153,"num_write_kb":417792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"22'137","reported_seq":149,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.956161+0000","last_change":"2024-04-25T10:31:13.677426+0000","last_active":"2024-04-25T10:34:43.956161+0000","last_peered":"2024-04-25T10:34:43.956161+0000","last_clean":"2024-04-25T10:34:43.956161+0000","last_became_active":"2024-04-25T10:31:13.677164+0000","last_became_peered":"2024-04-25T10:31:13.677164+0000","last_unstale":"2024-04-25T10:34:43.956161+0000","last_undegraded":"2024-04-25T10:34:43.956161+0000","last_fullsized":"2024-04-25T10:34:43.956161+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2a","version":"22'166","reported_seq":183,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.968866+0000","last_change":"2024-04-25T10:30:32.444618+0000","last_active":"2024-04-25T10:34:43.968866+0000","last_peered":"2024-04-25T10:34:43.968866+0000","last_clean":"2024-04-25T10:34:43.968866+0000","last_became_active":"2024-04-25T10:30:32.444273+0000","last_became_peered":"2024-04-25T10:30:32.444273+0000","last_unstale":"2024-04-25T10:34:43.968866+0000","last_undegraded":"2024-04-25T10:34:43.968866+0000","last_fullsized":"2024-04-25T10:34:43.968866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:09.295251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"22'134","reported_seq":152,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.960556+0000","last_change":"2024-04-25T10:30:32.426578+0000","last_active":"2024-04-25T10:34:43.960556+0000","last_peered":"2024-04-25T10:34:43.960556+0000","last_clean":"2024-04-25T10:34:43.960556+0000","last_became_active":"2024-04-25T10:30:32.426223+0000","last_became_peered":"2024-04-25T10:30:32.426223+0000","last_unstale":"2024-04-25T10:34:43.960556+0000","last_undegraded":"2024-04-25T10:34:43.960556+0000","last_fullsized":"2024-04-25T10:34:43.960556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"22'180","reported_seq":198,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.966080+0000","last_change":"2024-04-25T10:30:32.439767+0000","last_active":"2024-04-25T10:34:43.966080+0000","last_peered":"2024-04-25T10:34:43.966080+0000","last_clean":"2024-04-25T10:34:43.966080+0000","last_became_active":"2024-04-25T10:30:32.438801+0000","last_became_peered":"2024-04-25T10:30:32.438801+0000","last_unstale":"2024-04-25T10:34:43.966080+0000","last_undegraded":"2024-04-25T10:34:43.966080+0000","last_fullsized":"2024-04-25T10:34:43.966080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:58.231512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":174,"num_write_kb":475136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"22'134","reported_seq":153,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:54.541296+0000","last_change":"2024-04-25T10:30:32.427424+0000","last_active":"2024-04-25T10:34:54.541296+0000","last_peered":"2024-04-25T10:34:54.541296+0000","last_clean":"2024-04-25T10:34:54.541296+0000","last_became_active":"2024-04-25T10:30:32.427081+0000","last_became_peered":"2024-04-25T10:30:32.427081+0000","last_unstale":"2024-04-25T10:34:54.541296+0000","last_undegraded":"2024-04-25T10:34:54.541296+0000","last_fullsized":"2024-04-25T10:34:54.541296+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"22'198","reported_seq":216,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.969019+0000","last_change":"2024-04-25T10:30:32.432472+0000","last_active":"2024-04-25T10:34:43.969019+0000","last_peered":"2024-04-25T10:34:43.969019+0000","last_clean":"2024-04-25T10:34:43.969019+0000","last_became_active":"2024-04-25T10:30:32.432121+0000","last_became_peered":"2024-04-25T10:30:32.432121+0000","last_unstale":"2024-04-25T10:34:43.969019+0000","last_undegraded":"2024-04-25T10:34:43.969019+0000","last_fullsized":"2024-04-25T10:34:43.969019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":198,"log_dups_size":0,"ondisk_log_size":198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:34.191339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"22'144","reported_seq":163,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:53.610359+0000","last_change":"2024-04-25T10:30:32.425108+0000","last_active":"2024-04-25T10:34:53.610359+0000","last_peered":"2024-04-25T10:34:53.610359+0000","last_clean":"2024-04-25T10:34:53.610359+0000","last_became_active":"2024-04-25T10:30:32.424794+0000","last_became_peered":"2024-04-25T10:30:32.424794+0000","last_unstale":"2024-04-25T10:34:53.610359+0000","last_undegraded":"2024-04-25T10:34:53.610359+0000","last_fullsized":"2024-04-25T10:34:53.610359+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"22'133","reported_seq":152,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:54.541295+0000","last_change":"2024-04-25T10:30:32.426672+0000","last_active":"2024-04-25T10:34:54.541295+0000","last_peered":"2024-04-25T10:34:54.541295+0000","last_clean":"2024-04-25T10:34:54.541295+0000","last_became_active":"2024-04-25T10:30:32.426348+0000","last_became_peered":"2024-04-25T10:30:32.426348+0000","last_unstale":"2024-04-25T10:34:54.541295+0000","last_undegraded":"2024-04-25T10:34:54.541295+0000","last_fullsized":"2024-04-25T10:34:54.541295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"22'118","reported_seq":137,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:45.150736+0000","last_change":"2024-04-25T10:30:32.427989+0000","last_active":"2024-04-25T10:34:45.150736+0000","last_peered":"2024-04-25T10:34:45.150736+0000","last_clean":"2024-04-25T10:34:45.150736+0000","last_became_active":"2024-04-25T10:30:32.427676+0000","last_became_peered":"2024-04-25T10:30:32.427676+0000","last_unstale":"2024-04-25T10:34:45.150736+0000","last_undegraded":"2024-04-25T10:34:45.150736+0000","last_fullsized":"2024-04-25T10:34:45.150736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":111,"num_write_kb":303104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"22'131","reported_seq":149,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.960144+0000","last_change":"2024-04-25T10:30:32.441112+0000","last_active":"2024-04-25T10:34:43.960144+0000","last_peered":"2024-04-25T10:34:43.960144+0000","last_clean":"2024-04-25T10:34:43.960144+0000","last_became_active":"2024-04-25T10:30:32.440815+0000","last_became_peered":"2024-04-25T10:30:32.440815+0000","last_unstale":"2024-04-25T10:34:43.960144+0000","last_undegraded":"2024-04-25T10:34:43.960144+0000","last_fullsized":"2024-04-25T10:34:43.960144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:02.956759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"22'135","reported_seq":154,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:45.149406+0000","last_change":"2024-04-25T10:30:32.430902+0000","last_active":"2024-04-25T10:34:45.149406+0000","last_peered":"2024-04-25T10:34:45.149406+0000","last_clean":"2024-04-25T10:34:45.149406+0000","last_became_active":"2024-04-25T10:30:32.430615+0000","last_became_peered":"2024-04-25T10:30:32.430615+0000","last_unstale":"2024-04-25T10:34:45.149406+0000","last_undegraded":"2024-04-25T10:34:45.149406+0000","last_fullsized":"2024-04-25T10:34:45.149406+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"22'150","reported_seq":168,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.959087+0000","last_change":"2024-04-25T10:30:32.435379+0000","last_active":"2024-04-25T10:34:43.959087+0000","last_peered":"2024-04-25T10:34:43.959087+0000","last_clean":"2024-04-25T10:34:43.959087+0000","last_became_active":"2024-04-25T10:30:32.435055+0000","last_became_peered":"2024-04-25T10:30:32.435055+0000","last_unstale":"2024-04-25T10:34:43.959087+0000","last_undegraded":"2024-04-25T10:34:43.959087+0000","last_fullsized":"2024-04-25T10:34:43.959087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:13.623762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"22'130","reported_seq":148,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.959897+0000","last_change":"2024-04-25T10:30:32.422102+0000","last_active":"2024-04-25T10:34:43.959897+0000","last_peered":"2024-04-25T10:34:43.959897+0000","last_clean":"2024-04-25T10:34:43.959897+0000","last_became_active":"2024-04-25T10:30:32.421766+0000","last_became_peered":"2024-04-25T10:30:32.421766+0000","last_unstale":"2024-04-25T10:34:43.959897+0000","last_undegraded":"2024-04-25T10:34:43.959897+0000","last_fullsized":"2024-04-25T10:34:43.959897+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'7","reported_seq":27,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.902172+0000","last_change":"2024-04-25T10:30:31.418065+0000","last_active":"2024-04-25T10:34:43.902172+0000","last_peered":"2024-04-25T10:34:43.902172+0000","last_clean":"2024-04-25T10:34:43.902172+0000","last_became_active":"2024-04-25T10:30:31.417451+0000","last_became_peered":"2024-04-25T10:30:31.417451+0000","last_unstale":"2024-04-25T10:34:43.902172+0000","last_undegraded":"2024-04-25T10:34:43.902172+0000","last_fullsized":"2024-04-25T10:34:43.902172+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1204,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"22'137","reported_seq":155,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.941627+0000","last_change":"2024-04-25T10:30:32.426913+0000","last_active":"2024-04-25T10:34:43.941627+0000","last_peered":"2024-04-25T10:34:43.941627+0000","last_clean":"2024-04-25T10:34:43.941627+0000","last_became_active":"2024-04-25T10:30:32.426616+0000","last_became_peered":"2024-04-25T10:30:32.426616+0000","last_unstale":"2024-04-25T10:34:43.941627+0000","last_undegraded":"2024-04-25T10:34:43.941627+0000","last_fullsized":"2024-04-25T10:34:43.941627+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'7","reported_seq":26,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:31:13.638951+0000","last_change":"2024-04-25T10:30:31.416432+0000","last_active":"2024-04-25T10:31:13.638951+0000","last_peered":"2024-04-25T10:31:13.638951+0000","last_clean":"2024-04-25T10:31:13.638951+0000","last_became_active":"2024-04-25T10:30:31.415112+0000","last_became_peered":"2024-04-25T10:30:31.415112+0000","last_unstale":"2024-04-25T10:31:13.638951+0000","last_undegraded":"2024-04-25T10:31:13.638951+0000","last_fullsized":"2024-04-25T10:31:13.638951+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:22.674365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"20'3","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:33:00.134354+0000","last_change":"2024-04-25T10:30:31.418059+0000","last_active":"2024-04-25T10:33:00.134354+0000","last_peered":"2024-04-25T10:33:00.134354+0000","last_clean":"2024-04-25T10:33:00.134354+0000","last_became_active":"2024-04-25T10:30:31.417716+0000","last_became_peered":"2024-04-25T10:30:31.417716+0000","last_unstale":"2024-04-25T10:33:00.134354+0000","last_undegraded":"2024-04-25T10:33:00.134354+0000","last_fullsized":"2024-04-25T10:33:00.134354+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"22'170","reported_seq":188,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.938583+0000","last_change":"2024-04-25T10:30:32.428078+0000","last_active":"2024-04-25T10:34:43.938583+0000","last_peered":"2024-04-25T10:34:43.938583+0000","last_clean":"2024-04-25T10:34:43.938583+0000","last_became_active":"2024-04-25T10:30:32.427700+0000","last_became_peered":"2024-04-25T10:30:32.427700+0000","last_unstale":"2024-04-25T10:34:43.938583+0000","last_undegraded":"2024-04-25T10:34:43.938583+0000","last_fullsized":"2024-04-25T10:34:43.938583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":162,"num_write_kb":442368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"20'3","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:31:13.638118+0000","last_change":"2024-04-25T10:30:31.415030+0000","last_active":"2024-04-25T10:31:13.638118+0000","last_peered":"2024-04-25T10:31:13.638118+0000","last_clean":"2024-04-25T10:31:13.638118+0000","last_became_active":"2024-04-25T10:30:31.414542+0000","last_became_peered":"2024-04-25T10:30:31.414542+0000","last_unstale":"2024-04-25T10:31:13.638118+0000","last_undegraded":"2024-04-25T10:31:13.638118+0000","last_fullsized":"2024-04-25T10:31:13.638118+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:20.082420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"22'157","reported_seq":175,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.939966+0000","last_change":"2024-04-25T10:30:32.425207+0000","last_active":"2024-04-25T10:34:43.939966+0000","last_peered":"2024-04-25T10:34:43.939966+0000","last_clean":"2024-04-25T10:34:43.939966+0000","last_became_active":"2024-04-25T10:30:32.424849+0000","last_became_peered":"2024-04-25T10:30:32.424849+0000","last_unstale":"2024-04-25T10:34:43.939966+0000","last_undegraded":"2024-04-25T10:34:43.939966+0000","last_fullsized":"2024-04-25T10:34:43.939966+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"22'8","reported_seq":27,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.902339+0000","last_change":"2024-04-25T10:30:31.407700+0000","last_active":"2024-04-25T10:34:43.902339+0000","last_peered":"2024-04-25T10:34:43.902339+0000","last_clean":"2024-04-25T10:34:43.902339+0000","last_became_active":"2024-04-25T10:30:31.407376+0000","last_became_peered":"2024-04-25T10:30:31.407376+0000","last_unstale":"2024-04-25T10:34:43.902339+0000","last_undegraded":"2024-04-25T10:34:43.902339+0000","last_fullsized":"2024-04-25T10:34:43.902339+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:50.112742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"22'229","reported_seq":246,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.960310+0000","last_change":"2024-04-25T10:30:32.444599+0000","last_active":"2024-04-25T10:34:43.960310+0000","last_peered":"2024-04-25T10:34:43.960310+0000","last_clean":"2024-04-25T10:34:43.960310+0000","last_became_active":"2024-04-25T10:30:32.444250+0000","last_became_peered":"2024-04-25T10:30:32.444250+0000","last_unstale":"2024-04-25T10:34:43.960310+0000","last_undegraded":"2024-04-25T10:34:43.960310+0000","last_fullsized":"2024-04-25T10:34:43.960310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":229,"log_dups_size":0,"ondisk_log_size":229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:55.711525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":219,"num_write_kb":598016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"20'4","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:31:13.638996+0000","last_change":"2024-04-25T10:30:31.405051+0000","last_active":"2024-04-25T10:31:13.638996+0000","last_peered":"2024-04-25T10:31:13.638996+0000","last_clean":"2024-04-25T10:31:13.638996+0000","last_became_active":"2024-04-25T10:30:31.404669+0000","last_became_peered":"2024-04-25T10:30:31.404669+0000","last_unstale":"2024-04-25T10:31:13.638996+0000","last_undegraded":"2024-04-25T10:31:13.638996+0000","last_fullsized":"2024-04-25T10:31:13.638996+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:45.260847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"22'141","reported_seq":159,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.956745+0000","last_change":"2024-04-25T10:30:32.435804+0000","last_active":"2024-04-25T10:34:43.956745+0000","last_peered":"2024-04-25T10:34:43.956745+0000","last_clean":"2024-04-25T10:34:43.956745+0000","last_became_active":"2024-04-25T10:30:32.435471+0000","last_became_peered":"2024-04-25T10:30:32.435471+0000","last_unstale":"2024-04-25T10:34:43.956745+0000","last_undegraded":"2024-04-25T10:34:43.956745+0000","last_fullsized":"2024-04-25T10:34:43.956745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"22'14","reported_seq":42,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:53.648907+0000","last_change":"2024-04-25T10:34:53.648938+0000","last_active":"2024-04-25T10:34:53.648907+0000","last_peered":"2024-04-25T10:34:53.648907+0000","last_clean":"2024-04-25T10:34:53.648907+0000","last_became_active":"2024-04-25T10:30:31.403167+0000","last_became_peered":"2024-04-25T10:30:31.403167+0000","last_unstale":"2024-04-25T10:34:53.648907+0000","last_undegraded":"2024-04-25T10:34:53.648907+0000","last_fullsized":"2024-04-25T10:34:53.648907+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'14","last_scrub_stamp":"2024-04-25T10:34:53.648827+0000","last_deep_scrub":"22'14","last_deep_scrub_stamp":"2024-04-25T10:34:53.648827+0000","last_clean_scrub_stamp":"2024-04-25T10:34:53.648827+0000","objects_scrubbed":4,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:55.835283+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"22'149","reported_seq":167,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.968766+0000","last_change":"2024-04-25T10:30:32.424101+0000","last_active":"2024-04-25T10:34:43.968766+0000","last_peered":"2024-04-25T10:34:43.968766+0000","last_clean":"2024-04-25T10:34:43.968766+0000","last_became_active":"2024-04-25T10:30:32.423780+0000","last_became_peered":"2024-04-25T10:30:32.423780+0000","last_unstale":"2024-04-25T10:34:43.968766+0000","last_undegraded":"2024-04-25T10:34:43.968766+0000","last_fullsized":"2024-04-25T10:34:43.968766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"22'95","reported_seq":114,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.905445+0000","last_change":"2024-04-25T10:30:31.418201+0000","last_active":"2024-04-25T10:34:43.905445+0000","last_peered":"2024-04-25T10:34:43.905445+0000","last_clean":"2024-04-25T10:34:43.905445+0000","last_became_active":"2024-04-25T10:30:31.417824+0000","last_became_peered":"2024-04-25T10:30:31.417824+0000","last_unstale":"2024-04-25T10:34:43.905445+0000","last_undegraded":"2024-04-25T10:34:43.905445+0000","last_fullsized":"2024-04-25T10:34:43.905445+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:38.584225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":201790,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":96,"num_write_kb":233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"22'150","reported_seq":167,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.951616+0000","last_change":"2024-04-25T10:30:32.431139+0000","last_active":"2024-04-25T10:34:43.951616+0000","last_peered":"2024-04-25T10:34:43.951616+0000","last_clean":"2024-04-25T10:34:43.951616+0000","last_became_active":"2024-04-25T10:30:32.429622+0000","last_became_peered":"2024-04-25T10:30:32.429622+0000","last_unstale":"2024-04-25T10:34:43.951616+0000","last_undegraded":"2024-04-25T10:34:43.951616+0000","last_fullsized":"2024-04-25T10:34:43.951616+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:39.993647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'15","reported_seq":43,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:53.171724+0000","last_change":"2024-04-25T10:34:53.171761+0000","last_active":"2024-04-25T10:34:53.171724+0000","last_peered":"2024-04-25T10:34:53.171724+0000","last_clean":"2024-04-25T10:34:53.171724+0000","last_became_active":"2024-04-25T10:30:31.414422+0000","last_became_peered":"2024-04-25T10:30:31.414422+0000","last_unstale":"2024-04-25T10:34:53.171724+0000","last_undegraded":"2024-04-25T10:34:53.171724+0000","last_fullsized":"2024-04-25T10:34:53.171724+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'15","last_scrub_stamp":"2024-04-25T10:34:53.171619+0000","last_deep_scrub":"22'15","last_deep_scrub_stamp":"2024-04-25T10:34:53.171619+0000","last_clean_scrub_stamp":"2024-04-25T10:34:53.171619+0000","objects_scrubbed":6,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:29.872635+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"22'157","reported_seq":175,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.944923+0000","last_change":"2024-04-25T10:30:32.432884+0000","last_active":"2024-04-25T10:34:43.944923+0000","last_peered":"2024-04-25T10:34:43.944923+0000","last_clean":"2024-04-25T10:34:43.944923+0000","last_became_active":"2024-04-25T10:30:32.431743+0000","last_became_peered":"2024-04-25T10:30:32.431743+0000","last_unstale":"2024-04-25T10:34:43.944923+0000","last_undegraded":"2024-04-25T10:34:43.944923+0000","last_fullsized":"2024-04-25T10:34:43.944923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"20'5","reported_seq":24,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:31:13.638219+0000","last_change":"2024-04-25T10:30:31.419497+0000","last_active":"2024-04-25T10:31:13.638219+0000","last_peered":"2024-04-25T10:31:13.638219+0000","last_clean":"2024-04-25T10:31:13.638219+0000","last_became_active":"2024-04-25T10:30:31.419135+0000","last_became_peered":"2024-04-25T10:30:31.419135+0000","last_unstale":"2024-04-25T10:31:13.638219+0000","last_undegraded":"2024-04-25T10:31:13.638219+0000","last_fullsized":"2024-04-25T10:31:13.638219+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:59.403927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"22'171","reported_seq":188,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.964294+0000","last_change":"2024-04-25T10:30:32.434197+0000","last_active":"2024-04-25T10:34:43.964294+0000","last_peered":"2024-04-25T10:34:43.964294+0000","last_clean":"2024-04-25T10:34:43.964294+0000","last_became_active":"2024-04-25T10:30:32.433623+0000","last_became_peered":"2024-04-25T10:30:32.433623+0000","last_unstale":"2024-04-25T10:34:43.964294+0000","last_undegraded":"2024-04-25T10:34:43.964294+0000","last_fullsized":"2024-04-25T10:34:43.964294+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:08.027472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"22'7","reported_seq":26,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:32:55.135185+0000","last_change":"2024-04-25T10:30:31.416901+0000","last_active":"2024-04-25T10:32:55.135185+0000","last_peered":"2024-04-25T10:32:55.135185+0000","last_clean":"2024-04-25T10:32:55.135185+0000","last_became_active":"2024-04-25T10:30:31.415706+0000","last_became_peered":"2024-04-25T10:30:31.415706+0000","last_unstale":"2024-04-25T10:32:55.135185+0000","last_undegraded":"2024-04-25T10:32:55.135185+0000","last_fullsized":"2024-04-25T10:32:55.135185+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:23.243039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"22'165","reported_seq":182,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.946112+0000","last_change":"2024-04-25T10:30:32.435089+0000","last_active":"2024-04-25T10:34:43.946112+0000","last_peered":"2024-04-25T10:34:43.946112+0000","last_clean":"2024-04-25T10:34:43.946112+0000","last_became_active":"2024-04-25T10:30:32.434737+0000","last_became_peered":"2024-04-25T10:30:32.434737+0000","last_unstale":"2024-04-25T10:34:43.946112+0000","last_undegraded":"2024-04-25T10:34:43.946112+0000","last_fullsized":"2024-04-25T10:34:43.946112+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:19.738850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159,"num_write_kb":434176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"22'18","reported_seq":42,"reported_epoch":22,"state":"active+clean+scrubbing+deep","last_fresh":"2024-04-25T10:34:55.149446+0000","last_change":"2024-04-25T10:34:55.149102+0000","last_active":"2024-04-25T10:34:55.149446+0000","last_peered":"2024-04-25T10:34:55.149446+0000","last_clean":"2024-04-25T10:34:55.149446+0000","last_became_active":"2024-04-25T10:30:31.404775+0000","last_became_peered":"2024-04-25T10:30:31.404775+0000","last_unstale":"2024-04-25T10:34:55.149446+0000","last_undegraded":"2024-04-25T10:34:55.149446+0000","last_fullsized":"2024-04-25T10:34:55.149446+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 0s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"22'189","reported_seq":206,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.961535+0000","last_change":"2024-04-25T10:30:32.432061+0000","last_active":"2024-04-25T10:34:43.961535+0000","last_peered":"2024-04-25T10:34:43.961535+0000","last_clean":"2024-04-25T10:34:43.961535+0000","last_became_active":"2024-04-25T10:30:32.431651+0000","last_became_peered":"2024-04-25T10:30:32.431651+0000","last_unstale":"2024-04-25T10:34:43.961535+0000","last_undegraded":"2024-04-25T10:34:43.961535+0000","last_fullsized":"2024-04-25T10:34:43.961535+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:39.062490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'2","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:32:50.135142+0000","last_change":"2024-04-25T10:30:31.420258+0000","last_active":"2024-04-25T10:32:50.135142+0000","last_peered":"2024-04-25T10:32:50.135142+0000","last_clean":"2024-04-25T10:32:50.135142+0000","last_became_active":"2024-04-25T10:30:31.419902+0000","last_became_peered":"2024-04-25T10:30:31.419902+0000","last_unstale":"2024-04-25T10:32:50.135142+0000","last_undegraded":"2024-04-25T10:32:50.135142+0000","last_fullsized":"2024-04-25T10:32:50.135142+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"22'146","reported_seq":163,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.963108+0000","last_change":"2024-04-25T10:30:32.433568+0000","last_active":"2024-04-25T10:34:43.963108+0000","last_peered":"2024-04-25T10:34:43.963108+0000","last_clean":"2024-04-25T10:34:43.963108+0000","last_became_active":"2024-04-25T10:30:32.432572+0000","last_became_peered":"2024-04-25T10:30:32.432572+0000","last_unstale":"2024-04-25T10:34:43.963108+0000","last_undegraded":"2024-04-25T10:34:43.963108+0000","last_fullsized":"2024-04-25T10:34:43.963108+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:31.771919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'5","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:32:50.131473+0000","last_change":"2024-04-25T10:30:31.419308+0000","last_active":"2024-04-25T10:32:50.131473+0000","last_peered":"2024-04-25T10:32:50.131473+0000","last_clean":"2024-04-25T10:32:50.131473+0000","last_became_active":"2024-04-25T10:30:31.418978+0000","last_became_peered":"2024-04-25T10:30:31.418978+0000","last_unstale":"2024-04-25T10:32:50.131473+0000","last_undegraded":"2024-04-25T10:32:50.131473+0000","last_fullsized":"2024-04-25T10:32:50.131473+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"22'138","reported_seq":155,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.966851+0000","last_change":"2024-04-25T10:30:32.436109+0000","last_active":"2024-04-25T10:34:43.966851+0000","last_peered":"2024-04-25T10:34:43.966851+0000","last_clean":"2024-04-25T10:34:43.966851+0000","last_became_active":"2024-04-25T10:30:32.434701+0000","last_became_peered":"2024-04-25T10:30:32.434701+0000","last_unstale":"2024-04-25T10:34:43.966851+0000","last_undegraded":"2024-04-25T10:34:43.966851+0000","last_fullsized":"2024-04-25T10:34:43.966851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:19.471867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'3","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:32:50.132800+0000","last_change":"2024-04-25T10:30:31.416044+0000","last_active":"2024-04-25T10:32:50.132800+0000","last_peered":"2024-04-25T10:32:50.132800+0000","last_clean":"2024-04-25T10:32:50.132800+0000","last_became_active":"2024-04-25T10:30:31.415707+0000","last_became_peered":"2024-04-25T10:30:31.415707+0000","last_unstale":"2024-04-25T10:32:50.132800+0000","last_undegraded":"2024-04-25T10:32:50.132800+0000","last_fullsized":"2024-04-25T10:32:50.132800+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"22'144","reported_seq":161,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.958006+0000","last_change":"2024-04-25T10:30:32.442971+0000","last_active":"2024-04-25T10:34:43.958006+0000","last_peered":"2024-04-25T10:34:43.958006+0000","last_clean":"2024-04-25T10:34:43.958006+0000","last_became_active":"2024-04-25T10:30:32.442656+0000","last_became_peered":"2024-04-25T10:30:32.442656+0000","last_unstale":"2024-04-25T10:34:43.958006+0000","last_undegraded":"2024-04-25T10:34:43.958006+0000","last_fullsized":"2024-04-25T10:34:43.958006+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:29.951142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"22'9","reported_seq":37,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:54.115274+0000","last_change":"2024-04-25T10:34:54.115307+0000","last_active":"2024-04-25T10:34:54.115274+0000","last_peered":"2024-04-25T10:34:54.115274+0000","last_clean":"2024-04-25T10:34:54.115274+0000","last_became_active":"2024-04-25T10:30:31.404392+0000","last_became_peered":"2024-04-25T10:30:31.404392+0000","last_unstale":"2024-04-25T10:34:54.115274+0000","last_undegraded":"2024-04-25T10:34:54.115274+0000","last_fullsized":"2024-04-25T10:34:54.115274+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'9","last_scrub_stamp":"2024-04-25T10:34:54.115185+0000","last_deep_scrub":"22'9","last_deep_scrub_stamp":"2024-04-25T10:34:54.115185+0000","last_clean_scrub_stamp":"2024-04-25T10:34:54.115185+0000","objects_scrubbed":3,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:05.747122+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1752,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"22'151","reported_seq":168,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.967451+0000","last_change":"2024-04-25T10:30:32.441438+0000","last_active":"2024-04-25T10:34:43.967451+0000","last_peered":"2024-04-25T10:34:43.967451+0000","last_clean":"2024-04-25T10:34:43.967451+0000","last_became_active":"2024-04-25T10:30:32.440011+0000","last_became_peered":"2024-04-25T10:30:32.440011+0000","last_unstale":"2024-04-25T10:34:43.967451+0000","last_undegraded":"2024-04-25T10:34:43.967451+0000","last_fullsized":"2024-04-25T10:34:43.967451+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:09.127517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":116,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:32:25.132929+0000","last_change":"2024-04-25T10:30:21.429098+0000","last_active":"2024-04-25T10:32:25.132929+0000","last_peered":"2024-04-25T10:32:25.132929+0000","last_clean":"2024-04-25T10:32:25.132929+0000","last_became_active":"2024-04-25T10:30:21.427958+0000","last_became_peered":"2024-04-25T10:30:21.427958+0000","last_unstale":"2024-04-25T10:32:25.132929+0000","last_undegraded":"2024-04-25T10:32:25.132929+0000","last_fullsized":"2024-04-25T10:32:25.132929+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:20.186206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:20.186206+0000","last_clean_scrub_stamp":"2024-04-25T10:30:20.186206+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:46.046488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"20'6","reported_seq":26,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:32:40.133596+0000","last_change":"2024-04-25T10:30:31.404974+0000","last_active":"2024-04-25T10:32:40.133596+0000","last_peered":"2024-04-25T10:32:40.133596+0000","last_clean":"2024-04-25T10:32:40.133596+0000","last_became_active":"2024-04-25T10:30:31.404549+0000","last_became_peered":"2024-04-25T10:30:31.404549+0000","last_unstale":"2024-04-25T10:32:40.133596+0000","last_undegraded":"2024-04-25T10:32:40.133596+0000","last_fullsized":"2024-04-25T10:32:40.133596+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:06.191321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"22'158","reported_seq":177,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:45.157069+0000","last_change":"2024-04-25T10:30:32.434409+0000","last_active":"2024-04-25T10:34:45.157069+0000","last_peered":"2024-04-25T10:34:45.157069+0000","last_clean":"2024-04-25T10:34:45.157069+0000","last_became_active":"2024-04-25T10:30:32.433830+0000","last_became_peered":"2024-04-25T10:30:32.433830+0000","last_unstale":"2024-04-25T10:34:45.157069+0000","last_undegraded":"2024-04-25T10:34:45.157069+0000","last_fullsized":"2024-04-25T10:34:45.157069+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":409600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"20'4","reported_seq":24,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:32:35.134131+0000","last_change":"2024-04-25T10:30:31.416025+0000","last_active":"2024-04-25T10:32:35.134131+0000","last_peered":"2024-04-25T10:32:35.134131+0000","last_clean":"2024-04-25T10:32:35.134131+0000","last_became_active":"2024-04-25T10:30:31.415675+0000","last_became_peered":"2024-04-25T10:30:31.415675+0000","last_unstale":"2024-04-25T10:32:35.134131+0000","last_undegraded":"2024-04-25T10:32:35.134131+0000","last_fullsized":"2024-04-25T10:32:35.134131+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:28:48.185715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"22'131","reported_seq":158,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:55.120078+0000","last_change":"2024-04-25T10:34:55.120104+0000","last_active":"2024-04-25T10:34:55.120078+0000","last_peered":"2024-04-25T10:34:55.120078+0000","last_clean":"2024-04-25T10:34:55.120078+0000","last_became_active":"2024-04-25T10:30:32.424303+0000","last_became_peered":"2024-04-25T10:30:32.424303+0000","last_unstale":"2024-04-25T10:34:55.120078+0000","last_undegraded":"2024-04-25T10:34:55.120078+0000","last_fullsized":"2024-04-25T10:34:55.120078+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'131","last_scrub_stamp":"2024-04-25T10:34:55.119995+0000","last_deep_scrub":"22'131","last_deep_scrub_stamp":"2024-04-25T10:34:55.119995+0000","last_clean_scrub_stamp":"2024-04-25T10:34:55.119995+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:49.222993+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"22'13","reported_seq":39,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:45.149433+0000","last_change":"2024-04-25T10:30:31.419145+0000","last_active":"2024-04-25T10:34:45.149433+0000","last_peered":"2024-04-25T10:34:45.149433+0000","last_clean":"2024-04-25T10:34:45.149433+0000","last_became_active":"2024-04-25T10:30:31.418817+0000","last_became_peered":"2024-04-25T10:30:31.418817+0000","last_unstale":"2024-04-25T10:34:45.149433+0000","last_undegraded":"2024-04-25T10:34:45.149433+0000","last_fullsized":"2024-04-25T10:34:45.149433+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":12,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"22'131","reported_seq":150,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.933629+0000","last_change":"2024-04-25T10:30:32.419911+0000","last_active":"2024-04-25T10:34:43.933629+0000","last_peered":"2024-04-25T10:34:43.933629+0000","last_clean":"2024-04-25T10:34:43.933629+0000","last_became_active":"2024-04-25T10:30:32.419486+0000","last_became_peered":"2024-04-25T10:30:32.419486+0000","last_unstale":"2024-04-25T10:34:43.933629+0000","last_undegraded":"2024-04-25T10:34:43.933629+0000","last_fullsized":"2024-04-25T10:34:43.933629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"20'5","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:32:40.133576+0000","last_change":"2024-04-25T10:30:31.405646+0000","last_active":"2024-04-25T10:32:40.133576+0000","last_peered":"2024-04-25T10:32:40.133576+0000","last_clean":"2024-04-25T10:32:40.133576+0000","last_became_active":"2024-04-25T10:30:31.405238+0000","last_became_peered":"2024-04-25T10:30:31.405238+0000","last_unstale":"2024-04-25T10:32:40.133576+0000","last_undegraded":"2024-04-25T10:32:40.133576+0000","last_fullsized":"2024-04-25T10:32:40.133576+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:13.159787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"22'145","reported_seq":164,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:45.147461+0000","last_change":"2024-04-25T10:30:32.426130+0000","last_active":"2024-04-25T10:34:45.147461+0000","last_peered":"2024-04-25T10:34:45.147461+0000","last_clean":"2024-04-25T10:34:45.147461+0000","last_became_active":"2024-04-25T10:30:32.425783+0000","last_became_peered":"2024-04-25T10:30:32.425783+0000","last_unstale":"2024-04-25T10:34:45.147461+0000","last_undegraded":"2024-04-25T10:34:45.147461+0000","last_fullsized":"2024-04-25T10:34:45.147461+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"20'1","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:45.149442+0000","last_change":"2024-04-25T10:30:31.412736+0000","last_active":"2024-04-25T10:34:45.149442+0000","last_peered":"2024-04-25T10:34:45.149442+0000","last_clean":"2024-04-25T10:34:45.149442+0000","last_became_active":"2024-04-25T10:30:31.412413+0000","last_became_peered":"2024-04-25T10:30:31.412413+0000","last_unstale":"2024-04-25T10:34:45.149442+0000","last_undegraded":"2024-04-25T10:34:45.149442+0000","last_fullsized":"2024-04-25T10:34:45.149442+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"22'143","reported_seq":166,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.951783+0000","last_change":"2024-04-25T10:31:13.640815+0000","last_active":"2024-04-25T10:34:43.951783+0000","last_peered":"2024-04-25T10:34:43.951783+0000","last_clean":"2024-04-25T10:34:43.951783+0000","last_became_active":"2024-04-25T10:31:13.640368+0000","last_became_peered":"2024-04-25T10:31:13.640368+0000","last_unstale":"2024-04-25T10:34:43.951783+0000","last_undegraded":"2024-04-25T10:34:43.951783+0000","last_fullsized":"2024-04-25T10:34:43.951783+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:15.845263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"20'4","reported_seq":24,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:32:40.134663+0000","last_change":"2024-04-25T10:30:31.417958+0000","last_active":"2024-04-25T10:32:40.134663+0000","last_peered":"2024-04-25T10:32:40.134663+0000","last_clean":"2024-04-25T10:32:40.134663+0000","last_became_active":"2024-04-25T10:30:31.417541+0000","last_became_peered":"2024-04-25T10:30:31.417541+0000","last_unstale":"2024-04-25T10:32:40.134663+0000","last_undegraded":"2024-04-25T10:32:40.134663+0000","last_fullsized":"2024-04-25T10:32:40.134663+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:35.373351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"22'144","reported_seq":163,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:45.149408+0000","last_change":"2024-04-25T10:30:32.420451+0000","last_active":"2024-04-25T10:34:45.149408+0000","last_peered":"2024-04-25T10:34:45.149408+0000","last_clean":"2024-04-25T10:34:45.149408+0000","last_became_active":"2024-04-25T10:30:32.420109+0000","last_became_peered":"2024-04-25T10:30:32.420109+0000","last_unstale":"2024-04-25T10:34:45.149408+0000","last_undegraded":"2024-04-25T10:34:45.149408+0000","last_fullsized":"2024-04-25T10:34:45.149408+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'26","reported_seq":55,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:54.614914+0000","last_change":"2024-04-25T10:34:54.614914+0000","last_active":"2024-04-25T10:34:54.614914+0000","last_peered":"2024-04-25T10:34:54.614914+0000","last_clean":"2024-04-25T10:34:54.614914+0000","last_became_active":"2024-04-25T10:30:31.404739+0000","last_became_peered":"2024-04-25T10:30:31.404739+0000","last_unstale":"2024-04-25T10:34:54.614914+0000","last_undegraded":"2024-04-25T10:34:54.614914+0000","last_fullsized":"2024-04-25T10:34:54.614914+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'26","last_scrub_stamp":"2024-04-25T10:34:54.614752+0000","last_deep_scrub":"22'26","last_deep_scrub_stamp":"2024-04-25T10:34:54.614752+0000","last_clean_scrub_stamp":"2024-04-25T10:34:54.614752+0000","objects_scrubbed":4,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:05:11.607992+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"22'143","reported_seq":162,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:54.541222+0000","last_change":"2024-04-25T10:30:32.437513+0000","last_active":"2024-04-25T10:34:54.541222+0000","last_peered":"2024-04-25T10:34:54.541222+0000","last_clean":"2024-04-25T10:34:54.541222+0000","last_became_active":"2024-04-25T10:30:32.437171+0000","last_became_peered":"2024-04-25T10:30:32.437171+0000","last_unstale":"2024-04-25T10:34:54.541222+0000","last_undegraded":"2024-04-25T10:34:54.541222+0000","last_fullsized":"2024-04-25T10:34:54.541222+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"22'8","reported_seq":28,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:25.572652+0000","last_change":"2024-04-25T10:30:31.413140+0000","last_active":"2024-04-25T10:34:25.572652+0000","last_peered":"2024-04-25T10:34:25.572652+0000","last_clean":"2024-04-25T10:34:25.572652+0000","last_became_active":"2024-04-25T10:30:31.412821+0000","last_became_peered":"2024-04-25T10:30:31.412821+0000","last_unstale":"2024-04-25T10:34:25.572652+0000","last_undegraded":"2024-04-25T10:34:25.572652+0000","last_fullsized":"2024-04-25T10:34:25.572652+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:37.177254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"22'163","reported_seq":182,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:53.136335+0000","last_change":"2024-04-25T10:30:32.434333+0000","last_active":"2024-04-25T10:34:53.136335+0000","last_peered":"2024-04-25T10:34:53.136335+0000","last_clean":"2024-04-25T10:34:53.136335+0000","last_became_active":"2024-04-25T10:30:32.434000+0000","last_became_peered":"2024-04-25T10:30:32.434000+0000","last_unstale":"2024-04-25T10:34:53.136335+0000","last_undegraded":"2024-04-25T10:34:53.136335+0000","last_fullsized":"2024-04-25T10:34:53.136335+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"20'2","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:32:45.135041+0000","last_change":"2024-04-25T10:30:31.419475+0000","last_active":"2024-04-25T10:32:45.135041+0000","last_peered":"2024-04-25T10:32:45.135041+0000","last_clean":"2024-04-25T10:32:45.135041+0000","last_became_active":"2024-04-25T10:30:31.417605+0000","last_became_peered":"2024-04-25T10:30:31.417605+0000","last_unstale":"2024-04-25T10:32:45.135041+0000","last_undegraded":"2024-04-25T10:32:45.135041+0000","last_fullsized":"2024-04-25T10:32:45.135041+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:00.948269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"22'134","reported_seq":153,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:54.998978+0000","last_change":"2024-04-25T10:30:32.419693+0000","last_active":"2024-04-25T10:34:54.998978+0000","last_peered":"2024-04-25T10:34:54.998978+0000","last_clean":"2024-04-25T10:34:54.998978+0000","last_became_active":"2024-04-25T10:30:32.419370+0000","last_became_peered":"2024-04-25T10:30:32.419370+0000","last_unstale":"2024-04-25T10:34:54.998978+0000","last_undegraded":"2024-04-25T10:34:54.998978+0000","last_fullsized":"2024-04-25T10:34:54.998978+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"20'7","reported_seq":28,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:53.610125+0000","last_change":"2024-04-25T10:30:31.410767+0000","last_active":"2024-04-25T10:34:53.610125+0000","last_peered":"2024-04-25T10:34:53.610125+0000","last_clean":"2024-04-25T10:34:53.610125+0000","last_became_active":"2024-04-25T10:30:31.410404+0000","last_became_peered":"2024-04-25T10:30:31.410404+0000","last_unstale":"2024-04-25T10:34:53.610125+0000","last_undegraded":"2024-04-25T10:34:53.610125+0000","last_fullsized":"2024-04-25T10:34:53.610125+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"22'153","reported_seq":172,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:54.541292+0000","last_change":"2024-04-25T10:30:32.433687+0000","last_active":"2024-04-25T10:34:54.541292+0000","last_peered":"2024-04-25T10:34:54.541292+0000","last_clean":"2024-04-25T10:34:54.541292+0000","last_became_active":"2024-04-25T10:30:32.433389+0000","last_became_peered":"2024-04-25T10:30:32.433389+0000","last_unstale":"2024-04-25T10:34:54.541292+0000","last_undegraded":"2024-04-25T10:34:54.541292+0000","last_fullsized":"2024-04-25T10:34:54.541292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"20'3","reported_seq":24,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:53.610125+0000","last_change":"2024-04-25T10:30:31.419118+0000","last_active":"2024-04-25T10:34:53.610125+0000","last_peered":"2024-04-25T10:34:53.610125+0000","last_clean":"2024-04-25T10:34:53.610125+0000","last_became_active":"2024-04-25T10:30:31.418822+0000","last_became_peered":"2024-04-25T10:30:31.418822+0000","last_unstale":"2024-04-25T10:34:53.610125+0000","last_undegraded":"2024-04-25T10:34:53.610125+0000","last_fullsized":"2024-04-25T10:34:53.610125+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"22'143","reported_seq":161,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.938900+0000","last_change":"2024-04-25T10:30:32.427894+0000","last_active":"2024-04-25T10:34:43.938900+0000","last_peered":"2024-04-25T10:34:43.938900+0000","last_clean":"2024-04-25T10:34:43.938900+0000","last_became_active":"2024-04-25T10:30:32.427504+0000","last_became_peered":"2024-04-25T10:30:32.427504+0000","last_unstale":"2024-04-25T10:34:43.938900+0000","last_undegraded":"2024-04-25T10:34:43.938900+0000","last_fullsized":"2024-04-25T10:34:43.938900+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'5","reported_seq":24,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:32:50.135474+0000","last_change":"2024-04-25T10:30:31.417115+0000","last_active":"2024-04-25T10:32:50.135474+0000","last_peered":"2024-04-25T10:32:50.135474+0000","last_clean":"2024-04-25T10:32:50.135474+0000","last_became_active":"2024-04-25T10:30:31.416766+0000","last_became_peered":"2024-04-25T10:30:31.416766+0000","last_unstale":"2024-04-25T10:32:50.135474+0000","last_undegraded":"2024-04-25T10:32:50.135474+0000","last_fullsized":"2024-04-25T10:32:50.135474+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:06.021270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"22'128","reported_seq":146,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.962576+0000","last_change":"2024-04-25T10:30:32.431674+0000","last_active":"2024-04-25T10:34:43.962576+0000","last_peered":"2024-04-25T10:34:43.962576+0000","last_clean":"2024-04-25T10:34:43.962576+0000","last_became_active":"2024-04-25T10:30:32.431395+0000","last_became_peered":"2024-04-25T10:30:32.431395+0000","last_unstale":"2024-04-25T10:34:43.962576+0000","last_undegraded":"2024-04-25T10:34:43.962576+0000","last_fullsized":"2024-04-25T10:34:43.962576+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:49.726323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"22'183","reported_seq":200,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.949651+0000","last_change":"2024-04-25T10:30:32.437700+0000","last_active":"2024-04-25T10:34:43.949651+0000","last_peered":"2024-04-25T10:34:43.949651+0000","last_clean":"2024-04-25T10:34:43.949651+0000","last_became_active":"2024-04-25T10:30:32.437242+0000","last_became_peered":"2024-04-25T10:30:32.437242+0000","last_unstale":"2024-04-25T10:34:43.949651+0000","last_undegraded":"2024-04-25T10:34:43.949651+0000","last_fullsized":"2024-04-25T10:34:43.949651+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:06.499414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":168,"num_write_kb":458752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'5","reported_seq":24,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:31:13.675847+0000","last_change":"2024-04-25T10:30:31.419678+0000","last_active":"2024-04-25T10:31:13.675847+0000","last_peered":"2024-04-25T10:31:13.675847+0000","last_clean":"2024-04-25T10:31:13.675847+0000","last_became_active":"2024-04-25T10:30:31.419269+0000","last_became_peered":"2024-04-25T10:30:31.419269+0000","last_unstale":"2024-04-25T10:31:13.675847+0000","last_undegraded":"2024-04-25T10:31:13.675847+0000","last_fullsized":"2024-04-25T10:31:13.675847+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:33.034260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"22'141","reported_seq":158,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.966109+0000","last_change":"2024-04-25T10:30:32.434410+0000","last_active":"2024-04-25T10:34:43.966109+0000","last_peered":"2024-04-25T10:34:43.966109+0000","last_clean":"2024-04-25T10:34:43.966109+0000","last_became_active":"2024-04-25T10:30:32.434093+0000","last_became_peered":"2024-04-25T10:30:32.434093+0000","last_unstale":"2024-04-25T10:34:43.966109+0000","last_undegraded":"2024-04-25T10:34:43.966109+0000","last_fullsized":"2024-04-25T10:34:43.966109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:46.964704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'13","reported_seq":32,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:33:05.136991+0000","last_change":"2024-04-25T10:30:31.418360+0000","last_active":"2024-04-25T10:33:05.136991+0000","last_peered":"2024-04-25T10:33:05.136991+0000","last_clean":"2024-04-25T10:33:05.136991+0000","last_became_active":"2024-04-25T10:30:31.418005+0000","last_became_peered":"2024-04-25T10:30:31.418005+0000","last_unstale":"2024-04-25T10:33:05.136991+0000","last_undegraded":"2024-04-25T10:33:05.136991+0000","last_fullsized":"2024-04-25T10:33:05.136991+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:15.556859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"22'135","reported_seq":154,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:45.150781+0000","last_change":"2024-04-25T10:30:32.439515+0000","last_active":"2024-04-25T10:34:45.150781+0000","last_peered":"2024-04-25T10:34:45.150781+0000","last_clean":"2024-04-25T10:34:45.150781+0000","last_became_active":"2024-04-25T10:30:32.439101+0000","last_became_peered":"2024-04-25T10:30:32.439101+0000","last_unstale":"2024-04-25T10:34:45.150781+0000","last_undegraded":"2024-04-25T10:34:45.150781+0000","last_fullsized":"2024-04-25T10:34:45.150781+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'2","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:35.156051+0000","last_change":"2024-04-25T10:30:31.418070+0000","last_active":"2024-04-25T10:34:35.156051+0000","last_peered":"2024-04-25T10:34:35.156051+0000","last_clean":"2024-04-25T10:34:35.156051+0000","last_became_active":"2024-04-25T10:30:31.417737+0000","last_became_peered":"2024-04-25T10:30:31.417737+0000","last_unstale":"2024-04-25T10:34:35.156051+0000","last_undegraded":"2024-04-25T10:34:35.156051+0000","last_fullsized":"2024-04-25T10:34:35.156051+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:30.383223+0000","last_clean_scrub_stamp":"2024-04-25T10:30:30.383223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"22'152","reported_seq":171,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:43.960329+0000","last_change":"2024-04-25T10:30:32.435258+0000","last_active":"2024-04-25T10:34:43.960329+0000","last_peered":"2024-04-25T10:34:43.960329+0000","last_clean":"2024-04-25T10:34:43.960329+0000","last_became_active":"2024-04-25T10:30:32.434930+0000","last_became_peered":"2024-04-25T10:30:32.434930+0000","last_unstale":"2024-04-25T10:34:43.960329+0000","last_undegraded":"2024-04-25T10:34:43.960329+0000","last_fullsized":"2024-04-25T10:34:43.960329+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:31.390153+0000","last_clean_scrub_stamp":"2024-04-25T10:30:31.390153+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'19","reported_seq":48,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:54.172324+0000","last_change":"2024-04-25T10:34:54.172393+0000","last_active":"2024-04-25T10:34:54.172324+0000","last_peered":"2024-04-25T10:34:54.172324+0000","last_clean":"2024-04-25T10:34:54.172324+0000","last_became_active":"2024-04-25T10:30:31.415071+0000","last_became_peered":"2024-04-25T10:30:31.415071+0000","last_unstale":"2024-04-25T10:34:54.172324+0000","last_undegraded":"2024-04-25T10:34:54.172324+0000","last_fullsized":"2024-04-25T10:34:54.172324+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'19","last_scrub_stamp":"2024-04-25T10:34:54.172184+0000","last_deep_scrub":"22'19","last_deep_scrub_stamp":"2024-04-25T10:34:54.172184+0000","last_clean_scrub_stamp":"2024-04-25T10:34:54.172184+0000","objects_scrubbed":6,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:34.737710+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9216,"num_write_kb":25165824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9717,"ondisk_log_size":9717,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":210463,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":345,"num_write_kb":324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":10000,"num_omap_keys":20,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":671744,"data_stored":420926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":328,"ondisk_log_size":328,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574907,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40336,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331504,"statfs":{"total":96636764160,"available":96595460096,"internally_reserved":0,"allocated":860160,"data_stored":676182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574907,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35784,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336056,"statfs":{"total":96636764160,"available":96600121344,"internally_reserved":0,"allocated":458752,"data_stored":286294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574908,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39768,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94332072,"statfs":{"total":96636764160,"available":96596041728,"internally_reserved":0,"allocated":278528,"data_stored":87646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574908,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39744,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94332096,"statfs":{"total":96636764160,"available":96596066304,"internally_reserved":0,"allocated":253952,"data_stored":85044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574907,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39952,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331888,"statfs":{"total":96636764160,"available":96595853312,"internally_reserved":0,"allocated":466944,"data_stored":286833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574907,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35596,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336244,"statfs":{"total":96636764160,"available":96600313856,"internally_reserved":0,"allocated":266240,"data_stored":85681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574907,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35588,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336252,"statfs":{"total":96636764160,"available":96600322048,"internally_reserved":0,"allocated":258048,"data_stored":84664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574907,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40328,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331512,"statfs":{"total":96636764160,"available":96595468288,"internally_reserved":0,"allocated":851968,"data_stored":674374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":237568,"data_stored":203701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":229376,"data_stored":203162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:34:56.729 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2024-04-25T10:30:31.390153+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.730 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2024-04-25T10:30:31.390153+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.730 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2024-04-25T10:30:31.390153+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.730 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2024-04-25T10:30:31.390153+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.730 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-04-25T10:30:31.390153+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.730 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2024-04-25T10:30:31.390153+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.730 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-04-25T10:30:31.390153+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.730 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2024-04-25T10:30:31.390153+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.730 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-04-25T10:30:31.390153+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.731 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2024-04-25T10:30:31.390153+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.731 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2024-04-25T10:30:31.390153+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.731 INFO:tasks.ceph:pgid 3.34 last_scrub_stamp 2024-04-25T10:30:31.390153+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.731 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2024-04-25T10:30:31.390153+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.731 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2024-04-25T10:30:31.390153+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.731 INFO:tasks.ceph:pgid 3.31 last_scrub_stamp 2024-04-25T10:30:31.390153+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.731 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2024-04-25T10:30:31.390153+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.731 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2024-04-25T10:30:31.390153+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.731 INFO:tasks.ceph:pgid 3.2e last_scrub_stamp 2024-04-25T10:30:31.390153+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.731 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2024-04-25T10:30:31.390153+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.732 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2024-04-25T10:30:31.390153+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.732 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2024-04-25T10:30:31.390153+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.732 INFO:tasks.ceph:pgid 3.2a last_scrub_stamp 2024-04-25T10:30:31.390153+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.732 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2024-04-25T10:30:31.390153+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.732 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-04-25T10:30:31.390153+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.732 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2024-04-25T10:30:31.390153+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.732 INFO:tasks.ceph:pgid 3.26 last_scrub_stamp 2024-04-25T10:30:31.390153+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.732 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2024-04-25T10:30:31.390153+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.732 INFO:tasks.ceph:pgid 3.24 last_scrub_stamp 2024-04-25T10:30:31.390153+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.733 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2024-04-25T10:30:31.390153+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.733 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-04-25T10:30:31.390153+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.733 INFO:tasks.ceph:pgid 3.21 last_scrub_stamp 2024-04-25T10:30:31.390153+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.733 INFO:tasks.ceph:pgid 3.20 last_scrub_stamp 2024-04-25T10:30:31.390153+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.733 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-04-25T10:30:31.390153+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.733 INFO:tasks.ceph:pgid 2.1b last_scrub_stamp 2024-04-25T10:30:30.383223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.733 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-04-25T10:30:31.390153+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.733 INFO:tasks.ceph:pgid 2.1a last_scrub_stamp 2024-04-25T10:30:30.383223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.733 INFO:tasks.ceph:pgid 2.19 last_scrub_stamp 2024-04-25T10:30:30.383223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.734 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-04-25T10:30:31.390153+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.734 INFO:tasks.ceph:pgid 2.18 last_scrub_stamp 2024-04-25T10:30:30.383223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.734 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-04-25T10:30:31.390153+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.734 INFO:tasks.ceph:pgid 2.17 last_scrub_stamp 2024-04-25T10:30:30.383223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.734 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-04-25T10:30:31.390153+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.734 INFO:tasks.ceph:pgid 2.16 last_scrub_stamp 2024-04-25T10:30:30.383223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.734 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-04-25T10:30:31.390153+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.734 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-04-25T10:30:31.390153+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.735 INFO:tasks.ceph:pgid 2.14 last_scrub_stamp 2024-04-25T10:30:30.383223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.735 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-04-25T10:30:31.390153+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.735 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-04-25T10:30:31.390153+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.735 INFO:tasks.ceph:pgid 2.12 last_scrub_stamp 2024-04-25T10:30:30.383223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.735 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-04-25T10:30:31.390153+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.735 INFO:tasks.ceph:pgid 2.11 last_scrub_stamp 2024-04-25T10:30:30.383223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.735 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-04-25T10:30:31.390153+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.735 INFO:tasks.ceph:pgid 2.10 last_scrub_stamp 2024-04-25T10:30:30.383223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.735 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-04-25T10:30:31.390153+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.735 INFO:tasks.ceph:pgid 2.f last_scrub_stamp 2024-04-25T10:30:30.383223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.736 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-04-25T10:30:31.390153+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.736 INFO:tasks.ceph:pgid 2.e last_scrub_stamp 2024-04-25T10:30:30.383223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.736 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-04-25T10:30:31.390153+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.736 INFO:tasks.ceph:pgid 2.d last_scrub_stamp 2024-04-25T10:30:30.383223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.736 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-04-25T10:30:31.390153+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.736 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-04-25T10:30:31.390153+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.736 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-25T10:30:20.186206+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=20, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.736 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-04-25T10:30:30.383223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.736 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-04-25T10:30:31.390153+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.737 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-04-25T10:30:30.383223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.737 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-04-25T10:30:30.383223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.737 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-04-25T10:30:31.390153+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.737 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-04-25T10:30:30.383223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.737 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-04-25T10:30:31.390153+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.737 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-04-25T10:30:30.383223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.737 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-04-25T10:30:31.390153+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.737 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-04-25T10:30:30.383223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.737 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-04-25T10:30:31.390153+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.738 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-04-25T10:30:31.390153+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.738 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-04-25T10:30:30.383223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.738 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-04-25T10:30:31.390153+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.738 INFO:tasks.ceph:pgid 2.8 last_scrub_stamp 2024-04-25T10:30:30.383223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.738 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-04-25T10:30:31.390153+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.738 INFO:tasks.ceph:pgid 2.9 last_scrub_stamp 2024-04-25T10:30:30.383223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.738 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-04-25T10:30:31.390153+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.738 INFO:tasks.ceph:pgid 2.a last_scrub_stamp 2024-04-25T10:30:30.383223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.738 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-04-25T10:30:31.390153+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.739 INFO:tasks.ceph:pgid 2.b last_scrub_stamp 2024-04-25T10:30:30.383223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.739 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-04-25T10:30:31.390153+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.739 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-04-25T10:30:31.390153+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.739 INFO:tasks.ceph:pgid 2.1c last_scrub_stamp 2024-04-25T10:30:30.383223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.739 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-04-25T10:30:31.390153+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.739 INFO:tasks.ceph:pgid 2.1d last_scrub_stamp 2024-04-25T10:30:30.383223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.739 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-04-25T10:30:31.390153+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.739 INFO:tasks.ceph:pgid 2.1e last_scrub_stamp 2024-04-25T10:30:30.383223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.739 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-04-25T10:30:31.390153+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=30, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=34, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:34:56.740 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-25T10:35:03.042 DEBUG:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:35:03.045 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:35:16.741 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:35:16.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:16.860+0000 7f44290f9640 1 Processor -- start 2024-04-25T10:35:16.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:16.860+0000 7f44290f9640 1 -- start start 2024-04-25T10:35:16.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:16.861+0000 7f44290f9640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f44240c8d90 0x7f4424156ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:16.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:16.861+0000 7f44290f9640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f44240b4ce0 0x7f442415ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:16.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:16.861+0000 7f44290f9640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44240ce630 con 0x7f44240c7e30 2024-04-25T10:35:16.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:16.861+0000 7f44290f9640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f44240b50f0 con 0x7f44240c8d90 2024-04-25T10:35:16.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:16.861+0000 7f44290f9640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f44240c7c60 con 0x7f44240b4ce0 2024-04-25T10:35:16.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:16.861+0000 7f442259c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f44240c8d90 0x7f4424156ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:16.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:16.861+0000 7f442259c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f44240c8d90 0x7f4424156ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38790/0 (socket says 172.21.15.106:38790) 2024-04-25T10:35:16.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:16.861+0000 7f442259c640 1 -- 172.21.15.106:0/2101191946 learned_addr learned my addr 172.21.15.106:0/2101191946 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:35:16.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:16.861+0000 7f4422d9d640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f44240c7e30 0x7f442406d000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:58890/0 (socket says 172.21.15.106:58890) 2024-04-25T10:35:16.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:16.861+0000 7f442359e640 1 --2- 172.21.15.106:0/2101191946 >> v2:172.21.15.134:3300/0 conn(0x7f44240b4ce0 0x7f442415ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:16.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:16.862+0000 7f442259c640 1 -- 172.21.15.106:0/2101191946 >> v2:172.21.15.134:3300/0 conn(0x7f44240b4ce0 msgr2=0x7f442415ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:16.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:16.862+0000 7f442259c640 1 --2- 172.21.15.106:0/2101191946 >> v2:172.21.15.134:3300/0 conn(0x7f44240b4ce0 0x7f442415ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:16.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:16.862+0000 7f442259c640 1 -- 172.21.15.106:0/2101191946 >> v1:172.21.15.106:6789/0 conn(0x7f44240c7e30 legacy=0x7f442406d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:16.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:16.862+0000 7f4421d9b640 1 -- 172.21.15.106:0/2101191946 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3814834991 0 0) 0x7f44240ce630 con 0x7f44240c7e30 2024-04-25T10:35:16.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:16.862+0000 7f442259c640 1 -- 172.21.15.106:0/2101191946 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44240c7980 con 0x7f44240c8d90 2024-04-25T10:35:16.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:16.862+0000 7f442259c640 1 --2- 172.21.15.106:0/2101191946 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f44240c8d90 0x7f4424156ef0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f44240b6500 tx=0x7f441802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b241f5df47e53525 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:16.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:16.862+0000 7f4421d9b640 1 -- 172.21.15.106:0/2101191946 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4418004030 con 0x7f44240c8d90 2024-04-25T10:35:16.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:16.862+0000 7f4421d9b640 1 -- 172.21.15.106:0/2101191946 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44180041d0 con 0x7f44240c8d90 2024-04-25T10:35:16.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:16.862+0000 7f4421d9b640 1 -- 172.21.15.106:0/2101191946 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4418003610 con 0x7f44240c8d90 2024-04-25T10:35:16.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:16.863+0000 7f44290f9640 1 -- 172.21.15.106:0/2101191946 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f44240c8d90 msgr2=0x7f4424156ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:16.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:16.863+0000 7f44290f9640 1 --2- 172.21.15.106:0/2101191946 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f44240c8d90 0x7f4424156ef0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f44240b6500 tx=0x7f441802a1c0 comp rx=0 tx=0).stop 2024-04-25T10:35:16.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:16.863+0000 7f44290f9640 1 -- 172.21.15.106:0/2101191946 shutdown_connections 2024-04-25T10:35:16.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:16.863+0000 7f44290f9640 1 --2- 172.21.15.106:0/2101191946 >> v2:172.21.15.134:3300/0 conn(0x7f44240b4ce0 0x7f442415ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:16.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:16.863+0000 7f44290f9640 1 --2- 172.21.15.106:0/2101191946 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f44240c8d90 0x7f4424156ef0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:16.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:16.863+0000 7f44290f9640 1 -- 172.21.15.106:0/2101191946 >> 172.21.15.106:0/2101191946 conn(0x7f4424068b10 msgr2=0x7f44240d1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:16.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:16.863+0000 7f44290f9640 1 -- 172.21.15.106:0/2101191946 shutdown_connections 2024-04-25T10:35:16.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:16.863+0000 7f44290f9640 1 -- 172.21.15.106:0/2101191946 wait complete. 2024-04-25T10:35:16.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:16.865+0000 7f44290f9640 1 Processor -- start 2024-04-25T10:35:16.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:16.865+0000 7f44290f9640 1 -- start start 2024-04-25T10:35:16.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:16.867+0000 7f44290f9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f44240b4ce0 0x7f44241f3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:16.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:16.867+0000 7f44290f9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44240c7e30 0x7f44241512e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:16.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:16.867+0000 7f44290f9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f44240c8d90 0x7f44240d0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:16.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:16.867+0000 7f44290f9640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f442406d710 con 0x7f44240c8d90 2024-04-25T10:35:16.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:16.867+0000 7f44290f9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f44240c72f0 con 0x7f44240c7e30 2024-04-25T10:35:16.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:16.867+0000 7f44290f9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f442406d890 con 0x7f44240b4ce0 2024-04-25T10:35:16.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:16.867+0000 7f442359e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f44240c8d90 0x7f44240d0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:16.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:16.867+0000 7f442359e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f44240c8d90 0x7f44240d0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38798/0 (socket says 172.21.15.106:38798) 2024-04-25T10:35:16.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:16.867+0000 7f442359e640 1 -- 172.21.15.106:0/2183688130 learned_addr learned my addr 172.21.15.106:0/2183688130 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:35:16.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:16.867+0000 7f442259c640 1 --2- 172.21.15.106:0/2183688130 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44240c7e30 0x7f44241512e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:16.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:16.867+0000 7f4422d9d640 1 --2- 172.21.15.106:0/2183688130 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f44240b4ce0 0x7f44241f3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:16.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:16.867+0000 7f442359e640 1 -- 172.21.15.106:0/2183688130 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f44240b4ce0 msgr2=0x7f44241f3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:16.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:16.867+0000 7f442359e640 1 --2- 172.21.15.106:0/2183688130 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f44240b4ce0 0x7f44241f3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:16.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:16.867+0000 7f442359e640 1 -- 172.21.15.106:0/2183688130 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44240c7e30 msgr2=0x7f44241512e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:16.871 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:16.867+0000 7f442359e640 1 --2- 172.21.15.106:0/2183688130 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44240c7e30 0x7f44241512e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:16.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:16.867+0000 7f442359e640 1 -- 172.21.15.106:0/2183688130 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44241f4190 con 0x7f44240c8d90 2024-04-25T10:35:16.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:16.867+0000 7f442359e640 1 --2- 172.21.15.106:0/2183688130 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f44240c8d90 0x7f44240d0810 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f441c009fd0 tx=0x7f441c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:16.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:16.867+0000 7f440b7fe640 1 -- 172.21.15.106:0/2183688130 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441c014420 con 0x7f44240c8d90 2024-04-25T10:35:16.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:16.868+0000 7f440b7fe640 1 -- 172.21.15.106:0/2183688130 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f441c019070 con 0x7f44240c8d90 2024-04-25T10:35:16.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:16.868+0000 7f440b7fe640 1 -- 172.21.15.106:0/2183688130 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441c014800 con 0x7f44240c8d90 2024-04-25T10:35:16.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:16.868+0000 7f44290f9640 1 -- 172.21.15.106:0/2183688130 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4424151820 con 0x7f44240c8d90 2024-04-25T10:35:16.872 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:16.868+0000 7f44290f9640 1 -- 172.21.15.106:0/2183688130 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4424150800 con 0x7f44240c8d90 2024-04-25T10:35:16.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:16.869+0000 7f44290f9640 1 -- 172.21.15.106:0/2183688130 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f442406a3d0 con 0x7f44240c8d90 2024-04-25T10:35:16.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:16.869+0000 7f440b7fe640 1 -- 172.21.15.106:0/2183688130 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f441c0149a0 con 0x7f44240c8d90 2024-04-25T10:35:16.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:16.870+0000 7f440b7fe640 1 --2- 172.21.15.106:0/2183688130 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f44100744e0 0x7f44100769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:16.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:16.870+0000 7f440b7fe640 1 -- 172.21.15.106:0/2183688130 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f441c095950 con 0x7f44240c8d90 2024-04-25T10:35:16.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:16.870+0000 7f4422d9d640 1 --2- 172.21.15.106:0/2183688130 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f44100744e0 0x7f44100769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:16.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:16.870+0000 7f4422d9d640 1 --2- 172.21.15.106:0/2183688130 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f44100744e0 0x7f44100769f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4414007fd0 tx=0x7f4414007e00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:16.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:16.875+0000 7f440b7fe640 1 -- 172.21.15.106:0/2183688130 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f44241f4190 con 0x7f44240c8d90 2024-04-25T10:35:17.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.046+0000 7f44290f9640 1 -- 172.21.15.106:0/2183688130 --> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f44240d1d20 con 0x7f44100744e0 2024-04-25T10:35:17.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.052+0000 7f440b7fe640 1 -- 172.21.15.106:0/2183688130 <== mgr.4107 v2:172.21.15.106:6832/1744210167 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+253451 (secure 0 0 0) 0x7f4414004c70 con 0x7f44100744e0 2024-04-25T10:35:17.057 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:35:17.058 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-25T10:35:17.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.055+0000 7f44290f9640 1 -- 172.21.15.106:0/2183688130 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f44100744e0 msgr2=0x7f44100769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:17.060 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.055+0000 7f44290f9640 1 --2- 172.21.15.106:0/2183688130 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f44100744e0 0x7f44100769f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4414007fd0 tx=0x7f4414007e00 comp rx=0 tx=0).stop 2024-04-25T10:35:17.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.055+0000 7f44290f9640 1 -- 172.21.15.106:0/2183688130 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f44240c8d90 msgr2=0x7f44240d0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:17.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.055+0000 7f44290f9640 1 --2- 172.21.15.106:0/2183688130 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f44240c8d90 0x7f44240d0810 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f441c009fd0 tx=0x7f441c00e4b0 comp rx=0 tx=0).stop 2024-04-25T10:35:17.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.055+0000 7f44290f9640 1 -- 172.21.15.106:0/2183688130 shutdown_connections 2024-04-25T10:35:17.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.055+0000 7f44290f9640 1 --2- 172.21.15.106:0/2183688130 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f44100744e0 0x7f44100769f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:17.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.055+0000 7f44290f9640 1 --2- 172.21.15.106:0/2183688130 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f44240c8d90 0x7f44240d0810 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:17.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.055+0000 7f44290f9640 1 --2- 172.21.15.106:0/2183688130 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44240c7e30 0x7f44241512e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:17.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.055+0000 7f44290f9640 1 --2- 172.21.15.106:0/2183688130 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f44240b4ce0 0x7f44241f3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:17.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.055+0000 7f44290f9640 1 -- 172.21.15.106:0/2183688130 >> 172.21.15.106:0/2183688130 conn(0x7f4424068b10 msgr2=0x7f4424155420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:17.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.055+0000 7f44290f9640 1 -- 172.21.15.106:0/2183688130 shutdown_connections 2024-04-25T10:35:17.061 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.055+0000 7f44290f9640 1 -- 172.21.15.106:0/2183688130 wait complete. 2024-04-25T10:35:17.069 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":163,"stamp":"2024-04-25T10:35:16.167621+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":800831,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":71,"num_read_kb":60,"num_write":9662,"num_write_kb":25167776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":25000,"num_omap_keys":50,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10187,"ondisk_log_size":10187,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":307096,"kb_used_data":3608,"kb_used_omap":12,"kb_used_meta":303411,"kb_avail":754667624,"statfs":{"total":773094113280,"available":772779646976,"internally_reserved":0,"allocated":3694592,"data_stored":2266718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":310693424},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003257"},"pg_stats":[{"pgid":"3.3f","version":"22'146","reported_seq":173,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:35:12.282972+0000","last_change":"2024-04-25T10:35:12.283002+0000","last_active":"2024-04-25T10:35:12.282972+0000","last_peered":"2024-04-25T10:35:12.282972+0000","last_clean":"2024-04-25T10:35:12.282972+0000","last_became_active":"2024-04-25T10:30:32.440552+0000","last_became_peered":"2024-04-25T10:30:32.440552+0000","last_unstale":"2024-04-25T10:35:12.282972+0000","last_undegraded":"2024-04-25T10:35:12.282972+0000","last_fullsized":"2024-04-25T10:35:12.282972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'146","last_scrub_stamp":"2024-04-25T10:35:12.282859+0000","last_deep_scrub":"22'146","last_deep_scrub_stamp":"2024-04-25T10:35:12.282859+0000","last_clean_scrub_stamp":"2024-04-25T10:35:12.282859+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:47.704669+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"22'130","reported_seq":157,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:35:04.684705+0000","last_change":"2024-04-25T10:35:04.684737+0000","last_active":"2024-04-25T10:35:04.684705+0000","last_peered":"2024-04-25T10:35:04.684705+0000","last_clean":"2024-04-25T10:35:04.684705+0000","last_became_active":"2024-04-25T10:30:32.425368+0000","last_became_peered":"2024-04-25T10:30:32.425368+0000","last_unstale":"2024-04-25T10:35:04.684705+0000","last_undegraded":"2024-04-25T10:35:04.684705+0000","last_fullsized":"2024-04-25T10:35:04.684705+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'130","last_scrub_stamp":"2024-04-25T10:35:04.684590+0000","last_deep_scrub":"22'130","last_deep_scrub_stamp":"2024-04-25T10:35:04.684590+0000","last_clean_scrub_stamp":"2024-04-25T10:35:04.684590+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:00.739256+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"22'151","reported_seq":178,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:58.151405+0000","last_change":"2024-04-25T10:34:58.151457+0000","last_active":"2024-04-25T10:34:58.151405+0000","last_peered":"2024-04-25T10:34:58.151405+0000","last_clean":"2024-04-25T10:34:58.151405+0000","last_became_active":"2024-04-25T10:30:32.429754+0000","last_became_peered":"2024-04-25T10:30:32.429754+0000","last_unstale":"2024-04-25T10:34:58.151405+0000","last_undegraded":"2024-04-25T10:34:58.151405+0000","last_fullsized":"2024-04-25T10:34:58.151405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'151","last_scrub_stamp":"2024-04-25T10:34:58.151234+0000","last_deep_scrub":"22'151","last_deep_scrub_stamp":"2024-04-25T10:34:58.151234+0000","last_clean_scrub_stamp":"2024-04-25T10:34:58.151234+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:03.852196+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"22'173","reported_seq":200,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:35:07.132802+0000","last_change":"2024-04-25T10:35:07.132836+0000","last_active":"2024-04-25T10:35:07.132802+0000","last_peered":"2024-04-25T10:35:07.132802+0000","last_clean":"2024-04-25T10:35:07.132802+0000","last_became_active":"2024-04-25T10:30:32.442182+0000","last_became_peered":"2024-04-25T10:30:32.442182+0000","last_unstale":"2024-04-25T10:35:07.132802+0000","last_undegraded":"2024-04-25T10:35:07.132802+0000","last_fullsized":"2024-04-25T10:35:07.132802+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'173","last_scrub_stamp":"2024-04-25T10:35:07.132672+0000","last_deep_scrub":"22'173","last_deep_scrub_stamp":"2024-04-25T10:35:07.132672+0000","last_clean_scrub_stamp":"2024-04-25T10:35:07.132672+0000","objects_scrubbed":0,"log_size":173,"log_dups_size":0,"ondisk_log_size":173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:38.969664+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":450560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"22'148","reported_seq":174,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:35:10.304228+0000","last_change":"2024-04-25T10:35:10.304291+0000","last_active":"2024-04-25T10:35:10.304228+0000","last_peered":"2024-04-25T10:35:10.304228+0000","last_clean":"2024-04-25T10:35:10.304228+0000","last_became_active":"2024-04-25T10:30:32.437656+0000","last_became_peered":"2024-04-25T10:30:32.437656+0000","last_unstale":"2024-04-25T10:35:10.304228+0000","last_undegraded":"2024-04-25T10:35:10.304228+0000","last_fullsized":"2024-04-25T10:35:10.304228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'148","last_scrub_stamp":"2024-04-25T10:35:10.304108+0000","last_deep_scrub":"22'148","last_deep_scrub_stamp":"2024-04-25T10:35:10.304108+0000","last_clean_scrub_stamp":"2024-04-25T10:35:10.304108+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:22.993599+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"22'135","reported_seq":155,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:35:00.062347+0000","last_change":"2024-04-25T10:35:00.062378+0000","last_active":"2024-04-25T10:35:00.062347+0000","last_peered":"2024-04-25T10:35:00.062347+0000","last_clean":"2024-04-25T10:35:00.062347+0000","last_became_active":"2024-04-25T10:31:13.676013+0000","last_became_peered":"2024-04-25T10:31:13.676013+0000","last_unstale":"2024-04-25T10:35:00.062347+0000","last_undegraded":"2024-04-25T10:35:00.062347+0000","last_fullsized":"2024-04-25T10:35:00.062347+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'135","last_scrub_stamp":"2024-04-25T10:35:00.062218+0000","last_deep_scrub":"22'135","last_deep_scrub_stamp":"2024-04-25T10:35:00.062218+0000","last_clean_scrub_stamp":"2024-04-25T10:35:00.062218+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:45.338357+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"22'164","reported_seq":190,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:35:06.332209+0000","last_change":"2024-04-25T10:35:06.332239+0000","last_active":"2024-04-25T10:35:06.332209+0000","last_peered":"2024-04-25T10:35:06.332209+0000","last_clean":"2024-04-25T10:35:06.332209+0000","last_became_active":"2024-04-25T10:30:32.429131+0000","last_became_peered":"2024-04-25T10:30:32.429131+0000","last_unstale":"2024-04-25T10:35:06.332209+0000","last_undegraded":"2024-04-25T10:35:06.332209+0000","last_fullsized":"2024-04-25T10:35:06.332209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'164","last_scrub_stamp":"2024-04-25T10:35:06.332101+0000","last_deep_scrub":"22'164","last_deep_scrub_stamp":"2024-04-25T10:35:06.332101+0000","last_clean_scrub_stamp":"2024-04-25T10:35:06.332101+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:42.023986+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159,"num_write_kb":434176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"22'186","reported_seq":212,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:35:08.172768+0000","last_change":"2024-04-25T10:35:08.172798+0000","last_active":"2024-04-25T10:35:08.172768+0000","last_peered":"2024-04-25T10:35:08.172768+0000","last_clean":"2024-04-25T10:35:08.172768+0000","last_became_active":"2024-04-25T10:30:32.438491+0000","last_became_peered":"2024-04-25T10:30:32.438491+0000","last_unstale":"2024-04-25T10:35:08.172768+0000","last_undegraded":"2024-04-25T10:35:08.172768+0000","last_fullsized":"2024-04-25T10:35:08.172768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'186","last_scrub_stamp":"2024-04-25T10:35:08.172667+0000","last_deep_scrub":"22'186","last_deep_scrub_stamp":"2024-04-25T10:35:08.172667+0000","last_clean_scrub_stamp":"2024-04-25T10:35:08.172667+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:35:58.018831+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":177,"num_write_kb":483328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"22'152","reported_seq":178,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:35:13.296179+0000","last_change":"2024-04-25T10:35:13.296209+0000","last_active":"2024-04-25T10:35:13.296179+0000","last_peered":"2024-04-25T10:35:13.296179+0000","last_clean":"2024-04-25T10:35:13.296179+0000","last_became_active":"2024-04-25T10:30:32.440043+0000","last_became_peered":"2024-04-25T10:30:32.440043+0000","last_unstale":"2024-04-25T10:35:13.296179+0000","last_undegraded":"2024-04-25T10:35:13.296179+0000","last_fullsized":"2024-04-25T10:35:13.296179+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'152","last_scrub_stamp":"2024-04-25T10:35:13.296055+0000","last_deep_scrub":"22'152","last_deep_scrub_stamp":"2024-04-25T10:35:13.296055+0000","last_clean_scrub_stamp":"2024-04-25T10:35:13.296055+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:02.781216+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"22'156","reported_seq":182,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:35:06.088300+0000","last_change":"2024-04-25T10:35:06.088373+0000","last_active":"2024-04-25T10:35:06.088300+0000","last_peered":"2024-04-25T10:35:06.088300+0000","last_clean":"2024-04-25T10:35:06.088300+0000","last_became_active":"2024-04-25T10:30:32.434983+0000","last_became_peered":"2024-04-25T10:30:32.434983+0000","last_unstale":"2024-04-25T10:35:06.088300+0000","last_undegraded":"2024-04-25T10:35:06.088300+0000","last_fullsized":"2024-04-25T10:35:06.088300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'156","last_scrub_stamp":"2024-04-25T10:35:06.088179+0000","last_deep_scrub":"22'156","last_deep_scrub_stamp":"2024-04-25T10:35:06.088179+0000","last_clean_scrub_stamp":"2024-04-25T10:35:06.088179+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:49.772852+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"22'189","reported_seq":220,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:35:07.084439+0000","last_change":"2024-04-25T10:35:07.084469+0000","last_active":"2024-04-25T10:35:07.084439+0000","last_peered":"2024-04-25T10:35:07.084439+0000","last_clean":"2024-04-25T10:35:07.084439+0000","last_became_active":"2024-04-25T10:31:13.640582+0000","last_became_peered":"2024-04-25T10:31:13.640582+0000","last_unstale":"2024-04-25T10:35:07.084439+0000","last_undegraded":"2024-04-25T10:35:07.084439+0000","last_fullsized":"2024-04-25T10:35:07.084439+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'189","last_scrub_stamp":"2024-04-25T10:35:07.084311+0000","last_deep_scrub":"22'189","last_deep_scrub_stamp":"2024-04-25T10:35:07.084311+0000","last_clean_scrub_stamp":"2024-04-25T10:35:07.084311+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:57.975856+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"22'142","reported_seq":168,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:35:00.790682+0000","last_change":"2024-04-25T10:35:00.790713+0000","last_active":"2024-04-25T10:35:00.790682+0000","last_peered":"2024-04-25T10:35:00.790682+0000","last_clean":"2024-04-25T10:35:00.790682+0000","last_became_active":"2024-04-25T10:30:32.435908+0000","last_became_peered":"2024-04-25T10:30:32.435908+0000","last_unstale":"2024-04-25T10:35:00.790682+0000","last_undegraded":"2024-04-25T10:35:00.790682+0000","last_fullsized":"2024-04-25T10:35:00.790682+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'142","last_scrub_stamp":"2024-04-25T10:35:00.790568+0000","last_deep_scrub":"22'142","last_deep_scrub_stamp":"2024-04-25T10:35:00.790568+0000","last_clean_scrub_stamp":"2024-04-25T10:35:00.790568+0000","objects_scrubbed":0,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:37.715253+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"22'151","reported_seq":177,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:35:03.133179+0000","last_change":"2024-04-25T10:35:03.133227+0000","last_active":"2024-04-25T10:35:03.133179+0000","last_peered":"2024-04-25T10:35:03.133179+0000","last_clean":"2024-04-25T10:35:03.133179+0000","last_became_active":"2024-04-25T10:30:32.431792+0000","last_became_peered":"2024-04-25T10:30:32.431792+0000","last_unstale":"2024-04-25T10:35:03.133179+0000","last_undegraded":"2024-04-25T10:35:03.133179+0000","last_fullsized":"2024-04-25T10:35:03.133179+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'151","last_scrub_stamp":"2024-04-25T10:35:03.133042+0000","last_deep_scrub":"22'151","last_deep_scrub_stamp":"2024-04-25T10:35:03.133042+0000","last_clean_scrub_stamp":"2024-04-25T10:35:03.133042+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:25:37.925868+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"22'149","reported_seq":176,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:35:00.130347+0000","last_change":"2024-04-25T10:35:00.130347+0000","last_active":"2024-04-25T10:35:00.130347+0000","last_peered":"2024-04-25T10:35:00.130347+0000","last_clean":"2024-04-25T10:35:00.130347+0000","last_became_active":"2024-04-25T10:30:32.433329+0000","last_became_peered":"2024-04-25T10:30:32.433329+0000","last_unstale":"2024-04-25T10:35:00.130347+0000","last_undegraded":"2024-04-25T10:35:00.130347+0000","last_fullsized":"2024-04-25T10:35:00.130347+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'149","last_scrub_stamp":"2024-04-25T10:35:00.130203+0000","last_deep_scrub":"22'149","last_deep_scrub_stamp":"2024-04-25T10:35:00.130203+0000","last_clean_scrub_stamp":"2024-04-25T10:35:00.130203+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:11.353857+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"22'151","reported_seq":178,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:35:01.658015+0000","last_change":"2024-04-25T10:35:01.658015+0000","last_active":"2024-04-25T10:35:01.658015+0000","last_peered":"2024-04-25T10:35:01.658015+0000","last_clean":"2024-04-25T10:35:01.658015+0000","last_became_active":"2024-04-25T10:30:32.422991+0000","last_became_peered":"2024-04-25T10:30:32.422991+0000","last_unstale":"2024-04-25T10:35:01.658015+0000","last_undegraded":"2024-04-25T10:35:01.658015+0000","last_fullsized":"2024-04-25T10:35:01.658015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'151","last_scrub_stamp":"2024-04-25T10:35:01.657800+0000","last_deep_scrub":"22'151","last_deep_scrub_stamp":"2024-04-25T10:35:01.657800+0000","last_clean_scrub_stamp":"2024-04-25T10:35:01.657800+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:49.271953+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"22'147","reported_seq":174,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:35:04.177117+0000","last_change":"2024-04-25T10:35:04.177117+0000","last_active":"2024-04-25T10:35:04.177117+0000","last_peered":"2024-04-25T10:35:04.177117+0000","last_clean":"2024-04-25T10:35:04.177117+0000","last_became_active":"2024-04-25T10:30:32.439191+0000","last_became_peered":"2024-04-25T10:30:32.439191+0000","last_unstale":"2024-04-25T10:35:04.177117+0000","last_undegraded":"2024-04-25T10:35:04.177117+0000","last_fullsized":"2024-04-25T10:35:04.177117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'147","last_scrub_stamp":"2024-04-25T10:35:04.176944+0000","last_deep_scrub":"22'147","last_deep_scrub_stamp":"2024-04-25T10:35:04.176944+0000","last_clean_scrub_stamp":"2024-04-25T10:35:04.176944+0000","objects_scrubbed":0,"log_size":147,"log_dups_size":0,"ondisk_log_size":147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:38.707352+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"22'146","reported_seq":173,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:35:06.149142+0000","last_change":"2024-04-25T10:35:06.149142+0000","last_active":"2024-04-25T10:35:06.149142+0000","last_peered":"2024-04-25T10:35:06.149142+0000","last_clean":"2024-04-25T10:35:06.149142+0000","last_became_active":"2024-04-25T10:30:32.442125+0000","last_became_peered":"2024-04-25T10:30:32.442125+0000","last_unstale":"2024-04-25T10:35:06.149142+0000","last_undegraded":"2024-04-25T10:35:06.149142+0000","last_fullsized":"2024-04-25T10:35:06.149142+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'146","last_scrub_stamp":"2024-04-25T10:35:06.149003+0000","last_deep_scrub":"22'146","last_deep_scrub_stamp":"2024-04-25T10:35:06.149003+0000","last_clean_scrub_stamp":"2024-04-25T10:35:06.149003+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:45.585609+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"22'157","reported_seq":184,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:59.110175+0000","last_change":"2024-04-25T10:34:59.110175+0000","last_active":"2024-04-25T10:34:59.110175+0000","last_peered":"2024-04-25T10:34:59.110175+0000","last_clean":"2024-04-25T10:34:59.110175+0000","last_became_active":"2024-04-25T10:30:32.434492+0000","last_became_peered":"2024-04-25T10:30:32.434492+0000","last_unstale":"2024-04-25T10:34:59.110175+0000","last_undegraded":"2024-04-25T10:34:59.110175+0000","last_fullsized":"2024-04-25T10:34:59.110175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'157","last_scrub_stamp":"2024-04-25T10:34:59.110048+0000","last_deep_scrub":"22'157","last_deep_scrub_stamp":"2024-04-25T10:34:59.110048+0000","last_clean_scrub_stamp":"2024-04-25T10:34:59.110048+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:45.099862+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"22'172","reported_seq":199,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:35:11.280651+0000","last_change":"2024-04-25T10:35:11.280683+0000","last_active":"2024-04-25T10:35:11.280651+0000","last_peered":"2024-04-25T10:35:11.280651+0000","last_clean":"2024-04-25T10:35:11.280651+0000","last_became_active":"2024-04-25T10:30:32.438227+0000","last_became_peered":"2024-04-25T10:30:32.438227+0000","last_unstale":"2024-04-25T10:35:11.280651+0000","last_undegraded":"2024-04-25T10:35:11.280651+0000","last_fullsized":"2024-04-25T10:35:11.280651+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'172","last_scrub_stamp":"2024-04-25T10:35:11.280542+0000","last_deep_scrub":"22'172","last_deep_scrub_stamp":"2024-04-25T10:35:11.280542+0000","last_clean_scrub_stamp":"2024-04-25T10:35:11.280542+0000","objects_scrubbed":0,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:06.652990+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":450560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"22'161","reported_seq":188,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:35:01.018517+0000","last_change":"2024-04-25T10:35:01.018517+0000","last_active":"2024-04-25T10:35:01.018517+0000","last_peered":"2024-04-25T10:35:01.018517+0000","last_clean":"2024-04-25T10:35:01.018517+0000","last_became_active":"2024-04-25T10:30:32.431314+0000","last_became_peered":"2024-04-25T10:30:32.431314+0000","last_unstale":"2024-04-25T10:35:01.018517+0000","last_undegraded":"2024-04-25T10:35:01.018517+0000","last_fullsized":"2024-04-25T10:35:01.018517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'161","last_scrub_stamp":"2024-04-25T10:35:01.018383+0000","last_deep_scrub":"22'161","last_deep_scrub_stamp":"2024-04-25T10:35:01.018383+0000","last_clean_scrub_stamp":"2024-04-25T10:35:01.018383+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:45.582804+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":153,"num_write_kb":417792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"22'137","reported_seq":157,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:35:05.666569+0000","last_change":"2024-04-25T10:35:05.666600+0000","last_active":"2024-04-25T10:35:05.666569+0000","last_peered":"2024-04-25T10:35:05.666569+0000","last_clean":"2024-04-25T10:35:05.666569+0000","last_became_active":"2024-04-25T10:31:13.677164+0000","last_became_peered":"2024-04-25T10:31:13.677164+0000","last_unstale":"2024-04-25T10:35:05.666569+0000","last_undegraded":"2024-04-25T10:35:05.666569+0000","last_fullsized":"2024-04-25T10:35:05.666569+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'137","last_scrub_stamp":"2024-04-25T10:35:05.666456+0000","last_deep_scrub":"22'137","last_deep_scrub_stamp":"2024-04-25T10:35:05.666456+0000","last_clean_scrub_stamp":"2024-04-25T10:35:05.666456+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:19.794700+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2a","version":"22'166","reported_seq":193,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:35:10.085623+0000","last_change":"2024-04-25T10:35:10.085659+0000","last_active":"2024-04-25T10:35:10.085623+0000","last_peered":"2024-04-25T10:35:10.085623+0000","last_clean":"2024-04-25T10:35:10.085623+0000","last_became_active":"2024-04-25T10:30:32.444273+0000","last_became_peered":"2024-04-25T10:30:32.444273+0000","last_unstale":"2024-04-25T10:35:10.085623+0000","last_undegraded":"2024-04-25T10:35:10.085623+0000","last_fullsized":"2024-04-25T10:35:10.085623+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'166","last_scrub_stamp":"2024-04-25T10:35:10.085500+0000","last_deep_scrub":"22'166","last_deep_scrub_stamp":"2024-04-25T10:35:10.085500+0000","last_clean_scrub_stamp":"2024-04-25T10:35:10.085500+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:19.511182+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"22'134","reported_seq":161,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:58.194385+0000","last_change":"2024-04-25T10:34:58.194385+0000","last_active":"2024-04-25T10:34:58.194385+0000","last_peered":"2024-04-25T10:34:58.194385+0000","last_clean":"2024-04-25T10:34:58.194385+0000","last_became_active":"2024-04-25T10:30:32.426223+0000","last_became_peered":"2024-04-25T10:30:32.426223+0000","last_unstale":"2024-04-25T10:34:58.194385+0000","last_undegraded":"2024-04-25T10:34:58.194385+0000","last_fullsized":"2024-04-25T10:34:58.194385+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'134","last_scrub_stamp":"2024-04-25T10:34:58.194218+0000","last_deep_scrub":"22'134","last_deep_scrub_stamp":"2024-04-25T10:34:58.194218+0000","last_clean_scrub_stamp":"2024-04-25T10:34:58.194218+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:36.278147+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"22'180","reported_seq":207,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:35:03.147421+0000","last_change":"2024-04-25T10:35:03.147452+0000","last_active":"2024-04-25T10:35:03.147421+0000","last_peered":"2024-04-25T10:35:03.147421+0000","last_clean":"2024-04-25T10:35:03.147421+0000","last_became_active":"2024-04-25T10:30:32.438801+0000","last_became_peered":"2024-04-25T10:30:32.438801+0000","last_unstale":"2024-04-25T10:35:03.147421+0000","last_undegraded":"2024-04-25T10:35:03.147421+0000","last_fullsized":"2024-04-25T10:35:03.147421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'180","last_scrub_stamp":"2024-04-25T10:35:03.147323+0000","last_deep_scrub":"22'180","last_deep_scrub_stamp":"2024-04-25T10:35:03.147323+0000","last_clean_scrub_stamp":"2024-04-25T10:35:03.147323+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:04.781666+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":174,"num_write_kb":475136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"22'134","reported_seq":161,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:35:00.205860+0000","last_change":"2024-04-25T10:35:00.205891+0000","last_active":"2024-04-25T10:35:00.205860+0000","last_peered":"2024-04-25T10:35:00.205860+0000","last_clean":"2024-04-25T10:35:00.205860+0000","last_became_active":"2024-04-25T10:30:32.427081+0000","last_became_peered":"2024-04-25T10:30:32.427081+0000","last_unstale":"2024-04-25T10:35:00.205860+0000","last_undegraded":"2024-04-25T10:35:00.205860+0000","last_fullsized":"2024-04-25T10:35:00.205860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'134","last_scrub_stamp":"2024-04-25T10:35:00.205757+0000","last_deep_scrub":"22'134","last_deep_scrub_stamp":"2024-04-25T10:35:00.205757+0000","last_clean_scrub_stamp":"2024-04-25T10:35:00.205757+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:15.500924+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"22'198","reported_seq":225,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:35:03.061274+0000","last_change":"2024-04-25T10:35:03.061303+0000","last_active":"2024-04-25T10:35:03.061274+0000","last_peered":"2024-04-25T10:35:03.061274+0000","last_clean":"2024-04-25T10:35:03.061274+0000","last_became_active":"2024-04-25T10:30:32.432121+0000","last_became_peered":"2024-04-25T10:30:32.432121+0000","last_unstale":"2024-04-25T10:35:03.061274+0000","last_undegraded":"2024-04-25T10:35:03.061274+0000","last_fullsized":"2024-04-25T10:35:03.061274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'198","last_scrub_stamp":"2024-04-25T10:35:03.061165+0000","last_deep_scrub":"22'198","last_deep_scrub_stamp":"2024-04-25T10:35:03.061165+0000","last_clean_scrub_stamp":"2024-04-25T10:35:03.061165+0000","objects_scrubbed":0,"log_size":198,"log_dups_size":0,"ondisk_log_size":198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:00.972565+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"22'144","reported_seq":171,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:35:02.622639+0000","last_change":"2024-04-25T10:35:02.622672+0000","last_active":"2024-04-25T10:35:02.622639+0000","last_peered":"2024-04-25T10:35:02.622639+0000","last_clean":"2024-04-25T10:35:02.622639+0000","last_became_active":"2024-04-25T10:30:32.424794+0000","last_became_peered":"2024-04-25T10:30:32.424794+0000","last_unstale":"2024-04-25T10:35:02.622639+0000","last_undegraded":"2024-04-25T10:35:02.622639+0000","last_fullsized":"2024-04-25T10:35:02.622639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'144","last_scrub_stamp":"2024-04-25T10:35:02.622526+0000","last_deep_scrub":"22'144","last_deep_scrub_stamp":"2024-04-25T10:35:02.622526+0000","last_clean_scrub_stamp":"2024-04-25T10:35:02.622526+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:00.854979+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"22'133","reported_seq":160,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:59.185433+0000","last_change":"2024-04-25T10:34:59.185470+0000","last_active":"2024-04-25T10:34:59.185433+0000","last_peered":"2024-04-25T10:34:59.185433+0000","last_clean":"2024-04-25T10:34:59.185433+0000","last_became_active":"2024-04-25T10:30:32.426348+0000","last_became_peered":"2024-04-25T10:30:32.426348+0000","last_unstale":"2024-04-25T10:34:59.185433+0000","last_undegraded":"2024-04-25T10:34:59.185433+0000","last_fullsized":"2024-04-25T10:34:59.185433+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'133","last_scrub_stamp":"2024-04-25T10:34:59.185305+0000","last_deep_scrub":"22'133","last_deep_scrub_stamp":"2024-04-25T10:34:59.185305+0000","last_clean_scrub_stamp":"2024-04-25T10:34:59.185305+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:07.993845+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"22'118","reported_seq":145,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:57.161108+0000","last_change":"2024-04-25T10:34:57.161133+0000","last_active":"2024-04-25T10:34:57.161108+0000","last_peered":"2024-04-25T10:34:57.161108+0000","last_clean":"2024-04-25T10:34:57.161108+0000","last_became_active":"2024-04-25T10:30:32.427676+0000","last_became_peered":"2024-04-25T10:30:32.427676+0000","last_unstale":"2024-04-25T10:34:57.161108+0000","last_undegraded":"2024-04-25T10:34:57.161108+0000","last_fullsized":"2024-04-25T10:34:57.161108+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'118","last_scrub_stamp":"2024-04-25T10:34:57.161023+0000","last_deep_scrub":"22'118","last_deep_scrub_stamp":"2024-04-25T10:34:57.161023+0000","last_clean_scrub_stamp":"2024-04-25T10:34:57.161023+0000","objects_scrubbed":0,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:35:01.534169+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":111,"num_write_kb":303104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"22'131","reported_seq":158,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:35:05.185048+0000","last_change":"2024-04-25T10:35:05.185078+0000","last_active":"2024-04-25T10:35:05.185048+0000","last_peered":"2024-04-25T10:35:05.185048+0000","last_clean":"2024-04-25T10:35:05.185048+0000","last_became_active":"2024-04-25T10:30:32.440815+0000","last_became_peered":"2024-04-25T10:30:32.440815+0000","last_unstale":"2024-04-25T10:35:05.185048+0000","last_undegraded":"2024-04-25T10:35:05.185048+0000","last_fullsized":"2024-04-25T10:35:05.185048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'131","last_scrub_stamp":"2024-04-25T10:35:05.184948+0000","last_deep_scrub":"22'131","last_deep_scrub_stamp":"2024-04-25T10:35:05.184948+0000","last_clean_scrub_stamp":"2024-04-25T10:35:05.184948+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:52.673449+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"22'135","reported_seq":162,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:58.151604+0000","last_change":"2024-04-25T10:34:58.151638+0000","last_active":"2024-04-25T10:34:58.151604+0000","last_peered":"2024-04-25T10:34:58.151604+0000","last_clean":"2024-04-25T10:34:58.151604+0000","last_became_active":"2024-04-25T10:30:32.430615+0000","last_became_peered":"2024-04-25T10:30:32.430615+0000","last_unstale":"2024-04-25T10:34:58.151604+0000","last_undegraded":"2024-04-25T10:34:58.151604+0000","last_fullsized":"2024-04-25T10:34:58.151604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'135","last_scrub_stamp":"2024-04-25T10:34:58.151481+0000","last_deep_scrub":"22'135","last_deep_scrub_stamp":"2024-04-25T10:34:58.151481+0000","last_clean_scrub_stamp":"2024-04-25T10:34:58.151481+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:47.193327+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"22'150","reported_seq":177,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:35:05.048061+0000","last_change":"2024-04-25T10:35:05.048128+0000","last_active":"2024-04-25T10:35:05.048061+0000","last_peered":"2024-04-25T10:35:05.048061+0000","last_clean":"2024-04-25T10:35:05.048061+0000","last_became_active":"2024-04-25T10:30:32.435055+0000","last_became_peered":"2024-04-25T10:30:32.435055+0000","last_unstale":"2024-04-25T10:35:05.048061+0000","last_undegraded":"2024-04-25T10:35:05.048061+0000","last_fullsized":"2024-04-25T10:35:05.048061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'150","last_scrub_stamp":"2024-04-25T10:35:05.047911+0000","last_deep_scrub":"22'150","last_deep_scrub_stamp":"2024-04-25T10:35:05.047911+0000","last_clean_scrub_stamp":"2024-04-25T10:35:05.047911+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:58.564848+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"22'130","reported_seq":156,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:57.751211+0000","last_change":"2024-04-25T10:34:57.751256+0000","last_active":"2024-04-25T10:34:57.751211+0000","last_peered":"2024-04-25T10:34:57.751211+0000","last_clean":"2024-04-25T10:34:57.751211+0000","last_became_active":"2024-04-25T10:30:32.421766+0000","last_became_peered":"2024-04-25T10:30:32.421766+0000","last_unstale":"2024-04-25T10:34:57.751211+0000","last_undegraded":"2024-04-25T10:34:57.751211+0000","last_fullsized":"2024-04-25T10:34:57.751211+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'130","last_scrub_stamp":"2024-04-25T10:34:57.751102+0000","last_deep_scrub":"22'130","last_deep_scrub_stamp":"2024-04-25T10:34:57.751102+0000","last_clean_scrub_stamp":"2024-04-25T10:34:57.751102+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:24:25.482393+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'8","reported_seq":36,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:55.214035+0000","last_change":"2024-04-25T10:34:55.214084+0000","last_active":"2024-04-25T10:34:55.214035+0000","last_peered":"2024-04-25T10:34:55.214035+0000","last_clean":"2024-04-25T10:34:55.214035+0000","last_became_active":"2024-04-25T10:30:31.417451+0000","last_became_peered":"2024-04-25T10:30:31.417451+0000","last_unstale":"2024-04-25T10:34:55.214035+0000","last_undegraded":"2024-04-25T10:34:55.214035+0000","last_fullsized":"2024-04-25T10:34:55.214035+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'8","last_scrub_stamp":"2024-04-25T10:34:55.213920+0000","last_deep_scrub":"22'8","last_deep_scrub_stamp":"2024-04-25T10:34:55.213920+0000","last_clean_scrub_stamp":"2024-04-25T10:34:55.213920+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:11.238095+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1204,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"22'137","reported_seq":163,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:56.124373+0000","last_change":"2024-04-25T10:34:56.124422+0000","last_active":"2024-04-25T10:34:56.124373+0000","last_peered":"2024-04-25T10:34:56.124373+0000","last_clean":"2024-04-25T10:34:56.124373+0000","last_became_active":"2024-04-25T10:30:32.426616+0000","last_became_peered":"2024-04-25T10:30:32.426616+0000","last_unstale":"2024-04-25T10:34:56.124373+0000","last_undegraded":"2024-04-25T10:34:56.124373+0000","last_fullsized":"2024-04-25T10:34:56.124373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'137","last_scrub_stamp":"2024-04-25T10:34:56.124247+0000","last_deep_scrub":"22'137","last_deep_scrub_stamp":"2024-04-25T10:34:56.124247+0000","last_clean_scrub_stamp":"2024-04-25T10:34:56.124247+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:45.755314+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"22'10","reported_seq":38,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:58.028393+0000","last_change":"2024-04-25T10:34:58.028439+0000","last_active":"2024-04-25T10:34:58.028393+0000","last_peered":"2024-04-25T10:34:58.028393+0000","last_clean":"2024-04-25T10:34:58.028393+0000","last_became_active":"2024-04-25T10:30:31.415112+0000","last_became_peered":"2024-04-25T10:30:31.415112+0000","last_unstale":"2024-04-25T10:34:58.028393+0000","last_undegraded":"2024-04-25T10:34:58.028393+0000","last_fullsized":"2024-04-25T10:34:58.028393+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'10","last_scrub_stamp":"2024-04-25T10:34:58.028291+0000","last_deep_scrub":"22'10","last_deep_scrub_stamp":"2024-04-25T10:34:58.028291+0000","last_clean_scrub_stamp":"2024-04-25T10:34:58.028291+0000","objects_scrubbed":3,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:30.161497+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"22'5","reported_seq":33,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:56.167003+0000","last_change":"2024-04-25T10:34:56.167035+0000","last_active":"2024-04-25T10:34:56.167003+0000","last_peered":"2024-04-25T10:34:56.167003+0000","last_clean":"2024-04-25T10:34:56.167003+0000","last_became_active":"2024-04-25T10:30:31.417716+0000","last_became_peered":"2024-04-25T10:30:31.417716+0000","last_unstale":"2024-04-25T10:34:56.167003+0000","last_undegraded":"2024-04-25T10:34:56.167003+0000","last_fullsized":"2024-04-25T10:34:56.167003+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'5","last_scrub_stamp":"2024-04-25T10:34:56.166923+0000","last_deep_scrub":"22'5","last_deep_scrub_stamp":"2024-04-25T10:34:56.166923+0000","last_clean_scrub_stamp":"2024-04-25T10:34:56.166923+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:07.796547+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"22'170","reported_seq":196,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:35:02.162160+0000","last_change":"2024-04-25T10:35:02.162208+0000","last_active":"2024-04-25T10:35:02.162160+0000","last_peered":"2024-04-25T10:35:02.162160+0000","last_clean":"2024-04-25T10:35:02.162160+0000","last_became_active":"2024-04-25T10:30:32.427700+0000","last_became_peered":"2024-04-25T10:30:32.427700+0000","last_unstale":"2024-04-25T10:35:02.162160+0000","last_undegraded":"2024-04-25T10:35:02.162160+0000","last_fullsized":"2024-04-25T10:35:02.162160+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'170","last_scrub_stamp":"2024-04-25T10:35:02.162023+0000","last_deep_scrub":"22'170","last_deep_scrub_stamp":"2024-04-25T10:35:02.162023+0000","last_clean_scrub_stamp":"2024-04-25T10:35:02.162023+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:01.158042+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":162,"num_write_kb":442368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"22'4","reported_seq":32,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:59.211040+0000","last_change":"2024-04-25T10:34:59.211074+0000","last_active":"2024-04-25T10:34:59.211040+0000","last_peered":"2024-04-25T10:34:59.211040+0000","last_clean":"2024-04-25T10:34:59.211040+0000","last_became_active":"2024-04-25T10:30:31.414542+0000","last_became_peered":"2024-04-25T10:30:31.414542+0000","last_unstale":"2024-04-25T10:34:59.211040+0000","last_undegraded":"2024-04-25T10:34:59.211040+0000","last_fullsized":"2024-04-25T10:34:59.211040+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'4","last_scrub_stamp":"2024-04-25T10:34:59.210936+0000","last_deep_scrub":"22'4","last_deep_scrub_stamp":"2024-04-25T10:34:59.210936+0000","last_clean_scrub_stamp":"2024-04-25T10:34:59.210936+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:21:34.163160+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"22'157","reported_seq":183,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:35:03.663382+0000","last_change":"2024-04-25T10:35:03.663434+0000","last_active":"2024-04-25T10:35:03.663382+0000","last_peered":"2024-04-25T10:35:03.663382+0000","last_clean":"2024-04-25T10:35:03.663382+0000","last_became_active":"2024-04-25T10:30:32.424849+0000","last_became_peered":"2024-04-25T10:30:32.424849+0000","last_unstale":"2024-04-25T10:35:03.663382+0000","last_undegraded":"2024-04-25T10:35:03.663382+0000","last_fullsized":"2024-04-25T10:35:03.663382+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'157","last_scrub_stamp":"2024-04-25T10:35:03.663228+0000","last_deep_scrub":"22'157","last_deep_scrub_stamp":"2024-04-25T10:35:03.663228+0000","last_clean_scrub_stamp":"2024-04-25T10:35:03.663228+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:33.926676+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"22'9","reported_seq":37,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:56.026675+0000","last_change":"2024-04-25T10:34:56.026714+0000","last_active":"2024-04-25T10:34:56.026675+0000","last_peered":"2024-04-25T10:34:56.026675+0000","last_clean":"2024-04-25T10:34:56.026675+0000","last_became_active":"2024-04-25T10:30:31.407376+0000","last_became_peered":"2024-04-25T10:30:31.407376+0000","last_unstale":"2024-04-25T10:34:56.026675+0000","last_undegraded":"2024-04-25T10:34:56.026675+0000","last_fullsized":"2024-04-25T10:34:56.026675+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'9","last_scrub_stamp":"2024-04-25T10:34:56.026572+0000","last_deep_scrub":"22'9","last_deep_scrub_stamp":"2024-04-25T10:34:56.026572+0000","last_clean_scrub_stamp":"2024-04-25T10:34:56.026572+0000","objects_scrubbed":3,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:03.519624+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"22'229","reported_seq":255,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:35:09.108038+0000","last_change":"2024-04-25T10:35:09.108070+0000","last_active":"2024-04-25T10:35:09.108038+0000","last_peered":"2024-04-25T10:35:09.108038+0000","last_clean":"2024-04-25T10:35:09.108038+0000","last_became_active":"2024-04-25T10:30:32.444250+0000","last_became_peered":"2024-04-25T10:30:32.444250+0000","last_unstale":"2024-04-25T10:35:09.108038+0000","last_undegraded":"2024-04-25T10:35:09.108038+0000","last_fullsized":"2024-04-25T10:35:09.108038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'229","last_scrub_stamp":"2024-04-25T10:35:09.107919+0000","last_deep_scrub":"22'229","last_deep_scrub_stamp":"2024-04-25T10:35:09.107919+0000","last_clean_scrub_stamp":"2024-04-25T10:35:09.107919+0000","objects_scrubbed":0,"log_size":229,"log_dups_size":0,"ondisk_log_size":229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:07:08.195261+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":219,"num_write_kb":598016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"22'6","reported_seq":34,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:56.150196+0000","last_change":"2024-04-25T10:34:56.150235+0000","last_active":"2024-04-25T10:34:56.150196+0000","last_peered":"2024-04-25T10:34:56.150196+0000","last_clean":"2024-04-25T10:34:56.150196+0000","last_became_active":"2024-04-25T10:30:31.404669+0000","last_became_peered":"2024-04-25T10:30:31.404669+0000","last_unstale":"2024-04-25T10:34:56.150196+0000","last_undegraded":"2024-04-25T10:34:56.150196+0000","last_fullsized":"2024-04-25T10:34:56.150196+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'6","last_scrub_stamp":"2024-04-25T10:34:56.150090+0000","last_deep_scrub":"22'6","last_deep_scrub_stamp":"2024-04-25T10:34:56.150090+0000","last_clean_scrub_stamp":"2024-04-25T10:34:56.150090+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:24.868978+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"22'141","reported_seq":167,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:35:02.145809+0000","last_change":"2024-04-25T10:35:02.145841+0000","last_active":"2024-04-25T10:35:02.145809+0000","last_peered":"2024-04-25T10:35:02.145809+0000","last_clean":"2024-04-25T10:35:02.145809+0000","last_became_active":"2024-04-25T10:30:32.435471+0000","last_became_peered":"2024-04-25T10:30:32.435471+0000","last_unstale":"2024-04-25T10:35:02.145809+0000","last_undegraded":"2024-04-25T10:35:02.145809+0000","last_fullsized":"2024-04-25T10:35:02.145809+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'141","last_scrub_stamp":"2024-04-25T10:35:02.145682+0000","last_deep_scrub":"22'141","last_deep_scrub_stamp":"2024-04-25T10:35:02.145682+0000","last_clean_scrub_stamp":"2024-04-25T10:35:02.145682+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:52.634090+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"22'14","reported_seq":42,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:53.648907+0000","last_change":"2024-04-25T10:34:53.648938+0000","last_active":"2024-04-25T10:34:53.648907+0000","last_peered":"2024-04-25T10:34:53.648907+0000","last_clean":"2024-04-25T10:34:53.648907+0000","last_became_active":"2024-04-25T10:30:31.403167+0000","last_became_peered":"2024-04-25T10:30:31.403167+0000","last_unstale":"2024-04-25T10:34:53.648907+0000","last_undegraded":"2024-04-25T10:34:53.648907+0000","last_fullsized":"2024-04-25T10:34:53.648907+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'14","last_scrub_stamp":"2024-04-25T10:34:53.648827+0000","last_deep_scrub":"22'14","last_deep_scrub_stamp":"2024-04-25T10:34:53.648827+0000","last_clean_scrub_stamp":"2024-04-25T10:34:53.648827+0000","objects_scrubbed":4,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:55.835283+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"22'149","reported_seq":176,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:58.717703+0000","last_change":"2024-04-25T10:34:58.717703+0000","last_active":"2024-04-25T10:34:58.717703+0000","last_peered":"2024-04-25T10:34:58.717703+0000","last_clean":"2024-04-25T10:34:58.717703+0000","last_became_active":"2024-04-25T10:30:32.423780+0000","last_became_peered":"2024-04-25T10:30:32.423780+0000","last_unstale":"2024-04-25T10:34:58.717703+0000","last_undegraded":"2024-04-25T10:34:58.717703+0000","last_fullsized":"2024-04-25T10:34:58.717703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'149","last_scrub_stamp":"2024-04-25T10:34:58.717530+0000","last_deep_scrub":"22'149","last_deep_scrub_stamp":"2024-04-25T10:34:58.717530+0000","last_clean_scrub_stamp":"2024-04-25T10:34:58.717530+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:50.116068+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"22'99","reported_seq":128,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:59.980400+0000","last_change":"2024-04-25T10:34:59.980400+0000","last_active":"2024-04-25T10:34:59.980400+0000","last_peered":"2024-04-25T10:34:59.980400+0000","last_clean":"2024-04-25T10:34:59.980400+0000","last_became_active":"2024-04-25T10:30:31.417824+0000","last_became_peered":"2024-04-25T10:30:31.417824+0000","last_unstale":"2024-04-25T10:34:59.980400+0000","last_undegraded":"2024-04-25T10:34:59.980400+0000","last_fullsized":"2024-04-25T10:34:59.980400+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'99","last_scrub_stamp":"2024-04-25T10:34:59.980252+0000","last_deep_scrub":"22'99","last_deep_scrub_stamp":"2024-04-25T10:34:59.980252+0000","last_clean_scrub_stamp":"2024-04-25T10:34:59.980252+0000","objects_scrubbed":6,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:17:38.411549+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":201790,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":96,"num_write_kb":233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"22'150","reported_seq":176,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:58.256540+0000","last_change":"2024-04-25T10:34:58.256622+0000","last_active":"2024-04-25T10:34:58.256540+0000","last_peered":"2024-04-25T10:34:58.256540+0000","last_clean":"2024-04-25T10:34:58.256540+0000","last_became_active":"2024-04-25T10:30:32.429622+0000","last_became_peered":"2024-04-25T10:30:32.429622+0000","last_unstale":"2024-04-25T10:34:58.256540+0000","last_undegraded":"2024-04-25T10:34:58.256540+0000","last_fullsized":"2024-04-25T10:34:58.256540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'150","last_scrub_stamp":"2024-04-25T10:34:58.256423+0000","last_deep_scrub":"22'150","last_deep_scrub_stamp":"2024-04-25T10:34:58.256423+0000","last_clean_scrub_stamp":"2024-04-25T10:34:58.256423+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:34.837334+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'15","reported_seq":44,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:53.171761+0000","last_change":"2024-04-25T10:34:53.171761+0000","last_active":"2024-04-25T10:34:53.171761+0000","last_peered":"2024-04-25T10:34:53.171761+0000","last_clean":"2024-04-25T10:34:53.171761+0000","last_became_active":"2024-04-25T10:30:31.414422+0000","last_became_peered":"2024-04-25T10:30:31.414422+0000","last_unstale":"2024-04-25T10:34:53.171761+0000","last_undegraded":"2024-04-25T10:34:53.171761+0000","last_fullsized":"2024-04-25T10:34:53.171761+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'15","last_scrub_stamp":"2024-04-25T10:34:53.171619+0000","last_deep_scrub":"22'15","last_deep_scrub_stamp":"2024-04-25T10:34:53.171619+0000","last_clean_scrub_stamp":"2024-04-25T10:34:53.171619+0000","objects_scrubbed":6,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:29.872635+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"22'157","reported_seq":184,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:59.176953+0000","last_change":"2024-04-25T10:34:59.176953+0000","last_active":"2024-04-25T10:34:59.176953+0000","last_peered":"2024-04-25T10:34:59.176953+0000","last_clean":"2024-04-25T10:34:59.176953+0000","last_became_active":"2024-04-25T10:30:32.431743+0000","last_became_peered":"2024-04-25T10:30:32.431743+0000","last_unstale":"2024-04-25T10:34:59.176953+0000","last_undegraded":"2024-04-25T10:34:59.176953+0000","last_fullsized":"2024-04-25T10:34:59.176953+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'157","last_scrub_stamp":"2024-04-25T10:34:59.176800+0000","last_deep_scrub":"22'157","last_deep_scrub_stamp":"2024-04-25T10:34:59.176800+0000","last_clean_scrub_stamp":"2024-04-25T10:34:59.176800+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:42.231131+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"22'8","reported_seq":37,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:35:00.179816+0000","last_change":"2024-04-25T10:35:00.179816+0000","last_active":"2024-04-25T10:35:00.179816+0000","last_peered":"2024-04-25T10:35:00.179816+0000","last_clean":"2024-04-25T10:35:00.179816+0000","last_became_active":"2024-04-25T10:30:31.419135+0000","last_became_peered":"2024-04-25T10:30:31.419135+0000","last_unstale":"2024-04-25T10:35:00.179816+0000","last_undegraded":"2024-04-25T10:35:00.179816+0000","last_fullsized":"2024-04-25T10:35:00.179816+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'8","last_scrub_stamp":"2024-04-25T10:35:00.179658+0000","last_deep_scrub":"22'8","last_deep_scrub_stamp":"2024-04-25T10:35:00.179658+0000","last_clean_scrub_stamp":"2024-04-25T10:35:00.179658+0000","objects_scrubbed":3,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:00.714076+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"22'171","reported_seq":198,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:35:08.338712+0000","last_change":"2024-04-25T10:35:08.338712+0000","last_active":"2024-04-25T10:35:08.338712+0000","last_peered":"2024-04-25T10:35:08.338712+0000","last_clean":"2024-04-25T10:35:08.338712+0000","last_became_active":"2024-04-25T10:30:32.433623+0000","last_became_peered":"2024-04-25T10:30:32.433623+0000","last_unstale":"2024-04-25T10:35:08.338712+0000","last_undegraded":"2024-04-25T10:35:08.338712+0000","last_fullsized":"2024-04-25T10:35:08.338712+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'171","last_scrub_stamp":"2024-04-25T10:35:08.338572+0000","last_deep_scrub":"22'171","last_deep_scrub_stamp":"2024-04-25T10:35:08.338572+0000","last_clean_scrub_stamp":"2024-04-25T10:35:08.338572+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:36.560476+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"22'9","reported_seq":38,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:58.989887+0000","last_change":"2024-04-25T10:34:58.989887+0000","last_active":"2024-04-25T10:34:58.989887+0000","last_peered":"2024-04-25T10:34:58.989887+0000","last_clean":"2024-04-25T10:34:58.989887+0000","last_became_active":"2024-04-25T10:30:31.415706+0000","last_became_peered":"2024-04-25T10:30:31.415706+0000","last_unstale":"2024-04-25T10:34:58.989887+0000","last_undegraded":"2024-04-25T10:34:58.989887+0000","last_fullsized":"2024-04-25T10:34:58.989887+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'9","last_scrub_stamp":"2024-04-25T10:34:58.989732+0000","last_deep_scrub":"22'9","last_deep_scrub_stamp":"2024-04-25T10:34:58.989732+0000","last_clean_scrub_stamp":"2024-04-25T10:34:58.989732+0000","objects_scrubbed":3,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:49.525360+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"22'165","reported_seq":192,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:35:04.035058+0000","last_change":"2024-04-25T10:35:04.035058+0000","last_active":"2024-04-25T10:35:04.035058+0000","last_peered":"2024-04-25T10:35:04.035058+0000","last_clean":"2024-04-25T10:35:04.035058+0000","last_became_active":"2024-04-25T10:30:32.434737+0000","last_became_peered":"2024-04-25T10:30:32.434737+0000","last_unstale":"2024-04-25T10:35:04.035058+0000","last_undegraded":"2024-04-25T10:35:04.035058+0000","last_fullsized":"2024-04-25T10:35:04.035058+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'165","last_scrub_stamp":"2024-04-25T10:35:04.034920+0000","last_deep_scrub":"22'165","last_deep_scrub_stamp":"2024-04-25T10:35:04.034920+0000","last_clean_scrub_stamp":"2024-04-25T10:35:04.034920+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:25.640834+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159,"num_write_kb":434176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"22'26","reported_seq":55,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:55.155844+0000","last_change":"2024-04-25T10:34:55.155889+0000","last_active":"2024-04-25T10:34:55.155844+0000","last_peered":"2024-04-25T10:34:55.155844+0000","last_clean":"2024-04-25T10:34:55.155844+0000","last_became_active":"2024-04-25T10:30:31.404775+0000","last_became_peered":"2024-04-25T10:30:31.404775+0000","last_unstale":"2024-04-25T10:34:55.155844+0000","last_undegraded":"2024-04-25T10:34:55.155844+0000","last_fullsized":"2024-04-25T10:34:55.155844+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'26","last_scrub_stamp":"2024-04-25T10:34:55.155759+0000","last_deep_scrub":"22'26","last_deep_scrub_stamp":"2024-04-25T10:34:55.155759+0000","last_clean_scrub_stamp":"2024-04-25T10:34:55.155759+0000","objects_scrubbed":8,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:20.437218+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"22'189","reported_seq":216,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:57.285760+0000","last_change":"2024-04-25T10:34:57.285760+0000","last_active":"2024-04-25T10:34:57.285760+0000","last_peered":"2024-04-25T10:34:57.285760+0000","last_clean":"2024-04-25T10:34:57.285760+0000","last_became_active":"2024-04-25T10:30:32.431651+0000","last_became_peered":"2024-04-25T10:30:32.431651+0000","last_unstale":"2024-04-25T10:34:57.285760+0000","last_undegraded":"2024-04-25T10:34:57.285760+0000","last_fullsized":"2024-04-25T10:34:57.285760+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'189","last_scrub_stamp":"2024-04-25T10:34:57.285591+0000","last_deep_scrub":"22'189","last_deep_scrub_stamp":"2024-04-25T10:34:57.285591+0000","last_clean_scrub_stamp":"2024-04-25T10:34:57.285591+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:11.584859+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'4","reported_seq":33,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:55.709216+0000","last_change":"2024-04-25T10:34:55.709216+0000","last_active":"2024-04-25T10:34:55.709216+0000","last_peered":"2024-04-25T10:34:55.709216+0000","last_clean":"2024-04-25T10:34:55.709216+0000","last_became_active":"2024-04-25T10:30:31.419902+0000","last_became_peered":"2024-04-25T10:30:31.419902+0000","last_unstale":"2024-04-25T10:34:55.709216+0000","last_undegraded":"2024-04-25T10:34:55.709216+0000","last_fullsized":"2024-04-25T10:34:55.709216+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'4","last_scrub_stamp":"2024-04-25T10:34:55.709063+0000","last_deep_scrub":"22'4","last_deep_scrub_stamp":"2024-04-25T10:34:55.709063+0000","last_clean_scrub_stamp":"2024-04-25T10:34:55.709063+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:02.208739+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"22'146","reported_seq":173,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:35:07.321049+0000","last_change":"2024-04-25T10:35:07.321082+0000","last_active":"2024-04-25T10:35:07.321049+0000","last_peered":"2024-04-25T10:35:07.321049+0000","last_clean":"2024-04-25T10:35:07.321049+0000","last_became_active":"2024-04-25T10:30:32.432572+0000","last_became_peered":"2024-04-25T10:30:32.432572+0000","last_unstale":"2024-04-25T10:35:07.321049+0000","last_undegraded":"2024-04-25T10:35:07.321049+0000","last_fullsized":"2024-04-25T10:35:07.321049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'146","last_scrub_stamp":"2024-04-25T10:35:07.320941+0000","last_deep_scrub":"22'146","last_deep_scrub_stamp":"2024-04-25T10:35:07.320941+0000","last_clean_scrub_stamp":"2024-04-25T10:35:07.320941+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:30.535306+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'7","reported_seq":36,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:57.169279+0000","last_change":"2024-04-25T10:34:57.169279+0000","last_active":"2024-04-25T10:34:57.169279+0000","last_peered":"2024-04-25T10:34:57.169279+0000","last_clean":"2024-04-25T10:34:57.169279+0000","last_became_active":"2024-04-25T10:30:31.418978+0000","last_became_peered":"2024-04-25T10:30:31.418978+0000","last_unstale":"2024-04-25T10:34:57.169279+0000","last_undegraded":"2024-04-25T10:34:57.169279+0000","last_fullsized":"2024-04-25T10:34:57.169279+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'7","last_scrub_stamp":"2024-04-25T10:34:57.169146+0000","last_deep_scrub":"22'7","last_deep_scrub_stamp":"2024-04-25T10:34:57.169146+0000","last_clean_scrub_stamp":"2024-04-25T10:34:57.169146+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:20.649676+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"22'138","reported_seq":165,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:35:09.313851+0000","last_change":"2024-04-25T10:35:09.313886+0000","last_active":"2024-04-25T10:35:09.313851+0000","last_peered":"2024-04-25T10:35:09.313851+0000","last_clean":"2024-04-25T10:35:09.313851+0000","last_became_active":"2024-04-25T10:30:32.434701+0000","last_became_peered":"2024-04-25T10:30:32.434701+0000","last_unstale":"2024-04-25T10:35:09.313851+0000","last_undegraded":"2024-04-25T10:35:09.313851+0000","last_fullsized":"2024-04-25T10:35:09.313851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'138","last_scrub_stamp":"2024-04-25T10:35:09.313716+0000","last_deep_scrub":"22'138","last_deep_scrub_stamp":"2024-04-25T10:35:09.313716+0000","last_clean_scrub_stamp":"2024-04-25T10:35:09.313716+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:33.120238+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'4","reported_seq":33,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:57.612131+0000","last_change":"2024-04-25T10:34:57.612131+0000","last_active":"2024-04-25T10:34:57.612131+0000","last_peered":"2024-04-25T10:34:57.612131+0000","last_clean":"2024-04-25T10:34:57.612131+0000","last_became_active":"2024-04-25T10:30:31.415707+0000","last_became_peered":"2024-04-25T10:30:31.415707+0000","last_unstale":"2024-04-25T10:34:57.612131+0000","last_undegraded":"2024-04-25T10:34:57.612131+0000","last_fullsized":"2024-04-25T10:34:57.612131+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'4","last_scrub_stamp":"2024-04-25T10:34:57.611964+0000","last_deep_scrub":"22'4","last_deep_scrub_stamp":"2024-04-25T10:34:57.611964+0000","last_clean_scrub_stamp":"2024-04-25T10:34:57.611964+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:48:15.653553+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"22'144","reported_seq":171,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:35:08.111371+0000","last_change":"2024-04-25T10:35:08.111402+0000","last_active":"2024-04-25T10:35:08.111371+0000","last_peered":"2024-04-25T10:35:08.111371+0000","last_clean":"2024-04-25T10:35:08.111371+0000","last_became_active":"2024-04-25T10:30:32.442656+0000","last_became_peered":"2024-04-25T10:30:32.442656+0000","last_unstale":"2024-04-25T10:35:08.111371+0000","last_undegraded":"2024-04-25T10:35:08.111371+0000","last_fullsized":"2024-04-25T10:35:08.111371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'144","last_scrub_stamp":"2024-04-25T10:35:08.111265+0000","last_deep_scrub":"22'144","last_deep_scrub_stamp":"2024-04-25T10:35:08.111265+0000","last_clean_scrub_stamp":"2024-04-25T10:35:08.111265+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:25.876988+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"22'9","reported_seq":38,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:54.115307+0000","last_change":"2024-04-25T10:34:54.115307+0000","last_active":"2024-04-25T10:34:54.115307+0000","last_peered":"2024-04-25T10:34:54.115307+0000","last_clean":"2024-04-25T10:34:54.115307+0000","last_became_active":"2024-04-25T10:30:31.404392+0000","last_became_peered":"2024-04-25T10:30:31.404392+0000","last_unstale":"2024-04-25T10:34:54.115307+0000","last_undegraded":"2024-04-25T10:34:54.115307+0000","last_fullsized":"2024-04-25T10:34:54.115307+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'9","last_scrub_stamp":"2024-04-25T10:34:54.115185+0000","last_deep_scrub":"22'9","last_deep_scrub_stamp":"2024-04-25T10:34:54.115185+0000","last_clean_scrub_stamp":"2024-04-25T10:34:54.115185+0000","objects_scrubbed":3,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:05.747122+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1752,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"22'151","reported_seq":178,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:35:14.343206+0000","last_change":"2024-04-25T10:35:14.343239+0000","last_active":"2024-04-25T10:35:14.343206+0000","last_peered":"2024-04-25T10:35:14.343206+0000","last_clean":"2024-04-25T10:35:14.343206+0000","last_became_active":"2024-04-25T10:30:32.440011+0000","last_became_peered":"2024-04-25T10:30:32.440011+0000","last_unstale":"2024-04-25T10:35:14.343206+0000","last_undegraded":"2024-04-25T10:35:14.343206+0000","last_fullsized":"2024-04-25T10:35:14.343206+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'151","last_scrub_stamp":"2024-04-25T10:35:14.343082+0000","last_deep_scrub":"22'151","last_deep_scrub_stamp":"2024-04-25T10:35:14.343082+0000","last_clean_scrub_stamp":"2024-04-25T10:35:14.343082+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:10.267025+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'74","reported_seq":127,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:59.249692+0000","last_change":"2024-04-25T10:34:59.249737+0000","last_active":"2024-04-25T10:34:59.249692+0000","last_peered":"2024-04-25T10:34:59.249692+0000","last_clean":"2024-04-25T10:34:59.249692+0000","last_became_active":"2024-04-25T10:30:21.427958+0000","last_became_peered":"2024-04-25T10:30:21.427958+0000","last_unstale":"2024-04-25T10:34:59.249692+0000","last_undegraded":"2024-04-25T10:34:59.249692+0000","last_fullsized":"2024-04-25T10:34:59.249692+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'74","last_scrub_stamp":"2024-04-25T10:34:59.249587+0000","last_deep_scrub":"22'74","last_deep_scrub_stamp":"2024-04-25T10:34:59.249587+0000","last_clean_scrub_stamp":"2024-04-25T10:34:59.249587+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:19.886370+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"22'11","reported_seq":40,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:57.143288+0000","last_change":"2024-04-25T10:34:57.143344+0000","last_active":"2024-04-25T10:34:57.143288+0000","last_peered":"2024-04-25T10:34:57.143288+0000","last_clean":"2024-04-25T10:34:57.143288+0000","last_became_active":"2024-04-25T10:30:31.404549+0000","last_became_peered":"2024-04-25T10:30:31.404549+0000","last_unstale":"2024-04-25T10:34:57.143288+0000","last_undegraded":"2024-04-25T10:34:57.143288+0000","last_fullsized":"2024-04-25T10:34:57.143288+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'11","last_scrub_stamp":"2024-04-25T10:34:57.143168+0000","last_deep_scrub":"22'11","last_deep_scrub_stamp":"2024-04-25T10:34:57.143168+0000","last_clean_scrub_stamp":"2024-04-25T10:34:57.143168+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:43.816298+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"22'158","reported_seq":185,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:35:05.140532+0000","last_change":"2024-04-25T10:35:05.140563+0000","last_active":"2024-04-25T10:35:05.140532+0000","last_peered":"2024-04-25T10:35:05.140532+0000","last_clean":"2024-04-25T10:35:05.140532+0000","last_became_active":"2024-04-25T10:30:32.433830+0000","last_became_peered":"2024-04-25T10:30:32.433830+0000","last_unstale":"2024-04-25T10:35:05.140532+0000","last_undegraded":"2024-04-25T10:35:05.140532+0000","last_fullsized":"2024-04-25T10:35:05.140532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'158","last_scrub_stamp":"2024-04-25T10:35:05.140412+0000","last_deep_scrub":"22'158","last_deep_scrub_stamp":"2024-04-25T10:35:05.140412+0000","last_clean_scrub_stamp":"2024-04-25T10:35:05.140412+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:51.224214+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":409600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"22'5","reported_seq":34,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:35:00.215306+0000","last_change":"2024-04-25T10:35:00.215341+0000","last_active":"2024-04-25T10:35:00.215306+0000","last_peered":"2024-04-25T10:35:00.215306+0000","last_clean":"2024-04-25T10:35:00.215306+0000","last_became_active":"2024-04-25T10:30:31.415675+0000","last_became_peered":"2024-04-25T10:30:31.415675+0000","last_unstale":"2024-04-25T10:35:00.215306+0000","last_undegraded":"2024-04-25T10:35:00.215306+0000","last_fullsized":"2024-04-25T10:35:00.215306+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'5","last_scrub_stamp":"2024-04-25T10:35:00.215208+0000","last_deep_scrub":"22'5","last_deep_scrub_stamp":"2024-04-25T10:35:00.215208+0000","last_clean_scrub_stamp":"2024-04-25T10:35:00.215208+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:18.465424+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"22'131","reported_seq":158,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:55.120078+0000","last_change":"2024-04-25T10:34:55.120104+0000","last_active":"2024-04-25T10:34:55.120078+0000","last_peered":"2024-04-25T10:34:55.120078+0000","last_clean":"2024-04-25T10:34:55.120078+0000","last_became_active":"2024-04-25T10:30:32.424303+0000","last_became_peered":"2024-04-25T10:30:32.424303+0000","last_unstale":"2024-04-25T10:34:55.120078+0000","last_undegraded":"2024-04-25T10:34:55.120078+0000","last_fullsized":"2024-04-25T10:34:55.120078+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'131","last_scrub_stamp":"2024-04-25T10:34:55.119995+0000","last_deep_scrub":"22'131","last_deep_scrub_stamp":"2024-04-25T10:34:55.119995+0000","last_clean_scrub_stamp":"2024-04-25T10:34:55.119995+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:49.222993+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"22'15","reported_seq":49,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:56.124793+0000","last_change":"2024-04-25T10:34:56.124856+0000","last_active":"2024-04-25T10:34:56.124793+0000","last_peered":"2024-04-25T10:34:56.124793+0000","last_clean":"2024-04-25T10:34:56.124793+0000","last_became_active":"2024-04-25T10:30:31.418817+0000","last_became_peered":"2024-04-25T10:30:31.418817+0000","last_unstale":"2024-04-25T10:34:56.124793+0000","last_undegraded":"2024-04-25T10:34:56.124793+0000","last_fullsized":"2024-04-25T10:34:56.124793+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'15","last_scrub_stamp":"2024-04-25T10:34:56.124709+0000","last_deep_scrub":"22'15","last_deep_scrub_stamp":"2024-04-25T10:34:56.124709+0000","last_clean_scrub_stamp":"2024-04-25T10:34:56.124709+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:57.641228+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":12,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"22'131","reported_seq":158,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:59.640928+0000","last_change":"2024-04-25T10:34:59.640982+0000","last_active":"2024-04-25T10:34:59.640928+0000","last_peered":"2024-04-25T10:34:59.640928+0000","last_clean":"2024-04-25T10:34:59.640928+0000","last_became_active":"2024-04-25T10:30:32.419486+0000","last_became_peered":"2024-04-25T10:30:32.419486+0000","last_unstale":"2024-04-25T10:34:59.640928+0000","last_undegraded":"2024-04-25T10:34:59.640928+0000","last_fullsized":"2024-04-25T10:34:59.640928+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'131","last_scrub_stamp":"2024-04-25T10:34:59.640774+0000","last_deep_scrub":"22'131","last_deep_scrub_stamp":"2024-04-25T10:34:59.640774+0000","last_clean_scrub_stamp":"2024-04-25T10:34:59.640774+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:14.187312+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"22'7","reported_seq":36,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:58.162213+0000","last_change":"2024-04-25T10:34:58.162245+0000","last_active":"2024-04-25T10:34:58.162213+0000","last_peered":"2024-04-25T10:34:58.162213+0000","last_clean":"2024-04-25T10:34:58.162213+0000","last_became_active":"2024-04-25T10:30:31.405238+0000","last_became_peered":"2024-04-25T10:30:31.405238+0000","last_unstale":"2024-04-25T10:34:58.162213+0000","last_undegraded":"2024-04-25T10:34:58.162213+0000","last_fullsized":"2024-04-25T10:34:58.162213+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'7","last_scrub_stamp":"2024-04-25T10:34:58.162122+0000","last_deep_scrub":"22'7","last_deep_scrub_stamp":"2024-04-25T10:34:58.162122+0000","last_clean_scrub_stamp":"2024-04-25T10:34:58.162122+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:45.060717+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"22'145","reported_seq":172,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:59.760452+0000","last_change":"2024-04-25T10:34:59.760526+0000","last_active":"2024-04-25T10:34:59.760452+0000","last_peered":"2024-04-25T10:34:59.760452+0000","last_clean":"2024-04-25T10:34:59.760452+0000","last_became_active":"2024-04-25T10:30:32.425783+0000","last_became_peered":"2024-04-25T10:30:32.425783+0000","last_unstale":"2024-04-25T10:34:59.760452+0000","last_undegraded":"2024-04-25T10:34:59.760452+0000","last_fullsized":"2024-04-25T10:34:59.760452+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'145","last_scrub_stamp":"2024-04-25T10:34:59.760325+0000","last_deep_scrub":"22'145","last_deep_scrub_stamp":"2024-04-25T10:34:59.760325+0000","last_clean_scrub_stamp":"2024-04-25T10:34:59.760325+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:21.131777+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"20'1","reported_seq":30,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:56.598079+0000","last_change":"2024-04-25T10:34:56.598106+0000","last_active":"2024-04-25T10:34:56.598079+0000","last_peered":"2024-04-25T10:34:56.598079+0000","last_clean":"2024-04-25T10:34:56.598079+0000","last_became_active":"2024-04-25T10:30:31.412413+0000","last_became_peered":"2024-04-25T10:30:31.412413+0000","last_unstale":"2024-04-25T10:34:56.598079+0000","last_undegraded":"2024-04-25T10:34:56.598079+0000","last_fullsized":"2024-04-25T10:34:56.598079+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"20'1","last_scrub_stamp":"2024-04-25T10:34:56.597986+0000","last_deep_scrub":"20'1","last_deep_scrub_stamp":"2024-04-25T10:34:56.597986+0000","last_clean_scrub_stamp":"2024-04-25T10:34:56.597986+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:03.011544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"22'143","reported_seq":175,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:35:11.120329+0000","last_change":"2024-04-25T10:35:11.120402+0000","last_active":"2024-04-25T10:35:11.120329+0000","last_peered":"2024-04-25T10:35:11.120329+0000","last_clean":"2024-04-25T10:35:11.120329+0000","last_became_active":"2024-04-25T10:31:13.640368+0000","last_became_peered":"2024-04-25T10:31:13.640368+0000","last_unstale":"2024-04-25T10:35:11.120329+0000","last_undegraded":"2024-04-25T10:35:11.120329+0000","last_fullsized":"2024-04-25T10:35:11.120329+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'143","last_scrub_stamp":"2024-04-25T10:35:11.120192+0000","last_deep_scrub":"22'143","last_deep_scrub_stamp":"2024-04-25T10:35:11.120192+0000","last_clean_scrub_stamp":"2024-04-25T10:35:11.120192+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:12.798090+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"22'7","reported_seq":36,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:35:02.249897+0000","last_change":"2024-04-25T10:35:02.249936+0000","last_active":"2024-04-25T10:35:02.249897+0000","last_peered":"2024-04-25T10:35:02.249897+0000","last_clean":"2024-04-25T10:35:02.249897+0000","last_became_active":"2024-04-25T10:30:31.417541+0000","last_became_peered":"2024-04-25T10:30:31.417541+0000","last_unstale":"2024-04-25T10:35:02.249897+0000","last_undegraded":"2024-04-25T10:35:02.249897+0000","last_fullsized":"2024-04-25T10:35:02.249897+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'7","last_scrub_stamp":"2024-04-25T10:35:02.249788+0000","last_deep_scrub":"22'7","last_deep_scrub_stamp":"2024-04-25T10:35:02.249788+0000","last_clean_scrub_stamp":"2024-04-25T10:35:02.249788+0000","objects_scrubbed":3,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:12:38.526973+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"22'144","reported_seq":171,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:35:00.636716+0000","last_change":"2024-04-25T10:35:00.636746+0000","last_active":"2024-04-25T10:35:00.636716+0000","last_peered":"2024-04-25T10:35:00.636716+0000","last_clean":"2024-04-25T10:35:00.636716+0000","last_became_active":"2024-04-25T10:30:32.420109+0000","last_became_peered":"2024-04-25T10:30:32.420109+0000","last_unstale":"2024-04-25T10:35:00.636716+0000","last_undegraded":"2024-04-25T10:35:00.636716+0000","last_fullsized":"2024-04-25T10:35:00.636716+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'144","last_scrub_stamp":"2024-04-25T10:35:00.636595+0000","last_deep_scrub":"22'144","last_deep_scrub_stamp":"2024-04-25T10:35:00.636595+0000","last_clean_scrub_stamp":"2024-04-25T10:35:00.636595+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:51.174326+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'26","reported_seq":55,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:54.614914+0000","last_change":"2024-04-25T10:34:54.614914+0000","last_active":"2024-04-25T10:34:54.614914+0000","last_peered":"2024-04-25T10:34:54.614914+0000","last_clean":"2024-04-25T10:34:54.614914+0000","last_became_active":"2024-04-25T10:30:31.404739+0000","last_became_peered":"2024-04-25T10:30:31.404739+0000","last_unstale":"2024-04-25T10:34:54.614914+0000","last_undegraded":"2024-04-25T10:34:54.614914+0000","last_fullsized":"2024-04-25T10:34:54.614914+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'26","last_scrub_stamp":"2024-04-25T10:34:54.614752+0000","last_deep_scrub":"22'26","last_deep_scrub_stamp":"2024-04-25T10:34:54.614752+0000","last_clean_scrub_stamp":"2024-04-25T10:34:54.614752+0000","objects_scrubbed":4,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:05:11.607992+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"22'143","reported_seq":170,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:35:07.146576+0000","last_change":"2024-04-25T10:35:07.146628+0000","last_active":"2024-04-25T10:35:07.146576+0000","last_peered":"2024-04-25T10:35:07.146576+0000","last_clean":"2024-04-25T10:35:07.146576+0000","last_became_active":"2024-04-25T10:30:32.437171+0000","last_became_peered":"2024-04-25T10:30:32.437171+0000","last_unstale":"2024-04-25T10:35:07.146576+0000","last_undegraded":"2024-04-25T10:35:07.146576+0000","last_fullsized":"2024-04-25T10:35:07.146576+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'143","last_scrub_stamp":"2024-04-25T10:35:07.146429+0000","last_deep_scrub":"22'143","last_deep_scrub_stamp":"2024-04-25T10:35:07.146429+0000","last_clean_scrub_stamp":"2024-04-25T10:35:07.146429+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:23.403720+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"22'12","reported_seq":41,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:57.009977+0000","last_change":"2024-04-25T10:34:57.010018+0000","last_active":"2024-04-25T10:34:57.009977+0000","last_peered":"2024-04-25T10:34:57.009977+0000","last_clean":"2024-04-25T10:34:57.009977+0000","last_became_active":"2024-04-25T10:30:31.412821+0000","last_became_peered":"2024-04-25T10:30:31.412821+0000","last_unstale":"2024-04-25T10:34:57.009977+0000","last_undegraded":"2024-04-25T10:34:57.009977+0000","last_fullsized":"2024-04-25T10:34:57.009977+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'12","last_scrub_stamp":"2024-04-25T10:34:57.009872+0000","last_deep_scrub":"22'12","last_deep_scrub_stamp":"2024-04-25T10:34:57.009872+0000","last_clean_scrub_stamp":"2024-04-25T10:34:57.009872+0000","objects_scrubbed":4,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:38.017102+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"22'163","reported_seq":190,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:35:01.157415+0000","last_change":"2024-04-25T10:35:01.157443+0000","last_active":"2024-04-25T10:35:01.157415+0000","last_peered":"2024-04-25T10:35:01.157415+0000","last_clean":"2024-04-25T10:35:01.157415+0000","last_became_active":"2024-04-25T10:30:32.434000+0000","last_became_peered":"2024-04-25T10:30:32.434000+0000","last_unstale":"2024-04-25T10:35:01.157415+0000","last_undegraded":"2024-04-25T10:35:01.157415+0000","last_fullsized":"2024-04-25T10:35:01.157415+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'163","last_scrub_stamp":"2024-04-25T10:35:01.157314+0000","last_deep_scrub":"22'163","last_deep_scrub_stamp":"2024-04-25T10:35:01.157314+0000","last_clean_scrub_stamp":"2024-04-25T10:35:01.157314+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:09.213461+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"22'3","reported_seq":32,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:35:04.273661+0000","last_change":"2024-04-25T10:35:04.273698+0000","last_active":"2024-04-25T10:35:04.273661+0000","last_peered":"2024-04-25T10:35:04.273661+0000","last_clean":"2024-04-25T10:35:04.273661+0000","last_became_active":"2024-04-25T10:30:31.417605+0000","last_became_peered":"2024-04-25T10:30:31.417605+0000","last_unstale":"2024-04-25T10:35:04.273661+0000","last_undegraded":"2024-04-25T10:35:04.273661+0000","last_fullsized":"2024-04-25T10:35:04.273661+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'3","last_scrub_stamp":"2024-04-25T10:35:04.273567+0000","last_deep_scrub":"22'3","last_deep_scrub_stamp":"2024-04-25T10:35:04.273567+0000","last_clean_scrub_stamp":"2024-04-25T10:35:04.273567+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:30.148022+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"22'134","reported_seq":161,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:56.736421+0000","last_change":"2024-04-25T10:34:56.736454+0000","last_active":"2024-04-25T10:34:56.736421+0000","last_peered":"2024-04-25T10:34:56.736421+0000","last_clean":"2024-04-25T10:34:56.736421+0000","last_became_active":"2024-04-25T10:30:32.419370+0000","last_became_peered":"2024-04-25T10:30:32.419370+0000","last_unstale":"2024-04-25T10:34:56.736421+0000","last_undegraded":"2024-04-25T10:34:56.736421+0000","last_fullsized":"2024-04-25T10:34:56.736421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'134","last_scrub_stamp":"2024-04-25T10:34:56.736307+0000","last_deep_scrub":"22'134","last_deep_scrub_stamp":"2024-04-25T10:34:56.736307+0000","last_clean_scrub_stamp":"2024-04-25T10:34:56.736307+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:02.189813+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"22'11","reported_seq":40,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:55.622452+0000","last_change":"2024-04-25T10:34:55.622521+0000","last_active":"2024-04-25T10:34:55.622452+0000","last_peered":"2024-04-25T10:34:55.622452+0000","last_clean":"2024-04-25T10:34:55.622452+0000","last_became_active":"2024-04-25T10:30:31.410404+0000","last_became_peered":"2024-04-25T10:30:31.410404+0000","last_unstale":"2024-04-25T10:34:55.622452+0000","last_undegraded":"2024-04-25T10:34:55.622452+0000","last_fullsized":"2024-04-25T10:34:55.622452+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'11","last_scrub_stamp":"2024-04-25T10:34:55.622357+0000","last_deep_scrub":"22'11","last_deep_scrub_stamp":"2024-04-25T10:34:55.622357+0000","last_clean_scrub_stamp":"2024-04-25T10:34:55.622357+0000","objects_scrubbed":4,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:18:01.394524+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"22'153","reported_seq":180,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:35:04.151707+0000","last_change":"2024-04-25T10:35:04.151737+0000","last_active":"2024-04-25T10:35:04.151707+0000","last_peered":"2024-04-25T10:35:04.151707+0000","last_clean":"2024-04-25T10:35:04.151707+0000","last_became_active":"2024-04-25T10:30:32.433389+0000","last_became_peered":"2024-04-25T10:30:32.433389+0000","last_unstale":"2024-04-25T10:35:04.151707+0000","last_undegraded":"2024-04-25T10:35:04.151707+0000","last_fullsized":"2024-04-25T10:35:04.151707+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'153","last_scrub_stamp":"2024-04-25T10:35:04.151607+0000","last_deep_scrub":"22'153","last_deep_scrub_stamp":"2024-04-25T10:35:04.151607+0000","last_clean_scrub_stamp":"2024-04-25T10:35:04.151607+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:30.294349+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"22'5","reported_seq":34,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:58.650970+0000","last_change":"2024-04-25T10:34:58.651022+0000","last_active":"2024-04-25T10:34:58.650970+0000","last_peered":"2024-04-25T10:34:58.650970+0000","last_clean":"2024-04-25T10:34:58.650970+0000","last_became_active":"2024-04-25T10:30:31.418822+0000","last_became_peered":"2024-04-25T10:30:31.418822+0000","last_unstale":"2024-04-25T10:34:58.650970+0000","last_undegraded":"2024-04-25T10:34:58.650970+0000","last_fullsized":"2024-04-25T10:34:58.650970+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'5","last_scrub_stamp":"2024-04-25T10:34:58.650838+0000","last_deep_scrub":"22'5","last_deep_scrub_stamp":"2024-04-25T10:34:58.650838+0000","last_clean_scrub_stamp":"2024-04-25T10:34:58.650838+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:59.290839+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"22'143","reported_seq":170,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:35:01.171092+0000","last_change":"2024-04-25T10:35:01.171092+0000","last_active":"2024-04-25T10:35:01.171092+0000","last_peered":"2024-04-25T10:35:01.171092+0000","last_clean":"2024-04-25T10:35:01.171092+0000","last_became_active":"2024-04-25T10:30:32.427504+0000","last_became_peered":"2024-04-25T10:30:32.427504+0000","last_unstale":"2024-04-25T10:35:01.171092+0000","last_undegraded":"2024-04-25T10:35:01.171092+0000","last_fullsized":"2024-04-25T10:35:01.171092+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'143","last_scrub_stamp":"2024-04-25T10:35:01.170891+0000","last_deep_scrub":"22'143","last_deep_scrub_stamp":"2024-04-25T10:35:01.170891+0000","last_clean_scrub_stamp":"2024-04-25T10:35:01.170891+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:59.840338+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'7","reported_seq":36,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:35:01.229560+0000","last_change":"2024-04-25T10:35:01.229560+0000","last_active":"2024-04-25T10:35:01.229560+0000","last_peered":"2024-04-25T10:35:01.229560+0000","last_clean":"2024-04-25T10:35:01.229560+0000","last_became_active":"2024-04-25T10:30:31.416766+0000","last_became_peered":"2024-04-25T10:30:31.416766+0000","last_unstale":"2024-04-25T10:35:01.229560+0000","last_undegraded":"2024-04-25T10:35:01.229560+0000","last_fullsized":"2024-04-25T10:35:01.229560+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'7","last_scrub_stamp":"2024-04-25T10:35:01.229410+0000","last_deep_scrub":"22'7","last_deep_scrub_stamp":"2024-04-25T10:35:01.229410+0000","last_clean_scrub_stamp":"2024-04-25T10:35:01.229410+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:42:47.027862+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"22'128","reported_seq":155,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:35:02.062330+0000","last_change":"2024-04-25T10:35:02.062363+0000","last_active":"2024-04-25T10:35:02.062330+0000","last_peered":"2024-04-25T10:35:02.062330+0000","last_clean":"2024-04-25T10:35:02.062330+0000","last_became_active":"2024-04-25T10:30:32.431395+0000","last_became_peered":"2024-04-25T10:30:32.431395+0000","last_unstale":"2024-04-25T10:35:02.062330+0000","last_undegraded":"2024-04-25T10:35:02.062330+0000","last_fullsized":"2024-04-25T10:35:02.062330+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'128","last_scrub_stamp":"2024-04-25T10:35:02.062211+0000","last_deep_scrub":"22'128","last_deep_scrub_stamp":"2024-04-25T10:35:02.062211+0000","last_clean_scrub_stamp":"2024-04-25T10:35:02.062211+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:22.556083+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"22'183","reported_seq":209,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:35:02.181108+0000","last_change":"2024-04-25T10:35:02.181150+0000","last_active":"2024-04-25T10:35:02.181108+0000","last_peered":"2024-04-25T10:35:02.181108+0000","last_clean":"2024-04-25T10:35:02.181108+0000","last_became_active":"2024-04-25T10:30:32.437242+0000","last_became_peered":"2024-04-25T10:30:32.437242+0000","last_unstale":"2024-04-25T10:35:02.181108+0000","last_undegraded":"2024-04-25T10:35:02.181108+0000","last_fullsized":"2024-04-25T10:35:02.181108+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'183","last_scrub_stamp":"2024-04-25T10:35:02.180969+0000","last_deep_scrub":"22'183","last_deep_scrub_stamp":"2024-04-25T10:35:02.180969+0000","last_clean_scrub_stamp":"2024-04-25T10:35:02.180969+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:29.292924+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":168,"num_write_kb":458752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"22'8","reported_seq":36,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:35:05.298113+0000","last_change":"2024-04-25T10:35:05.298175+0000","last_active":"2024-04-25T10:35:05.298113+0000","last_peered":"2024-04-25T10:35:05.298113+0000","last_clean":"2024-04-25T10:35:05.298113+0000","last_became_active":"2024-04-25T10:30:31.419269+0000","last_became_peered":"2024-04-25T10:30:31.419269+0000","last_unstale":"2024-04-25T10:35:05.298113+0000","last_undegraded":"2024-04-25T10:35:05.298113+0000","last_fullsized":"2024-04-25T10:35:05.298113+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'8","last_scrub_stamp":"2024-04-25T10:35:05.297981+0000","last_deep_scrub":"22'8","last_deep_scrub_stamp":"2024-04-25T10:35:05.297981+0000","last_clean_scrub_stamp":"2024-04-25T10:35:05.297981+0000","objects_scrubbed":3,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:43.809142+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"22'141","reported_seq":167,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:35:01.149146+0000","last_change":"2024-04-25T10:35:01.149175+0000","last_active":"2024-04-25T10:35:01.149146+0000","last_peered":"2024-04-25T10:35:01.149146+0000","last_clean":"2024-04-25T10:35:01.149146+0000","last_became_active":"2024-04-25T10:30:32.434093+0000","last_became_peered":"2024-04-25T10:30:32.434093+0000","last_unstale":"2024-04-25T10:35:01.149146+0000","last_undegraded":"2024-04-25T10:35:01.149146+0000","last_fullsized":"2024-04-25T10:35:01.149146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'141","last_scrub_stamp":"2024-04-25T10:35:01.149047+0000","last_deep_scrub":"22'141","last_deep_scrub_stamp":"2024-04-25T10:35:01.149047+0000","last_clean_scrub_stamp":"2024-04-25T10:35:01.149047+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:01.268618+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'20","reported_seq":48,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:35:03.280948+0000","last_change":"2024-04-25T10:35:03.281002+0000","last_active":"2024-04-25T10:35:03.280948+0000","last_peered":"2024-04-25T10:35:03.280948+0000","last_clean":"2024-04-25T10:35:03.280948+0000","last_became_active":"2024-04-25T10:30:31.418005+0000","last_became_peered":"2024-04-25T10:30:31.418005+0000","last_unstale":"2024-04-25T10:35:03.280948+0000","last_undegraded":"2024-04-25T10:35:03.280948+0000","last_fullsized":"2024-04-25T10:35:03.280948+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'20","last_scrub_stamp":"2024-04-25T10:35:03.280844+0000","last_deep_scrub":"22'20","last_deep_scrub_stamp":"2024-04-25T10:35:03.280844+0000","last_clean_scrub_stamp":"2024-04-25T10:35:03.280844+0000","objects_scrubbed":7,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:10.154616+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"22'135","reported_seq":162,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:35:03.132821+0000","last_change":"2024-04-25T10:35:03.132889+0000","last_active":"2024-04-25T10:35:03.132821+0000","last_peered":"2024-04-25T10:35:03.132821+0000","last_clean":"2024-04-25T10:35:03.132821+0000","last_became_active":"2024-04-25T10:30:32.439101+0000","last_became_peered":"2024-04-25T10:30:32.439101+0000","last_unstale":"2024-04-25T10:35:03.132821+0000","last_undegraded":"2024-04-25T10:35:03.132821+0000","last_fullsized":"2024-04-25T10:35:03.132821+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'135","last_scrub_stamp":"2024-04-25T10:35:03.132659+0000","last_deep_scrub":"22'135","last_deep_scrub_stamp":"2024-04-25T10:35:03.132659+0000","last_clean_scrub_stamp":"2024-04-25T10:35:03.132659+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:16.574869+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'2","reported_seq":31,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:57.166093+0000","last_change":"2024-04-25T10:34:57.166126+0000","last_active":"2024-04-25T10:34:57.166093+0000","last_peered":"2024-04-25T10:34:57.166093+0000","last_clean":"2024-04-25T10:34:57.166093+0000","last_became_active":"2024-04-25T10:30:31.417737+0000","last_became_peered":"2024-04-25T10:30:31.417737+0000","last_unstale":"2024-04-25T10:34:57.166093+0000","last_undegraded":"2024-04-25T10:34:57.166093+0000","last_fullsized":"2024-04-25T10:34:57.166093+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"20'2","last_scrub_stamp":"2024-04-25T10:34:57.166002+0000","last_deep_scrub":"20'2","last_deep_scrub_stamp":"2024-04-25T10:34:57.166002+0000","last_clean_scrub_stamp":"2024-04-25T10:34:57.166002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:38.990658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"22'152","reported_seq":179,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:35:06.140868+0000","last_change":"2024-04-25T10:35:06.140925+0000","last_active":"2024-04-25T10:35:06.140868+0000","last_peered":"2024-04-25T10:35:06.140868+0000","last_clean":"2024-04-25T10:35:06.140868+0000","last_became_active":"2024-04-25T10:30:32.434930+0000","last_became_peered":"2024-04-25T10:30:32.434930+0000","last_unstale":"2024-04-25T10:35:06.140868+0000","last_undegraded":"2024-04-25T10:35:06.140868+0000","last_fullsized":"2024-04-25T10:35:06.140868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'152","last_scrub_stamp":"2024-04-25T10:35:06.140705+0000","last_deep_scrub":"22'152","last_deep_scrub_stamp":"2024-04-25T10:35:06.140705+0000","last_clean_scrub_stamp":"2024-04-25T10:35:06.140705+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:22.447778+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'19","reported_seq":48,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:34:54.172324+0000","last_change":"2024-04-25T10:34:54.172393+0000","last_active":"2024-04-25T10:34:54.172324+0000","last_peered":"2024-04-25T10:34:54.172324+0000","last_clean":"2024-04-25T10:34:54.172324+0000","last_became_active":"2024-04-25T10:30:31.415071+0000","last_became_peered":"2024-04-25T10:30:31.415071+0000","last_unstale":"2024-04-25T10:34:54.172324+0000","last_undegraded":"2024-04-25T10:34:54.172324+0000","last_fullsized":"2024-04-25T10:34:54.172324+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'19","last_scrub_stamp":"2024-04-25T10:34:54.172184+0000","last_deep_scrub":"22'19","last_deep_scrub_stamp":"2024-04-25T10:34:54.172184+0000","last_clean_scrub_stamp":"2024-04-25T10:34:54.172184+0000","objects_scrubbed":6,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:34.737710+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9216,"num_write_kb":25165824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9717,"ondisk_log_size":9717,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":210463,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":345,"num_write_kb":324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":25000,"num_omap_keys":50,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":671744,"data_stored":420926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":396,"ondisk_log_size":396,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574911,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40336,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331504,"statfs":{"total":96636764160,"available":96595460096,"internally_reserved":0,"allocated":860160,"data_stored":676182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574911,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35784,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336056,"statfs":{"total":96636764160,"available":96600121344,"internally_reserved":0,"allocated":458752,"data_stored":286294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574912,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39768,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94332072,"statfs":{"total":96636764160,"available":96596041728,"internally_reserved":0,"allocated":278528,"data_stored":87646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574912,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39744,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94332096,"statfs":{"total":96636764160,"available":96596066304,"internally_reserved":0,"allocated":253952,"data_stored":85044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574911,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39952,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331888,"statfs":{"total":96636764160,"available":96595853312,"internally_reserved":0,"allocated":466944,"data_stored":286833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574911,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35596,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336244,"statfs":{"total":96636764160,"available":96600313856,"internally_reserved":0,"allocated":266240,"data_stored":85681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35588,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336252,"statfs":{"total":96636764160,"available":96600322048,"internally_reserved":0,"allocated":258048,"data_stored":84664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574911,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40328,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331512,"statfs":{"total":96636764160,"available":96595468288,"internally_reserved":0,"allocated":851968,"data_stored":674374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":237568,"data_stored":203701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":229376,"data_stored":203162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:35:17.075 DEBUG:teuthology.orchestra.run.smithi106:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-25T10:35:17.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.214+0000 7f6180d3f640 1 Processor -- start 2024-04-25T10:35:17.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.214+0000 7f6180d3f640 1 -- start start 2024-04-25T10:35:17.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.215+0000 7f6180d3f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f617c156f70 0x7f617c15ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:17.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.215+0000 7f6180d3f640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f617c15d340 0x7f617c167680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:17.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.215+0000 7f6180d3f640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f617c073980 con 0x7f617c0ca170 2024-04-25T10:35:17.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.215+0000 7f6180d3f640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f617c073f00 con 0x7f617c15d340 2024-04-25T10:35:17.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.215+0000 7f6180d3f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f617c074990 con 0x7f617c156f70 2024-04-25T10:35:17.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.216+0000 7f617ad9d640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f617c15d340 0x7f617c167680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:17.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.216+0000 7f6173fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f617c156f70 0x7f617c15ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:17.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.216+0000 7f617a59c640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f617c0ca170 0x7f617c06c5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.106:52792/0 (socket says 172.21.15.106:52792) 2024-04-25T10:35:17.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.216+0000 7f617a59c640 1 -- 172.21.15.106:0/4276245991 learned_addr learned my addr 172.21.15.106:0/4276245991 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:35:17.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.216+0000 7f6173fff640 1 -- 172.21.15.106:0/4276245991 >> v2:172.21.15.134:3300/0 conn(0x7f617c15d340 msgr2=0x7f617c167680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:17.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.216+0000 7f6173fff640 1 --2- 172.21.15.106:0/4276245991 >> v2:172.21.15.134:3300/0 conn(0x7f617c15d340 0x7f617c167680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:17.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.216+0000 7f6173fff640 1 -- 172.21.15.106:0/4276245991 >> v1:172.21.15.134:6789/0 conn(0x7f617c0ca170 legacy=0x7f617c06c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:17.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.216+0000 7f6173fff640 1 -- 172.21.15.106:0/4276245991 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f617c160ea0 con 0x7f617c156f70 2024-04-25T10:35:17.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.216+0000 7f617ad9d640 1 --2- 172.21.15.106:0/4276245991 >> v2:172.21.15.134:3300/0 conn(0x7f617c15d340 0x7f617c167680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:35:17.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.217+0000 7f6173fff640 1 --2- 172.21.15.106:0/4276245991 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f617c156f70 0x7f617c15ce00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f61640097a0 tx=0x7f616402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b17d970492b3287 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:17.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.217+0000 7f6179d9b640 1 -- 172.21.15.106:0/4276245991 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6164037070 con 0x7f617c156f70 2024-04-25T10:35:17.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.217+0000 7f6179d9b640 1 -- 172.21.15.106:0/4276245991 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6164032440 con 0x7f617c156f70 2024-04-25T10:35:17.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.217+0000 7f6179d9b640 1 -- 172.21.15.106:0/4276245991 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6164032810 con 0x7f617c156f70 2024-04-25T10:35:17.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.218+0000 7f6180d3f640 1 -- 172.21.15.106:0/4276245991 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f617c156f70 msgr2=0x7f617c15ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:17.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.218+0000 7f6180d3f640 1 --2- 172.21.15.106:0/4276245991 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f617c156f70 0x7f617c15ce00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f61640097a0 tx=0x7f616402a210 comp rx=0 tx=0).stop 2024-04-25T10:35:17.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.218+0000 7f6180d3f640 1 -- 172.21.15.106:0/4276245991 shutdown_connections 2024-04-25T10:35:17.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.218+0000 7f6180d3f640 1 --2- 172.21.15.106:0/4276245991 >> v2:172.21.15.134:3300/0 conn(0x7f617c15d340 0x7f617c167680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:17.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.218+0000 7f6180d3f640 1 --2- 172.21.15.106:0/4276245991 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f617c156f70 0x7f617c15ce00 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:17.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.218+0000 7f6180d3f640 1 -- 172.21.15.106:0/4276245991 >> 172.21.15.106:0/4276245991 conn(0x7f617c0cef20 msgr2=0x7f617c0cf330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:17.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.218+0000 7f6180d3f640 1 -- 172.21.15.106:0/4276245991 shutdown_connections 2024-04-25T10:35:17.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.218+0000 7f6180d3f640 1 -- 172.21.15.106:0/4276245991 wait complete. 2024-04-25T10:35:17.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.219+0000 7f6180d3f640 1 Processor -- start 2024-04-25T10:35:17.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.220+0000 7f6180d3f640 1 -- start start 2024-04-25T10:35:17.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.221+0000 7f6180d3f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f617c0ca170 0x7f617c15ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:17.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.221+0000 7f6180d3f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f617c156f70 0x7f617c1f2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:17.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.221+0000 7f6180d3f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f617c15d340 0x7f617c1549f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:17.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.221+0000 7f6180d3f640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f617c073c20 con 0x7f617c156f70 2024-04-25T10:35:17.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.221+0000 7f6180d3f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f617c074640 con 0x7f617c0ca170 2024-04-25T10:35:17.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.221+0000 7f6180d3f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f617c161140 con 0x7f617c15d340 2024-04-25T10:35:17.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.221+0000 7f6173fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f617c156f70 0x7f617c1f2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:17.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.221+0000 7f6173fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f617c156f70 0x7f617c1f2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38812/0 (socket says 172.21.15.106:38812) 2024-04-25T10:35:17.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.221+0000 7f6173fff640 1 -- 172.21.15.106:0/3849048488 learned_addr learned my addr 172.21.15.106:0/3849048488 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:35:17.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.221+0000 7f617a59c640 1 --2- 172.21.15.106:0/3849048488 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f617c0ca170 0x7f617c15ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:17.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.221+0000 7f617ad9d640 1 --2- 172.21.15.106:0/3849048488 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f617c15d340 0x7f617c1549f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:17.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.221+0000 7f6173fff640 1 -- 172.21.15.106:0/3849048488 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f617c15d340 msgr2=0x7f617c1549f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:17.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.221+0000 7f6173fff640 1 --2- 172.21.15.106:0/3849048488 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f617c15d340 0x7f617c1549f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:17.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.221+0000 7f6173fff640 1 -- 172.21.15.106:0/3849048488 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f617c0ca170 msgr2=0x7f617c15ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:17.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.221+0000 7f6173fff640 1 --2- 172.21.15.106:0/3849048488 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f617c0ca170 0x7f617c15ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:17.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.221+0000 7f6173fff640 1 -- 172.21.15.106:0/3849048488 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f617c1f30a0 con 0x7f617c156f70 2024-04-25T10:35:17.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.221+0000 7f6173fff640 1 --2- 172.21.15.106:0/3849048488 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f617c156f70 0x7f617c1f2b60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6164035040 tx=0x7f61640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:17.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.222+0000 7f6172ffd640 1 -- 172.21.15.106:0/3849048488 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6164037040 con 0x7f617c156f70 2024-04-25T10:35:17.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.222+0000 7f6172ffd640 1 -- 172.21.15.106:0/3849048488 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6164030500 con 0x7f617c156f70 2024-04-25T10:35:17.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.222+0000 7f6172ffd640 1 -- 172.21.15.106:0/3849048488 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6164030910 con 0x7f617c156f70 2024-04-25T10:35:17.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.222+0000 7f6180d3f640 1 -- 172.21.15.106:0/3849048488 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f617c1f60b0 con 0x7f617c156f70 2024-04-25T10:35:17.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.222+0000 7f6180d3f640 1 -- 172.21.15.106:0/3849048488 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f617c1f6670 con 0x7f617c156f70 2024-04-25T10:35:17.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.223+0000 7f6180d3f640 1 -- 172.21.15.106:0/3849048488 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f617c1f28f0 con 0x7f617c156f70 2024-04-25T10:35:17.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.223+0000 7f6172ffd640 1 -- 172.21.15.106:0/3849048488 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6164044020 con 0x7f617c156f70 2024-04-25T10:35:17.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.224+0000 7f6172ffd640 1 --2- 172.21.15.106:0/3849048488 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f615c0744b0 0x7f615c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:17.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.224+0000 7f617a59c640 1 --2- 172.21.15.106:0/3849048488 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f615c0744b0 0x7f615c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:17.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.224+0000 7f6172ffd640 1 -- 172.21.15.106:0/3849048488 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f61640b55c0 con 0x7f617c156f70 2024-04-25T10:35:17.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.224+0000 7f617a59c640 1 --2- 172.21.15.106:0/3849048488 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f615c0744b0 0x7f615c0769c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f616c007fd0 tx=0x7f616c007e80 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:17.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.228+0000 7f6172ffd640 1 -- 172.21.15.106:0/3849048488 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f617c1f28f0 con 0x7f617c156f70 2024-04-25T10:35:17.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.406+0000 7f6180d3f640 1 -- 172.21.15.106:0/3849048488 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f617c06a3d0 con 0x7f617c156f70 2024-04-25T10:35:17.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.410+0000 7f6172ffd640 1 -- 172.21.15.106:0/3849048488 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v10)=0 v10) ==== 123+0+0 (secure 0 0 0) 0x7f616407fc50 con 0x7f617c156f70 2024-04-25T10:35:17.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.410+0000 7f6172ffd640 1 -- 172.21.15.106:0/3849048488 <== mon.0 v2:172.21.15.106:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f616407f060 con 0x7f617c156f70 2024-04-25T10:35:17.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.412+0000 7f6180d3f640 1 -- 172.21.15.106:0/3849048488 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f615c0744b0 msgr2=0x7f615c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:17.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.412+0000 7f6180d3f640 1 --2- 172.21.15.106:0/3849048488 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f615c0744b0 0x7f615c0769c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f616c007fd0 tx=0x7f616c007e80 comp rx=0 tx=0).stop 2024-04-25T10:35:17.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.412+0000 7f6180d3f640 1 -- 172.21.15.106:0/3849048488 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f617c156f70 msgr2=0x7f617c1f2b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:17.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.412+0000 7f6180d3f640 1 --2- 172.21.15.106:0/3849048488 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f617c156f70 0x7f617c1f2b60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6164035040 tx=0x7f61640046c0 comp rx=0 tx=0).stop 2024-04-25T10:35:17.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.412+0000 7f6180d3f640 1 -- 172.21.15.106:0/3849048488 shutdown_connections 2024-04-25T10:35:17.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.412+0000 7f6180d3f640 1 --2- 172.21.15.106:0/3849048488 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f615c0744b0 0x7f615c0769c0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:17.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.412+0000 7f6180d3f640 1 --2- 172.21.15.106:0/3849048488 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f617c15d340 0x7f617c1549f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:17.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.412+0000 7f6180d3f640 1 --2- 172.21.15.106:0/3849048488 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f617c156f70 0x7f617c1f2b60 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:17.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.412+0000 7f6180d3f640 1 --2- 172.21.15.106:0/3849048488 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f617c0ca170 0x7f617c15ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:17.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.412+0000 7f6180d3f640 1 -- 172.21.15.106:0/3849048488 >> 172.21.15.106:0/3849048488 conn(0x7f617c0cef20 msgr2=0x7f617c153550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:17.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.412+0000 7f6180d3f640 1 -- 172.21.15.106:0/3849048488 shutdown_connections 2024-04-25T10:35:17.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.412+0000 7f6180d3f640 1 -- 172.21.15.106:0/3849048488 wait complete. 2024-04-25T10:35:17.427 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:35:17.427 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:35:17.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.532+0000 7fe36db72640 1 Processor -- start 2024-04-25T10:35:17.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.532+0000 7fe36db72640 1 -- start start 2024-04-25T10:35:17.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.533+0000 7fe36db72640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3680c8050 0x7fe3680c8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:17.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.533+0000 7fe36db72640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fe36815eba0 0x7fe3681547a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:17.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.533+0000 7fe36db72640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3680c9b00 con 0x7fe3680cae80 2024-04-25T10:35:17.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.533+0000 7fe36db72640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fe3680c9da0 con 0x7fe36815eba0 2024-04-25T10:35:17.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.533+0000 7fe36db72640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe3680ca080 con 0x7fe3680c8050 2024-04-25T10:35:17.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.534+0000 7fe366ffd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fe3680cae80 0x7fe36815e490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:58906/0 (socket says 172.21.15.106:58906) 2024-04-25T10:35:17.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.534+0000 7fe366ffd640 1 -- 172.21.15.106:0/323786960 learned_addr learned my addr 172.21.15.106:0/323786960 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:35:17.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.534+0000 7fe3677fe640 1 --2- 172.21.15.106:0/323786960 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3680c8050 0x7fe3680c8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:17.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.534+0000 7fe367fff640 1 --2- 172.21.15.106:0/323786960 >> v2:172.21.15.134:3300/0 conn(0x7fe36815eba0 0x7fe3681547a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:17.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.534+0000 7fe3667fc640 1 -- 172.21.15.106:0/323786960 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2057888995 0 0) 0x7fe3680c9b00 con 0x7fe3680cae80 2024-04-25T10:35:17.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.534+0000 7fe3667fc640 1 -- 172.21.15.106:0/323786960 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe350003520 con 0x7fe3680cae80 2024-04-25T10:35:17.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.534+0000 7fe3667fc640 1 -- 172.21.15.106:0/323786960 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3333926295 0 0) 0x7fe350003520 con 0x7fe3680cae80 2024-04-25T10:35:17.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.534+0000 7fe367fff640 1 -- 172.21.15.106:0/323786960 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3680c8050 msgr2=0x7fe3680c8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:17.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.534+0000 7fe367fff640 1 --2- 172.21.15.106:0/323786960 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3680c8050 0x7fe3680c8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:17.537 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.534+0000 7fe367fff640 1 -- 172.21.15.106:0/323786960 >> v1:172.21.15.106:6789/0 conn(0x7fe3680cae80 legacy=0x7fe36815e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:17.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.534+0000 7fe367fff640 1 -- 172.21.15.106:0/323786960 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3680ca360 con 0x7fe36815eba0 2024-04-25T10:35:17.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.534+0000 7fe3677fe640 1 --2- 172.21.15.106:0/323786960 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3680c8050 0x7fe3680c8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:35:17.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.535+0000 7fe367fff640 1 --2- 172.21.15.106:0/323786960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe36815eba0 0x7fe3681547a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe360009d90 tx=0x7fe36002a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f759c2f06ec38a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:17.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.535+0000 7fe3667fc640 1 -- 172.21.15.106:0/323786960 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe35c001690 con 0x7fe36815eba0 2024-04-25T10:35:17.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.535+0000 7fe3667fc640 1 -- 172.21.15.106:0/323786960 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fe360007540 con 0x7fe36815eba0 2024-04-25T10:35:17.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.535+0000 7fe3667fc640 1 -- 172.21.15.106:0/323786960 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3600078f0 con 0x7fe36815eba0 2024-04-25T10:35:17.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.536+0000 7fe36db72640 1 -- 172.21.15.106:0/323786960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe36815eba0 msgr2=0x7fe3681547a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:17.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.536+0000 7fe36db72640 1 --2- 172.21.15.106:0/323786960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe36815eba0 0x7fe3681547a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe360009d90 tx=0x7fe36002a680 comp rx=0 tx=0).stop 2024-04-25T10:35:17.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.536+0000 7fe36db72640 1 -- 172.21.15.106:0/323786960 shutdown_connections 2024-04-25T10:35:17.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.536+0000 7fe36db72640 1 --2- 172.21.15.106:0/323786960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe36815eba0 0x7fe3681547a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:17.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.536+0000 7fe36db72640 1 --2- 172.21.15.106:0/323786960 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3680c8050 0x7fe3680c8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:17.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.536+0000 7fe36db72640 1 -- 172.21.15.106:0/323786960 >> 172.21.15.106:0/323786960 conn(0x7fe36806c9b0 msgr2=0x7fe36806cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:17.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.536+0000 7fe36db72640 1 -- 172.21.15.106:0/323786960 shutdown_connections 2024-04-25T10:35:17.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.536+0000 7fe36db72640 1 -- 172.21.15.106:0/323786960 wait complete. 2024-04-25T10:35:17.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.538+0000 7fe36db72640 1 Processor -- start 2024-04-25T10:35:17.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.538+0000 7fe36db72640 1 -- start start 2024-04-25T10:35:17.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.539+0000 7fe36db72640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3680cae80 0x7fe3681f41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:17.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.539+0000 7fe36db72640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe36815eba0 0x7fe368150480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:17.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.539+0000 7fe36db72640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe3681459a0 0x7fe368145db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:17.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.539+0000 7fe36db72640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe3680c9380 con 0x7fe3681459a0 2024-04-25T10:35:17.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.539+0000 7fe36db72640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe3680c8cc0 con 0x7fe36815eba0 2024-04-25T10:35:17.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.539+0000 7fe36db72640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe3680c9500 con 0x7fe3680cae80 2024-04-25T10:35:17.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.539+0000 7fe367fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe3681459a0 0x7fe368145db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:17.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.539+0000 7fe367fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe3681459a0 0x7fe368145db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38816/0 (socket says 172.21.15.106:38816) 2024-04-25T10:35:17.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.539+0000 7fe367fff640 1 -- 172.21.15.106:0/977800264 learned_addr learned my addr 172.21.15.106:0/977800264 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:35:17.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.539+0000 7fe366ffd640 1 --2- 172.21.15.106:0/977800264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe36815eba0 0x7fe368150480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:17.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.539+0000 7fe3677fe640 1 --2- 172.21.15.106:0/977800264 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3680cae80 0x7fe3681f41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:17.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.539+0000 7fe367fff640 1 -- 172.21.15.106:0/977800264 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3680cae80 msgr2=0x7fe3681f41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:17.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.539+0000 7fe367fff640 1 --2- 172.21.15.106:0/977800264 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3680cae80 0x7fe3681f41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:17.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.539+0000 7fe367fff640 1 -- 172.21.15.106:0/977800264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe36815eba0 msgr2=0x7fe368150480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:17.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.539+0000 7fe367fff640 1 --2- 172.21.15.106:0/977800264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe36815eba0 0x7fe368150480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:17.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.539+0000 7fe367fff640 1 -- 172.21.15.106:0/977800264 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3681452f0 con 0x7fe3681459a0 2024-04-25T10:35:17.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.539+0000 7fe366ffd640 1 --2- 172.21.15.106:0/977800264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe36815eba0 0x7fe368150480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:35:17.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.540+0000 7fe367fff640 1 --2- 172.21.15.106:0/977800264 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe3681459a0 0x7fe368145db0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe360009d90 tx=0x7fe360004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:17.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.540+0000 7fe34ffff640 1 -- 172.21.15.106:0/977800264 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe360009450 con 0x7fe3681459a0 2024-04-25T10:35:17.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.540+0000 7fe34ffff640 1 -- 172.21.15.106:0/977800264 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fe360004240 con 0x7fe3681459a0 2024-04-25T10:35:17.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.540+0000 7fe36db72640 1 -- 172.21.15.106:0/977800264 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3681f36e0 con 0x7fe3681459a0 2024-04-25T10:35:17.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.540+0000 7fe34ffff640 1 -- 172.21.15.106:0/977800264 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe360004630 con 0x7fe3681459a0 2024-04-25T10:35:17.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.540+0000 7fe36db72640 1 -- 172.21.15.106:0/977800264 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3681f3ca0 con 0x7fe3681459a0 2024-04-25T10:35:17.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.541+0000 7fe36db72640 1 -- 172.21.15.106:0/977800264 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe36806a3d0 con 0x7fe3681459a0 2024-04-25T10:35:17.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.542+0000 7fe34ffff640 1 -- 172.21.15.106:0/977800264 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe36003b410 con 0x7fe3681459a0 2024-04-25T10:35:17.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.542+0000 7fe34ffff640 1 --2- 172.21.15.106:0/977800264 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fe3540744e0 0x7fe3540769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:17.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.542+0000 7fe34ffff640 1 -- 172.21.15.106:0/977800264 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fe3600b6610 con 0x7fe3681459a0 2024-04-25T10:35:17.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.542+0000 7fe3677fe640 1 --2- 172.21.15.106:0/977800264 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fe3540744e0 0x7fe3540769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:17.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.542+0000 7fe3677fe640 1 --2- 172.21.15.106:0/977800264 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fe3540744e0 0x7fe3540769f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe358009a30 tx=0x7fe358008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:17.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.547+0000 7fe34ffff640 1 -- 172.21.15.106:0/977800264 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe36003f030 con 0x7fe3681459a0 2024-04-25T10:35:17.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.756+0000 7fe36db72640 1 -- 172.21.15.106:0/977800264 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe36806d080 con 0x7fe3681459a0 2024-04-25T10:35:17.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.757+0000 7fe34ffff640 1 -- 172.21.15.106:0/977800264 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+6922 (secure 0 0 0) 0x7fe360080ca0 con 0x7fe3681459a0 2024-04-25T10:35:17.759 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:35:17.760 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/2834155852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":2834155852},{"type":"v1","addr":"172.21.15.106:6835","nonce":2834155852}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:30:31.761107+0000","modified":"2024-04-25T10:31:00.589522+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4290,"mds_1":4287,"mds_2":4284,"mds_3":4269,"mds_4":4266},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4266":{"gid":4266,"name":"c","rank":4,"incarnation":16,"state":"up:active","state_seq":10,"addr":"172.21.15.106:6837/2632121774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":2632121774},{"type":"v1","addr":"172.21.15.106:6837","nonce":2632121774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4269":{"gid":4269,"name":"e","rank":3,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.106:6839/486339834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":486339834},{"type":"v1","addr":"172.21.15.106:6839","nonce":486339834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4284":{"gid":4284,"name":"b","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.134:6836/2781656864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6833","nonce":2781656864},{"type":"v1","addr":"172.21.15.134:6836","nonce":2781656864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4287":{"gid":4287,"name":"f","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.134:6835/1993918351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1993918351},{"type":"v1","addr":"172.21.15.134:6835","nonce":1993918351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4290":{"gid":4290,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.134:6837/2662003420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2662003420},{"type":"v1","addr":"172.21.15.134:6837","nonce":2662003420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4290,"qdb_cluster":[4266,4269,4284,4287,4290]},"id":1}]} 2024-04-25T10:35:17.760 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 17 2024-04-25T10:35:17.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.759+0000 7fe36db72640 1 -- 172.21.15.106:0/977800264 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fe3540744e0 msgr2=0x7fe3540769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:17.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.759+0000 7fe36db72640 1 --2- 172.21.15.106:0/977800264 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fe3540744e0 0x7fe3540769f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe358009a30 tx=0x7fe358008040 comp rx=0 tx=0).stop 2024-04-25T10:35:17.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.759+0000 7fe36db72640 1 -- 172.21.15.106:0/977800264 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe3681459a0 msgr2=0x7fe368145db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:17.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.759+0000 7fe36db72640 1 --2- 172.21.15.106:0/977800264 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe3681459a0 0x7fe368145db0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe360009d90 tx=0x7fe360004a40 comp rx=0 tx=0).stop 2024-04-25T10:35:17.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.759+0000 7fe36db72640 1 -- 172.21.15.106:0/977800264 shutdown_connections 2024-04-25T10:35:17.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.759+0000 7fe36db72640 1 --2- 172.21.15.106:0/977800264 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fe3540744e0 0x7fe3540769f0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:17.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.759+0000 7fe36db72640 1 --2- 172.21.15.106:0/977800264 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe3681459a0 0x7fe368145db0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:17.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.759+0000 7fe36db72640 1 --2- 172.21.15.106:0/977800264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe36815eba0 0x7fe368150480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:17.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.759+0000 7fe36db72640 1 --2- 172.21.15.106:0/977800264 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3680cae80 0x7fe3681f41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:17.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.759+0000 7fe36db72640 1 -- 172.21.15.106:0/977800264 >> 172.21.15.106:0/977800264 conn(0x7fe36806c9b0 msgr2=0x7fe3680c82f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:17.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.759+0000 7fe36db72640 1 -- 172.21.15.106:0/977800264 shutdown_connections 2024-04-25T10:35:17.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.759+0000 7fe36db72640 1 -- 172.21.15.106:0/977800264 wait complete. 2024-04-25T10:35:17.773 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:35:17.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.878+0000 7f397daf2640 1 Processor -- start 2024-04-25T10:35:17.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.878+0000 7f397daf2640 1 -- start start 2024-04-25T10:35:17.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.880+0000 7f397daf2640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3978073020 0x7f397806f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:17.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.880+0000 7f397daf2640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3978068ad0 con 0x7f3978073db0 2024-04-25T10:35:17.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.880+0000 7f397daf2640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39780c9010 con 0x7f3978068600 2024-04-25T10:35:17.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.880+0000 7f397daf2640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f3978071190 con 0x7f3978073020 2024-04-25T10:35:17.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.880+0000 7f3977fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f3978073db0 0x7f39780741c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:58912/0 (socket says 172.21.15.106:58912) 2024-04-25T10:35:17.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.880+0000 7f3977fff640 1 -- 172.21.15.106:0/3447545938 learned_addr learned my addr 172.21.15.106:0/3447545938 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:35:17.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.880+0000 7f3976ffd640 1 --2- 172.21.15.106:0/3447545938 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3978073020 0x7f397806f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:17.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.880+0000 7f39767fc640 1 -- 172.21.15.106:0/3447545938 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 787036223 0 0) 0x7f3978068ad0 con 0x7f3978073db0 2024-04-25T10:35:17.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.880+0000 7f39767fc640 1 -- 172.21.15.106:0/3447545938 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3960003520 con 0x7f3978073db0 2024-04-25T10:35:17.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.880+0000 7f39767fc640 1 -- 172.21.15.106:0/3447545938 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3460294434 0 0) 0x7f3960003520 con 0x7f3978073db0 2024-04-25T10:35:17.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.880+0000 7f3976ffd640 1 -- 172.21.15.106:0/3447545938 >> v1:172.21.15.134:6789/0 conn(0x7f3978068600 legacy=0x7f3978070a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:17.883 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.880+0000 7f3976ffd640 1 -- 172.21.15.106:0/3447545938 >> v1:172.21.15.106:6789/0 conn(0x7f3978073db0 legacy=0x7f39780741c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:17.884 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.880+0000 7f3976ffd640 1 -- 172.21.15.106:0/3447545938 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39780c92f0 con 0x7f3978073020 2024-04-25T10:35:17.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.881+0000 7f3976ffd640 1 --2- 172.21.15.106:0/3447545938 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3978073020 0x7f397806f780 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f396c0097d0 tx=0x7f396c02a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc3081133815185d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:17.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.881+0000 7f39767fc640 1 -- 172.21.15.106:0/3447545938 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f396c037070 con 0x7f3978073020 2024-04-25T10:35:17.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.881+0000 7f39767fc640 1 -- 172.21.15.106:0/3447545938 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f396c02ac30 con 0x7f3978073020 2024-04-25T10:35:17.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.881+0000 7f39767fc640 1 -- 172.21.15.106:0/3447545938 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39700019f0 con 0x7f3978073020 2024-04-25T10:35:17.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.882+0000 7f397daf2640 1 -- 172.21.15.106:0/3447545938 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3978073020 msgr2=0x7f397806f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:17.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.882+0000 7f397daf2640 1 --2- 172.21.15.106:0/3447545938 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3978073020 0x7f397806f780 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f396c0097d0 tx=0x7f396c02a1a0 comp rx=0 tx=0).stop 2024-04-25T10:35:17.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.882+0000 7f397daf2640 1 -- 172.21.15.106:0/3447545938 shutdown_connections 2024-04-25T10:35:17.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.882+0000 7f397daf2640 1 --2- 172.21.15.106:0/3447545938 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3978073020 0x7f397806f780 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:17.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.882+0000 7f397daf2640 1 -- 172.21.15.106:0/3447545938 >> 172.21.15.106:0/3447545938 conn(0x7f39780ce570 msgr2=0x7f39780ce980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:17.885 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.882+0000 7f397daf2640 1 -- 172.21.15.106:0/3447545938 shutdown_connections 2024-04-25T10:35:17.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.882+0000 7f397daf2640 1 -- 172.21.15.106:0/3447545938 wait complete. 2024-04-25T10:35:17.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.884+0000 7f397daf2640 1 Processor -- start 2024-04-25T10:35:17.886 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.884+0000 7f397daf2640 1 -- start start 2024-04-25T10:35:17.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.885+0000 7f397daf2640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3978068600 0x7f397814c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:17.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.885+0000 7f397daf2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3978073020 0x7f39781f0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:17.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.885+0000 7f397daf2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3978073db0 0x7f39780cd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:17.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.885+0000 7f397daf2640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f39780c9ce0 con 0x7f3978073db0 2024-04-25T10:35:17.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.885+0000 7f397daf2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f397806fcc0 con 0x7f3978073020 2024-04-25T10:35:17.889 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.885+0000 7f397daf2640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f3978150ed0 con 0x7f3978068600 2024-04-25T10:35:17.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.885+0000 7f3977fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3978073db0 0x7f39780cd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:17.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.885+0000 7f3977fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3978073db0 0x7f39780cd3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38828/0 (socket says 172.21.15.106:38828) 2024-04-25T10:35:17.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.885+0000 7f3977fff640 1 -- 172.21.15.106:0/3717371752 learned_addr learned my addr 172.21.15.106:0/3717371752 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:35:17.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.885+0000 7f39777fe640 1 --2- 172.21.15.106:0/3717371752 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3978068600 0x7f397814c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:17.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.885+0000 7f3976ffd640 1 --2- 172.21.15.106:0/3717371752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3978073020 0x7f39781f0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:17.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.886+0000 7f3977fff640 1 -- 172.21.15.106:0/3717371752 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3978068600 msgr2=0x7f397814c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:17.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.886+0000 7f3977fff640 1 --2- 172.21.15.106:0/3717371752 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3978068600 0x7f397814c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:17.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.886+0000 7f3977fff640 1 -- 172.21.15.106:0/3717371752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3978073020 msgr2=0x7f39781f0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:17.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.886+0000 7f3977fff640 1 --2- 172.21.15.106:0/3717371752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3978073020 0x7f39781f0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:17.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.886+0000 7f3977fff640 1 -- 172.21.15.106:0/3717371752 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39781523f0 con 0x7f3978073db0 2024-04-25T10:35:17.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.886+0000 7f3976ffd640 1 --2- 172.21.15.106:0/3717371752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3978073020 0x7f39781f0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:35:17.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.886+0000 7f3977fff640 1 --2- 172.21.15.106:0/3717371752 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3978073db0 0x7f39780cd3c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f397000cc00 tx=0x7f397000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:17.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.886+0000 7f395ffff640 1 -- 172.21.15.106:0/3717371752 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3970017070 con 0x7f3978073db0 2024-04-25T10:35:17.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.886+0000 7f395ffff640 1 -- 172.21.15.106:0/3717371752 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f397000ce10 con 0x7f3978073db0 2024-04-25T10:35:17.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.886+0000 7f395ffff640 1 -- 172.21.15.106:0/3717371752 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3970012620 con 0x7f3978073db0 2024-04-25T10:35:17.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.886+0000 7f397daf2640 1 -- 172.21.15.106:0/3717371752 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39781526d0 con 0x7f3978073db0 2024-04-25T10:35:17.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.886+0000 7f397daf2640 1 -- 172.21.15.106:0/3717371752 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f397814c8c0 con 0x7f3978073db0 2024-04-25T10:35:17.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.887+0000 7f397daf2640 1 -- 172.21.15.106:0/3717371752 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3978073b40 con 0x7f3978073db0 2024-04-25T10:35:17.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.888+0000 7f395ffff640 1 -- 172.21.15.106:0/3717371752 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f397001a070 con 0x7f3978073db0 2024-04-25T10:35:17.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.888+0000 7f395ffff640 1 --2- 172.21.15.106:0/3717371752 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f39640744b0 0x7f39640769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:17.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.888+0000 7f39777fe640 1 --2- 172.21.15.106:0/3717371752 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f39640744b0 0x7f39640769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:17.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.888+0000 7f395ffff640 1 -- 172.21.15.106:0/3717371752 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f3970095250 con 0x7f3978073db0 2024-04-25T10:35:17.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.888+0000 7f39777fe640 1 --2- 172.21.15.106:0/3717371752 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f39640744b0 0x7f39640769c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3968000f60 tx=0x7f3968009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:17.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:17.892+0000 7f395ffff640 1 -- 172.21.15.106:0/3717371752 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3978073b40 con 0x7f3978073db0 2024-04-25T10:35:18.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.101+0000 7f397daf2640 1 -- 172.21.15.106:0/3717371752 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f39780c81f0 con 0x7f3978073db0 2024-04-25T10:35:18.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.102+0000 7f395ffff640 1 -- 172.21.15.106:0/3717371752 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+6922 (secure 0 0 0) 0x7f39780c81f0 con 0x7f3978073db0 2024-04-25T10:35:18.104 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:35:18.104 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/2834155852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":2834155852},{"type":"v1","addr":"172.21.15.106:6835","nonce":2834155852}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:30:31.761107+0000","modified":"2024-04-25T10:31:00.589522+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4290,"mds_1":4287,"mds_2":4284,"mds_3":4269,"mds_4":4266},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4266":{"gid":4266,"name":"c","rank":4,"incarnation":16,"state":"up:active","state_seq":10,"addr":"172.21.15.106:6837/2632121774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":2632121774},{"type":"v1","addr":"172.21.15.106:6837","nonce":2632121774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4269":{"gid":4269,"name":"e","rank":3,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.106:6839/486339834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":486339834},{"type":"v1","addr":"172.21.15.106:6839","nonce":486339834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4284":{"gid":4284,"name":"b","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.134:6836/2781656864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6833","nonce":2781656864},{"type":"v1","addr":"172.21.15.134:6836","nonce":2781656864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4287":{"gid":4287,"name":"f","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.134:6835/1993918351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1993918351},{"type":"v1","addr":"172.21.15.134:6835","nonce":1993918351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4290":{"gid":4290,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.134:6837/2662003420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2662003420},{"type":"v1","addr":"172.21.15.134:6837","nonce":2662003420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4290,"qdb_cluster":[4266,4269,4284,4287,4290]},"id":1}]} 2024-04-25T10:35:18.105 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 17 2024-04-25T10:35:18.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.104+0000 7f397daf2640 1 -- 172.21.15.106:0/3717371752 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f39640744b0 msgr2=0x7f39640769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:18.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.104+0000 7f397daf2640 1 --2- 172.21.15.106:0/3717371752 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f39640744b0 0x7f39640769c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3968000f60 tx=0x7f3968009040 comp rx=0 tx=0).stop 2024-04-25T10:35:18.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.104+0000 7f397daf2640 1 -- 172.21.15.106:0/3717371752 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3978073db0 msgr2=0x7f39780cd3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:18.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.104+0000 7f397daf2640 1 --2- 172.21.15.106:0/3717371752 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3978073db0 0x7f39780cd3c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f397000cc00 tx=0x7f397000a430 comp rx=0 tx=0).stop 2024-04-25T10:35:18.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.104+0000 7f397daf2640 1 -- 172.21.15.106:0/3717371752 shutdown_connections 2024-04-25T10:35:18.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.104+0000 7f397daf2640 1 --2- 172.21.15.106:0/3717371752 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f39640744b0 0x7f39640769c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:18.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.104+0000 7f397daf2640 1 --2- 172.21.15.106:0/3717371752 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3978073db0 0x7f39780cd3c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:18.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.104+0000 7f397daf2640 1 --2- 172.21.15.106:0/3717371752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3978073020 0x7f39781f0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:18.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.104+0000 7f397daf2640 1 --2- 172.21.15.106:0/3717371752 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3978068600 0x7f397814c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:18.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.104+0000 7f397daf2640 1 -- 172.21.15.106:0/3717371752 >> 172.21.15.106:0/3717371752 conn(0x7f39780ce570 msgr2=0x7f3978073430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:18.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.104+0000 7f397daf2640 1 -- 172.21.15.106:0/3717371752 shutdown_connections 2024-04-25T10:35:18.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.104+0000 7f397daf2640 1 -- 172.21.15.106:0/3717371752 wait complete. 2024-04-25T10:35:18.118 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:35:18.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.225+0000 7fce5199d640 1 Processor -- start 2024-04-25T10:35:18.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.225+0000 7fce5199d640 1 -- start start 2024-04-25T10:35:18.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.226+0000 7fce5199d640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fce4c153bd0 0x7fce4c0c96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:18.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.226+0000 7fce5199d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fce4c073a00 0x7fce4c073e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:18.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.226+0000 7fce5199d640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce4c1596a0 con 0x7fce4c0c8e90 2024-04-25T10:35:18.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.226+0000 7fce5199d640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7fce4c159920 con 0x7fce4c153bd0 2024-04-25T10:35:18.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.226+0000 7fce5199d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fce4c159c00 con 0x7fce4c073a00 2024-04-25T10:35:18.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.226+0000 7fce5119c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fce4c073a00 0x7fce4c073e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:18.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.226+0000 7fce5119c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fce4c073a00 0x7fce4c073e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.106:49974/0 (socket says 172.21.15.106:49974) 2024-04-25T10:35:18.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.226+0000 7fce5119c640 1 -- 172.21.15.106:0/892885482 learned_addr learned my addr 172.21.15.106:0/892885482 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:35:18.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.226+0000 7fce4bfff640 1 --2- 172.21.15.106:0/892885482 >> v2:172.21.15.134:3300/0 conn(0x7fce4c153bd0 0x7fce4c0c96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:18.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.227+0000 7fce5119c640 1 -- 172.21.15.106:0/892885482 >> v2:172.21.15.134:3300/0 conn(0x7fce4c153bd0 msgr2=0x7fce4c0c96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:18.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.227+0000 7fce5119c640 1 --2- 172.21.15.106:0/892885482 >> v2:172.21.15.134:3300/0 conn(0x7fce4c153bd0 0x7fce4c0c96b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:18.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.227+0000 7fce5119c640 1 -- 172.21.15.106:0/892885482 >> v1:172.21.15.134:6789/0 conn(0x7fce4c0c8e90 legacy=0x7fce4c1534c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:18.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.227+0000 7fce5119c640 1 -- 172.21.15.106:0/892885482 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce4c159ee0 con 0x7fce4c073a00 2024-04-25T10:35:18.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.227+0000 7fce4bfff640 1 --2- 172.21.15.106:0/892885482 >> v2:172.21.15.134:3300/0 conn(0x7fce4c153bd0 0x7fce4c0c96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:35:18.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.227+0000 7fce5119c640 1 --2- 172.21.15.106:0/892885482 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fce4c073a00 0x7fce4c073e10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fce44009e50 tx=0x7fce4402a7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0ecf7dfc759b75d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:18.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.228+0000 7fce4b7fe640 1 -- 172.21.15.106:0/892885482 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce44036070 con 0x7fce4c073a00 2024-04-25T10:35:18.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.228+0000 7fce4b7fe640 1 -- 172.21.15.106:0/892885482 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fce44031600 con 0x7fce4c073a00 2024-04-25T10:35:18.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.228+0000 7fce4b7fe640 1 -- 172.21.15.106:0/892885482 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce440319f0 con 0x7fce4c073a00 2024-04-25T10:35:18.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.228+0000 7fce5199d640 1 -- 172.21.15.106:0/892885482 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fce4c073a00 msgr2=0x7fce4c073e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:18.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.228+0000 7fce5199d640 1 --2- 172.21.15.106:0/892885482 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fce4c073a00 0x7fce4c073e10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fce44009e50 tx=0x7fce4402a7a0 comp rx=0 tx=0).stop 2024-04-25T10:35:18.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.228+0000 7fce5199d640 1 -- 172.21.15.106:0/892885482 shutdown_connections 2024-04-25T10:35:18.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.228+0000 7fce5199d640 1 --2- 172.21.15.106:0/892885482 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fce4c073a00 0x7fce4c073e10 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:18.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.228+0000 7fce5199d640 1 --2- 172.21.15.106:0/892885482 >> v2:172.21.15.134:3300/0 conn(0x7fce4c153bd0 0x7fce4c0c96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:18.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.228+0000 7fce5199d640 1 -- 172.21.15.106:0/892885482 >> 172.21.15.106:0/892885482 conn(0x7fce4c0713d0 msgr2=0x7fce4c0717e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:18.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.228+0000 7fce5199d640 1 -- 172.21.15.106:0/892885482 shutdown_connections 2024-04-25T10:35:18.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.228+0000 7fce5199d640 1 -- 172.21.15.106:0/892885482 wait complete. 2024-04-25T10:35:18.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.230+0000 7fce5199d640 1 Processor -- start 2024-04-25T10:35:18.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.230+0000 7fce5199d640 1 -- start start 2024-04-25T10:35:18.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.231+0000 7fce5199d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce4c073a00 0x7fce4c14ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:18.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.231+0000 7fce5199d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fce4c0c8e90 0x7fce4c14c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:18.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.231+0000 7fce5199d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce4c1e5fe0 0x7fce4c150c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:18.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.231+0000 7fce5199d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fce4c074350 con 0x7fce4c073a00 2024-04-25T10:35:18.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.231+0000 7fce5199d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fce4c15a4a0 con 0x7fce4c1e5fe0 2024-04-25T10:35:18.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.231+0000 7fce5199d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fce4c0c6450 con 0x7fce4c0c8e90 2024-04-25T10:35:18.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.232+0000 7fce5099b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce4c073a00 0x7fce4c14ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:18.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.232+0000 7fce5099b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce4c073a00 0x7fce4c14ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38842/0 (socket says 172.21.15.106:38842) 2024-04-25T10:35:18.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.232+0000 7fce5099b640 1 -- 172.21.15.106:0/1045842484 learned_addr learned my addr 172.21.15.106:0/1045842484 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:35:18.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.232+0000 7fce4bfff640 1 --2- 172.21.15.106:0/1045842484 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fce4c0c8e90 0x7fce4c14c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:18.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.232+0000 7fce5119c640 1 --2- 172.21.15.106:0/1045842484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce4c1e5fe0 0x7fce4c150c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:18.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.232+0000 7fce5099b640 1 -- 172.21.15.106:0/1045842484 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fce4c0c8e90 msgr2=0x7fce4c14c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:18.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.232+0000 7fce5099b640 1 --2- 172.21.15.106:0/1045842484 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fce4c0c8e90 0x7fce4c14c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:18.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.232+0000 7fce5099b640 1 -- 172.21.15.106:0/1045842484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce4c1e5fe0 msgr2=0x7fce4c150c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:18.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.232+0000 7fce5099b640 1 --2- 172.21.15.106:0/1045842484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce4c1e5fe0 0x7fce4c150c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:18.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.232+0000 7fce5099b640 1 -- 172.21.15.106:0/1045842484 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce4c14cc40 con 0x7fce4c073a00 2024-04-25T10:35:18.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.232+0000 7fce4bfff640 1 --2- 172.21.15.106:0/1045842484 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fce4c0c8e90 0x7fce4c14c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:35:18.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.232+0000 7fce5099b640 1 --2- 172.21.15.106:0/1045842484 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce4c073a00 0x7fce4c14ba70 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fce4000c3f0 tx=0x7fce4000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:18.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.232+0000 7fce497fa640 1 -- 172.21.15.106:0/1045842484 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce400052d0 con 0x7fce4c073a00 2024-04-25T10:35:18.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.232+0000 7fce497fa640 1 -- 172.21.15.106:0/1045842484 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fce40005490 con 0x7fce4c073a00 2024-04-25T10:35:18.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.232+0000 7fce5199d640 1 -- 172.21.15.106:0/1045842484 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce4c1f5d00 con 0x7fce4c073a00 2024-04-25T10:35:18.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.232+0000 7fce497fa640 1 -- 172.21.15.106:0/1045842484 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce40007280 con 0x7fce4c073a00 2024-04-25T10:35:18.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.232+0000 7fce5199d640 1 -- 172.21.15.106:0/1045842484 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce4c1f62c0 con 0x7fce4c073a00 2024-04-25T10:35:18.237 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.233+0000 7fce5199d640 1 -- 172.21.15.106:0/1045842484 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce4c050740 con 0x7fce4c073a00 2024-04-25T10:35:18.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.234+0000 7fce497fa640 1 -- 172.21.15.106:0/1045842484 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fce40007420 con 0x7fce4c073a00 2024-04-25T10:35:18.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.234+0000 7fce497fa640 1 --2- 172.21.15.106:0/1045842484 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fce300744b0 0x7fce300769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:18.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.235+0000 7fce497fa640 1 -- 172.21.15.106:0/1045842484 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fce400947f0 con 0x7fce4c073a00 2024-04-25T10:35:18.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.235+0000 7fce4bfff640 1 --2- 172.21.15.106:0/1045842484 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fce300744b0 0x7fce300769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:18.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.235+0000 7fce4bfff640 1 --2- 172.21.15.106:0/1045842484 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fce300744b0 0x7fce300769c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fce4c06d600 tx=0x7fce38008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:18.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.239+0000 7fce497fa640 1 -- 172.21.15.106:0/1045842484 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fce4c1596a0 con 0x7fce4c073a00 2024-04-25T10:35:18.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.415+0000 7fce5199d640 1 -- 172.21.15.106:0/1045842484 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fce4c152bf0 con 0x7fce4c073a00 2024-04-25T10:35:18.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.416+0000 7fce497fa640 1 -- 172.21.15.106:0/1045842484 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16890 (secure 0 0 0) 0x7fce4005ee80 con 0x7fce4c073a00 2024-04-25T10:35:18.419 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T10:35:18.419 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":22,"fsid":"3ee77985-e8e4-41cf-81e9-7c774cf7891f","created":"2024-04-25T10:30:07.981054+0000","modified":"2024-04-25T10:31:13.621853+0000","last_up_change":"2024-04-25T10:30:19.184325+0000","last_in_change":"2024-04-25T10:30:10.217553+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:30:20.111046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:30:30.314914+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:30:30.715500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a5962cb1-dbbd-47b1-8d15-9bae114bfdf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1240717079},{"type":"v1","addr":"172.21.15.106:6801","nonce":1240717079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1240717079},{"type":"v1","addr":"172.21.15.106:6803","nonce":1240717079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1240717079},{"type":"v1","addr":"172.21.15.106:6807","nonce":1240717079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1240717079},{"type":"v1","addr":"172.21.15.106:6805","nonce":1240717079}]},"public_addr":"172.21.15.106:6801/1240717079","cluster_addr":"172.21.15.106:6803/1240717079","heartbeat_back_addr":"172.21.15.106:6807/1240717079","heartbeat_front_addr":"172.21.15.106:6805/1240717079","state":["exists","up"]},{"osd":1,"uuid":"94436e13-4769-4752-8397-b5e396a6f7ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4089653051},{"type":"v1","addr":"172.21.15.106:6825","nonce":4089653051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4089653051},{"type":"v1","addr":"172.21.15.106:6827","nonce":4089653051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4089653051},{"type":"v1","addr":"172.21.15.106:6831","nonce":4089653051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4089653051},{"type":"v1","addr":"172.21.15.106:6829","nonce":4089653051}]},"public_addr":"172.21.15.106:6825/4089653051","cluster_addr":"172.21.15.106:6827/4089653051","heartbeat_back_addr":"172.21.15.106:6831/4089653051","heartbeat_front_addr":"172.21.15.106:6829/4089653051","state":["exists","up"]},{"osd":2,"uuid":"429aae0b-70c6-4e7b-9043-6a11334c7efc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":1441018239},{"type":"v1","addr":"172.21.15.106:6809","nonce":1441018239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1441018239},{"type":"v1","addr":"172.21.15.106:6811","nonce":1441018239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1441018239},{"type":"v1","addr":"172.21.15.106:6815","nonce":1441018239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":1441018239},{"type":"v1","addr":"172.21.15.106:6813","nonce":1441018239}]},"public_addr":"172.21.15.106:6809/1441018239","cluster_addr":"172.21.15.106:6811/1441018239","heartbeat_back_addr":"172.21.15.106:6815/1441018239","heartbeat_front_addr":"172.21.15.106:6813/1441018239","state":["exists","up"]},{"osd":3,"uuid":"9593a4d1-772f-4667-ab0d-2982f841e342","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":1846968711},{"type":"v1","addr":"172.21.15.106:6817","nonce":1846968711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1846968711},{"type":"v1","addr":"172.21.15.106:6819","nonce":1846968711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":1846968711},{"type":"v1","addr":"172.21.15.106:6823","nonce":1846968711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":1846968711},{"type":"v1","addr":"172.21.15.106:6821","nonce":1846968711}]},"public_addr":"172.21.15.106:6817/1846968711","cluster_addr":"172.21.15.106:6819/1846968711","heartbeat_back_addr":"172.21.15.106:6823/1846968711","heartbeat_front_addr":"172.21.15.106:6821/1846968711","state":["exists","up"]},{"osd":4,"uuid":"1b61f16d-5c60-480a-b449-c745aa3f6b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":729224739},{"type":"v1","addr":"172.21.15.134:6825","nonce":729224739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":729224739},{"type":"v1","addr":"172.21.15.134:6827","nonce":729224739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":729224739},{"type":"v1","addr":"172.21.15.134:6831","nonce":729224739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":729224739},{"type":"v1","addr":"172.21.15.134:6829","nonce":729224739}]},"public_addr":"172.21.15.134:6825/729224739","cluster_addr":"172.21.15.134:6827/729224739","heartbeat_back_addr":"172.21.15.134:6831/729224739","heartbeat_front_addr":"172.21.15.134:6829/729224739","state":["exists","up"]},{"osd":5,"uuid":"944cdd7f-47c0-4b71-b7e0-d093542f4528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3132275652},{"type":"v1","addr":"172.21.15.134:6809","nonce":3132275652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3132275652},{"type":"v1","addr":"172.21.15.134:6811","nonce":3132275652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3132275652},{"type":"v1","addr":"172.21.15.134:6815","nonce":3132275652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3132275652},{"type":"v1","addr":"172.21.15.134:6813","nonce":3132275652}]},"public_addr":"172.21.15.134:6809/3132275652","cluster_addr":"172.21.15.134:6811/3132275652","heartbeat_back_addr":"172.21.15.134:6815/3132275652","heartbeat_front_addr":"172.21.15.134:6813/3132275652","state":["exists","up"]},{"osd":6,"uuid":"cc2446e2-1d21-4135-ac04-6af7c990058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":992702668},{"type":"v1","addr":"172.21.15.134:6801","nonce":992702668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":992702668},{"type":"v1","addr":"172.21.15.134:6803","nonce":992702668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":992702668},{"type":"v1","addr":"172.21.15.134:6807","nonce":992702668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":992702668},{"type":"v1","addr":"172.21.15.134:6805","nonce":992702668}]},"public_addr":"172.21.15.134:6801/992702668","cluster_addr":"172.21.15.134:6803/992702668","heartbeat_back_addr":"172.21.15.134:6807/992702668","heartbeat_front_addr":"172.21.15.134:6805/992702668","state":["exists","up"]},{"osd":7,"uuid":"3df1fc7a-82fc-4257-8629-98027b6e2aed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":3291062284},{"type":"v1","addr":"172.21.15.134:6817","nonce":3291062284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":3291062284},{"type":"v1","addr":"172.21.15.134:6819","nonce":3291062284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":3291062284},{"type":"v1","addr":"172.21.15.134:6823","nonce":3291062284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":3291062284},{"type":"v1","addr":"172.21.15.134:6821","nonce":3291062284}]},"public_addr":"172.21.15.134:6817/3291062284","cluster_addr":"172.21.15.134:6819/3291062284","heartbeat_back_addr":"172.21.15.134:6823/3291062284","heartbeat_front_addr":"172.21.15.134:6821/3291062284","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.403941+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.382563+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.411544+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.330444+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.405430+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.387737+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.408922+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:30:15.454450+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":2}]},{"pgid":"3.2b","mappings":[{"from":0,"to":1}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:35:18.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.418+0000 7fce5199d640 1 -- 172.21.15.106:0/1045842484 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fce300744b0 msgr2=0x7fce300769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:18.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.418+0000 7fce5199d640 1 --2- 172.21.15.106:0/1045842484 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fce300744b0 0x7fce300769c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fce4c06d600 tx=0x7fce38008040 comp rx=0 tx=0).stop 2024-04-25T10:35:18.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.418+0000 7fce5199d640 1 -- 172.21.15.106:0/1045842484 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce4c073a00 msgr2=0x7fce4c14ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:18.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.418+0000 7fce5199d640 1 --2- 172.21.15.106:0/1045842484 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce4c073a00 0x7fce4c14ba70 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fce4000c3f0 tx=0x7fce4000c8b0 comp rx=0 tx=0).stop 2024-04-25T10:35:18.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.418+0000 7fce5199d640 1 -- 172.21.15.106:0/1045842484 shutdown_connections 2024-04-25T10:35:18.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.418+0000 7fce5199d640 1 --2- 172.21.15.106:0/1045842484 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7fce300744b0 0x7fce300769c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:18.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.418+0000 7fce5199d640 1 --2- 172.21.15.106:0/1045842484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce4c1e5fe0 0x7fce4c150c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:18.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.418+0000 7fce5199d640 1 --2- 172.21.15.106:0/1045842484 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fce4c0c8e90 0x7fce4c14c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:18.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.418+0000 7fce5199d640 1 --2- 172.21.15.106:0/1045842484 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce4c073a00 0x7fce4c14ba70 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:18.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.418+0000 7fce5199d640 1 -- 172.21.15.106:0/1045842484 >> 172.21.15.106:0/1045842484 conn(0x7fce4c0713d0 msgr2=0x7fce4c0cb640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:18.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.418+0000 7fce5199d640 1 -- 172.21.15.106:0/1045842484 shutdown_connections 2024-04-25T10:35:18.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.418+0000 7fce5199d640 1 -- 172.21.15.106:0/1045842484 wait complete. 2024-04-25T10:35:18.433 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:35:18.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.538+0000 7f09a73e3640 1 Processor -- start 2024-04-25T10:35:18.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.538+0000 7f09a73e3640 1 -- start start 2024-04-25T10:35:18.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.539+0000 7f09a73e3640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f09a006c670 0x7f09a00cc280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:18.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.540+0000 7f09a73e3640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09a0154ca0 0x7f09a00c8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:18.544 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.540+0000 7f09a73e3640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09a006cf10 con 0x7f09a00738d0 2024-04-25T10:35:18.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.540+0000 7f09a73e3640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f09a00cc7c0 con 0x7f09a006c670 2024-04-25T10:35:18.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.540+0000 7f09a73e3640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f09a00ca310 con 0x7f09a0154ca0 2024-04-25T10:35:18.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.540+0000 7f09a517f640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f09a00738d0 0x7f09a015b5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:58928/0 (socket says 172.21.15.106:58928) 2024-04-25T10:35:18.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.540+0000 7f09a517f640 1 -- 172.21.15.106:0/567503880 learned_addr learned my addr 172.21.15.106:0/567503880 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:35:18.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.540+0000 7f09a497e640 1 --2- 172.21.15.106:0/567503880 >> v2:172.21.15.134:3300/0 conn(0x7f09a006c670 0x7f09a00cc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:18.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.540+0000 7f09a5980640 1 --2- 172.21.15.106:0/567503880 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09a0154ca0 0x7f09a00c8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:18.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.540+0000 7f0997fff640 1 -- 172.21.15.106:0/567503880 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4222583070 0 0) 0x7f09a006cf10 con 0x7f09a00738d0 2024-04-25T10:35:18.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.540+0000 7f0997fff640 1 -- 172.21.15.106:0/567503880 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f098c003520 con 0x7f09a00738d0 2024-04-25T10:35:18.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.540+0000 7f0997fff640 1 -- 172.21.15.106:0/567503880 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3880839299 0 0) 0x7f098c003520 con 0x7f09a00738d0 2024-04-25T10:35:18.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.540+0000 7f0997fff640 1 -- 172.21.15.106:0/567503880 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09a0154ca0 msgr2=0x7f09a00c8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:18.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.540+0000 7f0997fff640 1 --2- 172.21.15.106:0/567503880 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09a0154ca0 0x7f09a00c8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:18.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.540+0000 7f0997fff640 1 -- 172.21.15.106:0/567503880 >> v2:172.21.15.134:3300/0 conn(0x7f09a006c670 msgr2=0x7f09a00cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:18.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.540+0000 7f0997fff640 1 --2- 172.21.15.106:0/567503880 >> v2:172.21.15.134:3300/0 conn(0x7f09a006c670 0x7f09a00cc280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:18.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.540+0000 7f0997fff640 1 -- 172.21.15.106:0/567503880 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09a00ca5d0 con 0x7f09a00738d0 2024-04-25T10:35:18.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.540+0000 7f0997fff640 1 -- 172.21.15.106:0/567503880 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3776459350 0 0) 0x7f099c0019f0 con 0x7f09a00738d0 2024-04-25T10:35:18.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.540+0000 7f0997fff640 1 -- 172.21.15.106:0/567503880 >> v1:172.21.15.106:6789/0 conn(0x7f09a00738d0 legacy=0x7f09a015b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:18.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.540+0000 7f0997fff640 1 --2- 172.21.15.106:0/567503880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f098c0039b0 0x7f098c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:18.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.540+0000 7f0997fff640 1 --2- 172.21.15.106:0/567503880 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09a0154ca0 0x7f098c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:18.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.540+0000 7f0997fff640 1 --2- 172.21.15.106:0/567503880 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f098c004d40 0x7f098c005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:18.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.540+0000 7f0997fff640 1 -- 172.21.15.106:0/567503880 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f09a00cc7c0 con 0x7f098c004d40 2024-04-25T10:35:18.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.540+0000 7f0997fff640 1 -- 172.21.15.106:0/567503880 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f09a00ca310 con 0x7f098c0039b0 2024-04-25T10:35:18.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.540+0000 7f0997fff640 1 -- 172.21.15.106:0/567503880 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f098c00b9c0 con 0x7f09a0154ca0 2024-04-25T10:35:18.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.540+0000 7f09a5980640 1 --2- 172.21.15.106:0/567503880 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f098c004d40 0x7f098c005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:18.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.540+0000 7f09a517f640 1 --2- 172.21.15.106:0/567503880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f098c0039b0 0x7f098c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:18.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.540+0000 7f09a497e640 1 --2- 172.21.15.106:0/567503880 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09a0154ca0 0x7f098c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:18.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.541+0000 7f09a5980640 1 -- 172.21.15.106:0/567503880 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09a0154ca0 msgr2=0x7f098c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:18.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.541+0000 7f09a5980640 1 --2- 172.21.15.106:0/567503880 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09a0154ca0 0x7f098c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:18.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.541+0000 7f09a5980640 1 -- 172.21.15.106:0/567503880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f098c0039b0 msgr2=0x7f098c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:18.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.541+0000 7f09a5980640 1 --2- 172.21.15.106:0/567503880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f098c0039b0 0x7f098c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:18.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.541+0000 7f09a5980640 1 -- 172.21.15.106:0/567503880 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f098c003520 con 0x7f098c004d40 2024-04-25T10:35:18.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.541+0000 7f09a5980640 1 --2- 172.21.15.106:0/567503880 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f098c004d40 0x7f098c005150 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0998009ac0 tx=0x7f098c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f79ee56463548caf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:18.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.541+0000 7f0997fff640 1 -- 172.21.15.106:0/567503880 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0998036070 con 0x7f098c004d40 2024-04-25T10:35:18.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.541+0000 7f0997fff640 1 -- 172.21.15.106:0/567503880 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f0998031400 con 0x7f098c004d40 2024-04-25T10:35:18.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.541+0000 7f0997fff640 1 -- 172.21.15.106:0/567503880 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0998009690 con 0x7f098c004d40 2024-04-25T10:35:18.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.542+0000 7f09a73e3640 1 -- 172.21.15.106:0/567503880 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f098c004d40 msgr2=0x7f098c005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:18.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.542+0000 7f09a73e3640 1 --2- 172.21.15.106:0/567503880 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f098c004d40 0x7f098c005150 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0998009ac0 tx=0x7f098c005830 comp rx=0 tx=0).stop 2024-04-25T10:35:18.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.542+0000 7f09a5980640 1 -- 172.21.15.106:0/567503880 reap_dead start 2024-04-25T10:35:18.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.542+0000 7f09a73e3640 1 -- 172.21.15.106:0/567503880 shutdown_connections 2024-04-25T10:35:18.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.542+0000 7f09a73e3640 1 -- 172.21.15.106:0/567503880 >> 172.21.15.106:0/567503880 conn(0x7f09a006d6d0 msgr2=0x7f09a006dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:18.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.542+0000 7f09a73e3640 1 -- 172.21.15.106:0/567503880 shutdown_connections 2024-04-25T10:35:18.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.542+0000 7f09a73e3640 1 -- 172.21.15.106:0/567503880 wait complete. 2024-04-25T10:35:18.549 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.544+0000 7f09a73e3640 1 Processor -- start 2024-04-25T10:35:18.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.544+0000 7f09a73e3640 1 -- start start 2024-04-25T10:35:18.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.545+0000 7f09a73e3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09a01560d0 0x7f09a01564e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:18.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.545+0000 7f09a73e3640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09a014a710 0x7f09a014ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:18.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.545+0000 7f09a73e3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09a01f40a0 0x7f09a01f44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:18.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.545+0000 7f09a73e3640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f09a00ca8b0 con 0x7f09a01f40a0 2024-04-25T10:35:18.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.545+0000 7f09a73e3640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f09a00cba80 con 0x7f09a01560d0 2024-04-25T10:35:18.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.545+0000 7f09a73e3640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f09a006ca80 con 0x7f09a014a710 2024-04-25T10:35:18.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.545+0000 7f09a5980640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09a01f40a0 0x7f09a01f44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:18.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.545+0000 7f09a5980640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09a01f40a0 0x7f09a01f44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38860/0 (socket says 172.21.15.106:38860) 2024-04-25T10:35:18.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.545+0000 7f09a5980640 1 -- 172.21.15.106:0/727408876 learned_addr learned my addr 172.21.15.106:0/727408876 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:35:18.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.545+0000 7f09a517f640 1 --2- 172.21.15.106:0/727408876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09a01560d0 0x7f09a01564e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:18.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.545+0000 7f09a497e640 1 --2- 172.21.15.106:0/727408876 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09a014a710 0x7f09a014ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:18.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.546+0000 7f09a5980640 1 -- 172.21.15.106:0/727408876 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09a014a710 msgr2=0x7f09a014ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:18.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.546+0000 7f09a5980640 1 --2- 172.21.15.106:0/727408876 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09a014a710 0x7f09a014ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:18.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.546+0000 7f09a5980640 1 -- 172.21.15.106:0/727408876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09a01560d0 msgr2=0x7f09a01564e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:18.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.546+0000 7f09a5980640 1 --2- 172.21.15.106:0/727408876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09a01560d0 0x7f09a01564e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:18.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.546+0000 7f09a5980640 1 -- 172.21.15.106:0/727408876 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09a0157e90 con 0x7f09a01f40a0 2024-04-25T10:35:18.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.546+0000 7f09a5980640 1 --2- 172.21.15.106:0/727408876 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09a01f40a0 0x7f09a01f44b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0998009ac0 tx=0x7f0998004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:18.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.546+0000 7f0995ffb640 1 -- 172.21.15.106:0/727408876 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0998036070 con 0x7f09a01f40a0 2024-04-25T10:35:18.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.546+0000 7f0995ffb640 1 -- 172.21.15.106:0/727408876 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f0998041070 con 0x7f09a01f40a0 2024-04-25T10:35:18.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.546+0000 7f09a73e3640 1 -- 172.21.15.106:0/727408876 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09a0158170 con 0x7f09a01f40a0 2024-04-25T10:35:18.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.546+0000 7f0995ffb640 1 -- 172.21.15.106:0/727408876 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0998009690 con 0x7f09a01f40a0 2024-04-25T10:35:18.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.546+0000 7f09a73e3640 1 -- 172.21.15.106:0/727408876 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09a01558d0 con 0x7f09a01f40a0 2024-04-25T10:35:18.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.547+0000 7f09a73e3640 1 -- 172.21.15.106:0/727408876 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09a006a3d0 con 0x7f09a01f40a0 2024-04-25T10:35:18.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.548+0000 7f0995ffb640 1 -- 172.21.15.106:0/727408876 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f099803b840 con 0x7f09a01f40a0 2024-04-25T10:35:18.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.548+0000 7f0995ffb640 1 --2- 172.21.15.106:0/727408876 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f09840744b0 0x7f09840769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:18.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.548+0000 7f0995ffb640 1 -- 172.21.15.106:0/727408876 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f09980b53a0 con 0x7f09a01f40a0 2024-04-25T10:35:18.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.548+0000 7f09a517f640 1 --2- 172.21.15.106:0/727408876 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f09840744b0 0x7f09840769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:18.552 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.549+0000 7f09a517f640 1 --2- 172.21.15.106:0/727408876 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f09840744b0 0x7f09840769c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f099c000f80 tx=0x7f099c006040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:18.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.553+0000 7f0995ffb640 1 -- 172.21.15.106:0/727408876 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f099807f520 con 0x7f09a01f40a0 2024-04-25T10:35:18.769 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.767+0000 7f09a73e3640 1 -- 172.21.15.106:0/727408876 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f09a00c9ee0 con 0x7f09a01f40a0 2024-04-25T10:35:18.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.769+0000 7f0995ffb640 1 -- 172.21.15.106:0/727408876 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) ==== 161+0+0 (secure 0 0 0) 0x7f0998083820 con 0x7f09a01f40a0 2024-04-25T10:35:18.771 INFO:teuthology.orchestra.run.smithi106.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:35:18.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.772+0000 7f09a73e3640 1 -- 172.21.15.106:0/727408876 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f09840744b0 msgr2=0x7f09840769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:18.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.772+0000 7f09a73e3640 1 --2- 172.21.15.106:0/727408876 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f09840744b0 0x7f09840769c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f099c000f80 tx=0x7f099c006040 comp rx=0 tx=0).stop 2024-04-25T10:35:18.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.772+0000 7f09a73e3640 1 -- 172.21.15.106:0/727408876 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09a01f40a0 msgr2=0x7f09a01f44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:18.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.772+0000 7f09a73e3640 1 --2- 172.21.15.106:0/727408876 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09a01f40a0 0x7f09a01f44b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0998009ac0 tx=0x7f0998004730 comp rx=0 tx=0).stop 2024-04-25T10:35:18.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.773+0000 7f09a73e3640 1 -- 172.21.15.106:0/727408876 shutdown_connections 2024-04-25T10:35:18.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.773+0000 7f09a73e3640 1 --2- 172.21.15.106:0/727408876 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f09840744b0 0x7f09840769c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:18.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.773+0000 7f09a73e3640 1 --2- 172.21.15.106:0/727408876 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09a01f40a0 0x7f09a01f44b0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:18.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.773+0000 7f09a73e3640 1 --2- 172.21.15.106:0/727408876 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09a014a710 0x7f09a014ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:18.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.773+0000 7f09a73e3640 1 --2- 172.21.15.106:0/727408876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09a01560d0 0x7f09a01564e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:18.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.773+0000 7f09a73e3640 1 -- 172.21.15.106:0/727408876 >> 172.21.15.106:0/727408876 conn(0x7f09a006d6d0 msgr2=0x7f09a015af00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:18.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.774+0000 7f09a73e3640 1 -- 172.21.15.106:0/727408876 shutdown_connections 2024-04-25T10:35:18.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.774+0000 7f09a73e3640 1 -- 172.21.15.106:0/727408876 wait complete. 2024-04-25T10:35:18.789 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:35:18.837 INFO:tasks.ceph.mds.e.smithi106.stdout:starting mds.e at 2024-04-25T10:35:18.837 INFO:tasks.ceph.mds.c.smithi106.stdout:starting mds.c at 2024-04-25T10:35:18.843 INFO:tasks.ceph.mds.b.smithi134.stdout:starting mds.b at 2024-04-25T10:35:18.844 INFO:tasks.ceph.mds.f.smithi134.stdout:starting mds.f at 2024-04-25T10:35:18.844 INFO:tasks.ceph.mds.d.smithi134.stdout:starting mds.d at 2024-04-25T10:35:18.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.904+0000 7f52ffd46640 1 Processor -- start 2024-04-25T10:35:18.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.905+0000 7f52ffd46640 1 -- start start 2024-04-25T10:35:18.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.906+0000 7f52ffd46640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f52f80c7ad0 0x7f52f806ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:18.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.906+0000 7f52ffd46640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f52f8156f70 0x7f52f806d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:18.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.906+0000 7f52ffd46640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52f815d6d0 con 0x7f52f806daa0 2024-04-25T10:35:18.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.906+0000 7f52ffd46640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f52f815d9b0 con 0x7f52f8156f70 2024-04-25T10:35:18.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.906+0000 7f52ffd46640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f52f80c7150 con 0x7f52f80c7ad0 2024-04-25T10:35:18.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.906+0000 7f52fdae2640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f52f80c7ad0 0x7f52f806ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:18.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.906+0000 7f52fd2e1640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f52f8156f70 0x7f52f806d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:18.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.906+0000 7f52fdae2640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f52f80c7ad0 0x7f52f806ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.106:50040/0 (socket says 172.21.15.106:50040) 2024-04-25T10:35:18.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.906+0000 7f52fdae2640 1 -- 172.21.15.106:0/1735498271 learned_addr learned my addr 172.21.15.106:0/1735498271 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:35:18.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.907+0000 7f52fdae2640 1 -- 172.21.15.106:0/1735498271 >> v2:172.21.15.134:3300/0 conn(0x7f52f8156f70 msgr2=0x7f52f806d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:18.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.907+0000 7f52fdae2640 1 --2- 172.21.15.106:0/1735498271 >> v2:172.21.15.134:3300/0 conn(0x7f52f8156f70 0x7f52f806d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:18.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.907+0000 7f52fdae2640 1 -- 172.21.15.106:0/1735498271 >> v1:172.21.15.134:6789/0 conn(0x7f52f806daa0 legacy=0x7f52f80c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:18.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.907+0000 7f52fdae2640 1 -- 172.21.15.106:0/1735498271 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52f80c7430 con 0x7f52f80c7ad0 2024-04-25T10:35:18.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.907+0000 7f52fdae2640 1 --2- 172.21.15.106:0/1735498271 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f52f80c7ad0 0x7f52f806ea60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f52e8009870 tx=0x7f52e802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25084c03734c6ccc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:18.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.907+0000 7f52fcae0640 1 -- 172.21.15.106:0/1735498271 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52e8037070 con 0x7f52f80c7ad0 2024-04-25T10:35:18.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.907+0000 7f52fcae0640 1 -- 172.21.15.106:0/1735498271 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f52e8032440 con 0x7f52f80c7ad0 2024-04-25T10:35:18.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.907+0000 7f52fcae0640 1 -- 172.21.15.106:0/1735498271 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52e8032830 con 0x7f52f80c7ad0 2024-04-25T10:35:18.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.908+0000 7f52ffd46640 1 -- 172.21.15.106:0/1735498271 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f52f80c7ad0 msgr2=0x7f52f806ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:18.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.908+0000 7f52ffd46640 1 --2- 172.21.15.106:0/1735498271 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f52f80c7ad0 0x7f52f806ea60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f52e8009870 tx=0x7f52e802a270 comp rx=0 tx=0).stop 2024-04-25T10:35:18.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.908+0000 7f52ffd46640 1 -- 172.21.15.106:0/1735498271 shutdown_connections 2024-04-25T10:35:18.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.908+0000 7f52ffd46640 1 --2- 172.21.15.106:0/1735498271 >> v2:172.21.15.134:3300/0 conn(0x7f52f8156f70 0x7f52f806d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:18.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.908+0000 7f52ffd46640 1 --2- 172.21.15.106:0/1735498271 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f52f80c7ad0 0x7f52f806ea60 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:18.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.908+0000 7f52ffd46640 1 -- 172.21.15.106:0/1735498271 >> 172.21.15.106:0/1735498271 conn(0x7f52f8068ed0 msgr2=0x7f52f80d20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:18.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.908+0000 7f52ffd46640 1 -- 172.21.15.106:0/1735498271 shutdown_connections 2024-04-25T10:35:18.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.908+0000 7f52ffd46640 1 -- 172.21.15.106:0/1735498271 wait complete. 2024-04-25T10:35:18.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.910+0000 7f52ffd46640 1 Processor -- start 2024-04-25T10:35:18.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.910+0000 7f52ffd46640 1 -- start start 2024-04-25T10:35:18.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.911+0000 7f52ffd46640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52f806daa0 0x7f52f81e8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:18.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.911+0000 7f52ffd46640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f52f80c7ad0 0x7f52f81f6a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:18.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.911+0000 7f52ffd46640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f52f8156f70 0x7f52f815ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:18.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.911+0000 7f52ffd46640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f52f80ce6c0 con 0x7f52f8156f70 2024-04-25T10:35:18.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.911+0000 7f52ffd46640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f52f80ce360 con 0x7f52f806daa0 2024-04-25T10:35:18.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.911+0000 7f52ffd46640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f52f80c8830 con 0x7f52f80c7ad0 2024-04-25T10:35:18.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.911+0000 7f52fe2e3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f52f8156f70 0x7f52f815ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:18.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.911+0000 7f52fe2e3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f52f8156f70 0x7f52f815ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38890/0 (socket says 172.21.15.106:38890) 2024-04-25T10:35:18.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.911+0000 7f52fe2e3640 1 -- 172.21.15.106:0/1967566508 learned_addr learned my addr 172.21.15.106:0/1967566508 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:35:18.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.911+0000 7f52fdae2640 1 --2- 172.21.15.106:0/1967566508 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52f806daa0 0x7f52f81e8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:18.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.911+0000 7f52fd2e1640 1 --2- 172.21.15.106:0/1967566508 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f52f80c7ad0 0x7f52f81f6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:18.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.912+0000 7f52fe2e3640 1 -- 172.21.15.106:0/1967566508 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f52f80c7ad0 msgr2=0x7f52f81f6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:18.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.912+0000 7f52fe2e3640 1 --2- 172.21.15.106:0/1967566508 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f52f80c7ad0 0x7f52f81f6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:18.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.912+0000 7f52fe2e3640 1 -- 172.21.15.106:0/1967566508 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52f806daa0 msgr2=0x7f52f81e8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:18.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.912+0000 7f52fe2e3640 1 --2- 172.21.15.106:0/1967566508 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52f806daa0 0x7f52f81e8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:18.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.912+0000 7f52fe2e3640 1 -- 172.21.15.106:0/1967566508 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52f81f61e0 con 0x7f52f8156f70 2024-04-25T10:35:18.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.912+0000 7f52fe2e3640 1 --2- 172.21.15.106:0/1967566508 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f52f8156f70 0x7f52f815ae10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f52f000c680 tx=0x7f52f000cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:18.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.912+0000 7f52ee7fc640 1 -- 172.21.15.106:0/1967566508 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f0003b70 con 0x7f52f8156f70 2024-04-25T10:35:18.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.912+0000 7f52ee7fc640 1 -- 172.21.15.106:0/1967566508 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f52f0003d10 con 0x7f52f8156f70 2024-04-25T10:35:18.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.912+0000 7f52ffd46640 1 -- 172.21.15.106:0/1967566508 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52f81e8d60 con 0x7f52f8156f70 2024-04-25T10:35:18.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.912+0000 7f52ee7fc640 1 -- 172.21.15.106:0/1967566508 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f0005330 con 0x7f52f8156f70 2024-04-25T10:35:18.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.912+0000 7f52ffd46640 1 -- 172.21.15.106:0/1967566508 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52f81e8340 con 0x7f52f8156f70 2024-04-25T10:35:18.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.913+0000 7f52ffd46640 1 -- 172.21.15.106:0/1967566508 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52f80d2240 con 0x7f52f8156f70 2024-04-25T10:35:18.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.914+0000 7f52ee7fc640 1 -- 172.21.15.106:0/1967566508 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f52f00054d0 con 0x7f52f8156f70 2024-04-25T10:35:18.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.914+0000 7f52ee7fc640 1 --2- 172.21.15.106:0/1967566508 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f52dc0744e0 0x7f52dc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:18.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.914+0000 7f52ee7fc640 1 -- 172.21.15.106:0/1967566508 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 6255+0+0 (secure 0 0 0) 0x7f52f00950e0 con 0x7f52f8156f70 2024-04-25T10:35:18.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.914+0000 7f52fdae2640 1 --2- 172.21.15.106:0/1967566508 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f52dc0744e0 0x7f52dc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:18.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.915+0000 7f52fdae2640 1 --2- 172.21.15.106:0/1967566508 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f52dc0744e0 0x7f52dc0769f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f52e8034000 tx=0x7f52e802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:18.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:18.919+0000 7f52ee7fc640 1 -- 172.21.15.106:0/1967566508 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f52f005f000 con 0x7f52f8156f70 2024-04-25T10:35:19.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.135+0000 7f52ffd46640 1 -- 172.21.15.106:0/1967566508 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f52f81f63b0 con 0x7f52f8156f70 2024-04-25T10:35:19.773 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.771+0000 7f52ee7fc640 1 -- 172.21.15.106:0/1967566508 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v19) ==== 104+0+0 (secure 0 0 0) 0x7f52f0063300 con 0x7f52f8156f70 2024-04-25T10:35:19.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.774+0000 7f52ffd46640 1 -- 172.21.15.106:0/1967566508 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f52dc0744e0 msgr2=0x7f52dc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:19.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.774+0000 7f52ffd46640 1 --2- 172.21.15.106:0/1967566508 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f52dc0744e0 0x7f52dc0769f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f52e8034000 tx=0x7f52e802e000 comp rx=0 tx=0).stop 2024-04-25T10:35:19.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.774+0000 7f52ffd46640 1 -- 172.21.15.106:0/1967566508 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f52f8156f70 msgr2=0x7f52f815ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:19.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.774+0000 7f52ffd46640 1 --2- 172.21.15.106:0/1967566508 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f52f8156f70 0x7f52f815ae10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f52f000c680 tx=0x7f52f000cb40 comp rx=0 tx=0).stop 2024-04-25T10:35:19.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.775+0000 7f52ffd46640 1 -- 172.21.15.106:0/1967566508 shutdown_connections 2024-04-25T10:35:19.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.775+0000 7f52ffd46640 1 --2- 172.21.15.106:0/1967566508 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f52dc0744e0 0x7f52dc0769f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:19.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.775+0000 7f52ffd46640 1 --2- 172.21.15.106:0/1967566508 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f52f8156f70 0x7f52f815ae10 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:19.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.775+0000 7f52ffd46640 1 --2- 172.21.15.106:0/1967566508 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f52f80c7ad0 0x7f52f81f6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:19.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.775+0000 7f52ffd46640 1 --2- 172.21.15.106:0/1967566508 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52f806daa0 0x7f52f81e8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:19.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.775+0000 7f52ffd46640 1 -- 172.21.15.106:0/1967566508 >> 172.21.15.106:0/1967566508 conn(0x7f52f8068ed0 msgr2=0x7f52f81547a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:19.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.775+0000 7f52ffd46640 1 -- 172.21.15.106:0/1967566508 shutdown_connections 2024-04-25T10:35:19.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.775+0000 7f52ffd46640 1 -- 172.21.15.106:0/1967566508 wait complete. 2024-04-25T10:35:19.793 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:35:19.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.908+0000 7ff012884640 1 Processor -- start 2024-04-25T10:35:19.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.908+0000 7ff012884640 1 -- start start 2024-04-25T10:35:19.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.909+0000 7ff012884640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff00c15ae60 0x7ff00c153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:19.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.909+0000 7ff012884640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff00c0c8b50 0x7ff00c0c8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:19.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.909+0000 7ff012884640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff00c0c7000 con 0x7ff00c15a470 2024-04-25T10:35:19.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.909+0000 7ff012884640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7ff00c0740c0 con 0x7ff00c0c8b50 2024-04-25T10:35:19.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.909+0000 7ff012884640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff00c0c72e0 con 0x7ff00c15ae60 2024-04-25T10:35:19.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.909+0000 7ff010e21640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff00c0c8b50 0x7ff00c0c8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:19.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.909+0000 7ff00bfff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff00c15ae60 0x7ff00c153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:19.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.909+0000 7ff00b7fe640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7ff00c15a470 0x7ff00c074550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.106:52850/0 (socket says 172.21.15.106:52850) 2024-04-25T10:35:19.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.909+0000 7ff00b7fe640 1 -- 172.21.15.106:0/231029803 learned_addr learned my addr 172.21.15.106:0/231029803 (peer_addr_for_me v1:172.21.15.106:0/0) 2024-04-25T10:35:19.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.909+0000 7ff010e21640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff00c0c8b50 0x7ff00c0c8f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.106:58438/0 (socket says 172.21.15.106:58438) 2024-04-25T10:35:19.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.909+0000 7ff00bfff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff00c15ae60 0x7ff00c153dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.106:50062/0 (socket says 172.21.15.106:50062) 2024-04-25T10:35:19.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.910+0000 7ff00bfff640 1 -- 172.21.15.106:0/231029803 >> v2:172.21.15.134:3300/0 conn(0x7ff00c0c8b50 msgr2=0x7ff00c0c8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:19.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.910+0000 7ff00bfff640 1 --2- 172.21.15.106:0/231029803 >> v2:172.21.15.134:3300/0 conn(0x7ff00c0c8b50 0x7ff00c0c8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:19.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.910+0000 7ff00bfff640 1 -- 172.21.15.106:0/231029803 >> v1:172.21.15.134:6789/0 conn(0x7ff00c15a470 legacy=0x7ff00c074550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:19.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.910+0000 7ff00bfff640 1 -- 172.21.15.106:0/231029803 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff00c073a70 con 0x7ff00c15ae60 2024-04-25T10:35:19.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.910+0000 7ff010e21640 1 --2- 172.21.15.106:0/231029803 >> v2:172.21.15.134:3300/0 conn(0x7ff00c0c8b50 0x7ff00c0c8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:35:19.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.910+0000 7ff00bfff640 1 --2- 172.21.15.106:0/231029803 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff00c15ae60 0x7ff00c153dd0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7feffc0097a0 tx=0x7feffc02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13f5bb4703ddcda6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:19.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.911+0000 7ff00affd640 1 -- 172.21.15.106:0/231029803 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feffc037070 con 0x7ff00c15ae60 2024-04-25T10:35:19.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.911+0000 7ff00affd640 1 -- 172.21.15.106:0/231029803 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7feffc032440 con 0x7ff00c15ae60 2024-04-25T10:35:19.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.911+0000 7ff00affd640 1 -- 172.21.15.106:0/231029803 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feffc032830 con 0x7ff00c15ae60 2024-04-25T10:35:19.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.911+0000 7ff012884640 1 -- 172.21.15.106:0/231029803 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff00c15ae60 msgr2=0x7ff00c153dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:19.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.911+0000 7ff012884640 1 --2- 172.21.15.106:0/231029803 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff00c15ae60 0x7ff00c153dd0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7feffc0097a0 tx=0x7feffc02a180 comp rx=0 tx=0).stop 2024-04-25T10:35:19.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.911+0000 7ff012884640 1 -- 172.21.15.106:0/231029803 shutdown_connections 2024-04-25T10:35:19.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.911+0000 7ff012884640 1 --2- 172.21.15.106:0/231029803 >> v2:172.21.15.134:3300/0 conn(0x7ff00c0c8b50 0x7ff00c0c8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:19.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.911+0000 7ff012884640 1 --2- 172.21.15.106:0/231029803 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff00c15ae60 0x7ff00c153dd0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:19.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.911+0000 7ff012884640 1 -- 172.21.15.106:0/231029803 >> 172.21.15.106:0/231029803 conn(0x7ff00c0cba20 msgr2=0x7ff00c0cbe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:19.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.911+0000 7ff012884640 1 -- 172.21.15.106:0/231029803 shutdown_connections 2024-04-25T10:35:19.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.911+0000 7ff012884640 1 -- 172.21.15.106:0/231029803 wait complete. 2024-04-25T10:35:19.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.913+0000 7ff012884640 1 Processor -- start 2024-04-25T10:35:19.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.913+0000 7ff012884640 1 -- start start 2024-04-25T10:35:19.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.914+0000 7ff012884640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff00c0c8b50 0x7ff00c1f24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:19.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.914+0000 7ff012884640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff00c15a470 0x7ff00c06c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:19.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.914+0000 7ff012884640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff00c15ae60 0x7ff00c1e4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:19.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.914+0000 7ff012884640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff00c0c86c0 con 0x7ff00c15ae60 2024-04-25T10:35:19.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.914+0000 7ff012884640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff00c0c6d10 con 0x7ff00c0c8b50 2024-04-25T10:35:19.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.914+0000 7ff012884640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff00c0cc540 con 0x7ff00c15a470 2024-04-25T10:35:19.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.915+0000 7ff010e21640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff00c15ae60 0x7ff00c1e4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:19.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.915+0000 7ff010e21640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff00c15ae60 0x7ff00c1e4730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38896/0 (socket says 172.21.15.106:38896) 2024-04-25T10:35:19.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.915+0000 7ff010e21640 1 -- 172.21.15.106:0/2574485270 learned_addr learned my addr 172.21.15.106:0/2574485270 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:35:19.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.915+0000 7ff00bfff640 1 --2- 172.21.15.106:0/2574485270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff00c0c8b50 0x7ff00c1f24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:19.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.915+0000 7ff00b7fe640 1 --2- 172.21.15.106:0/2574485270 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff00c15a470 0x7ff00c06c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:19.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.915+0000 7ff010e21640 1 -- 172.21.15.106:0/2574485270 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff00c15a470 msgr2=0x7ff00c06c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:19.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.915+0000 7ff010e21640 1 --2- 172.21.15.106:0/2574485270 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff00c15a470 0x7ff00c06c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:19.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.915+0000 7ff010e21640 1 -- 172.21.15.106:0/2574485270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff00c0c8b50 msgr2=0x7ff00c1f24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:19.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.915+0000 7ff010e21640 1 --2- 172.21.15.106:0/2574485270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff00c0c8b50 0x7ff00c1f24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:19.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.915+0000 7ff010e21640 1 -- 172.21.15.106:0/2574485270 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff00c1f29f0 con 0x7ff00c15ae60 2024-04-25T10:35:19.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.915+0000 7ff010e21640 1 --2- 172.21.15.106:0/2574485270 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff00c15ae60 0x7ff00c1e4730 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff004009e60 tx=0x7ff00400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:19.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.915+0000 7ff008ff9640 1 -- 172.21.15.106:0/2574485270 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff004018030 con 0x7ff00c15ae60 2024-04-25T10:35:19.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.915+0000 7ff008ff9640 1 -- 172.21.15.106:0/2574485270 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7ff00400cdf0 con 0x7ff00c15ae60 2024-04-25T10:35:19.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.915+0000 7ff008ff9640 1 -- 172.21.15.106:0/2574485270 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff004003bd0 con 0x7ff00c15ae60 2024-04-25T10:35:19.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.915+0000 7ff012884640 1 -- 172.21.15.106:0/2574485270 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff00c157dd0 con 0x7ff00c15ae60 2024-04-25T10:35:19.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.916+0000 7ff012884640 1 -- 172.21.15.106:0/2574485270 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff00c158370 con 0x7ff00c15ae60 2024-04-25T10:35:19.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.916+0000 7ff012884640 1 -- 172.21.15.106:0/2574485270 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff00c0cd430 con 0x7ff00c15ae60 2024-04-25T10:35:19.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.917+0000 7ff008ff9640 1 -- 172.21.15.106:0/2574485270 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff004005d10 con 0x7ff00c15ae60 2024-04-25T10:35:19.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.917+0000 7ff008ff9640 1 --2- 172.21.15.106:0/2574485270 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7feff80744e0 0x7feff80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:19.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.918+0000 7ff00bfff640 1 --2- 172.21.15.106:0/2574485270 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7feff80744e0 0x7feff80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:19.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.918+0000 7ff008ff9640 1 -- 172.21.15.106:0/2574485270 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 6255+0+0 (secure 0 0 0) 0x7ff004095dd0 con 0x7ff00c15ae60 2024-04-25T10:35:19.922 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.918+0000 7ff00bfff640 1 --2- 172.21.15.106:0/2574485270 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7feff80744e0 0x7feff80769f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7feffc034000 tx=0x7feffc02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:19.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:19.923+0000 7ff008ff9640 1 -- 172.21.15.106:0/2574485270 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff00c0cd430 con 0x7ff00c15ae60 2024-04-25T10:35:20.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.100+0000 7ff012884640 1 -- 172.21.15.106:0/2574485270 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff00c0c68e0 con 0x7ff00c15ae60 2024-04-25T10:35:20.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.777+0000 7ff008ff9640 1 -- 172.21.15.106:0/2574485270 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v24) ==== 181+0+0 (secure 0 0 0) 0x7ff004022070 con 0x7ff00c15ae60 2024-04-25T10:35:20.779 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:35:20.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.780+0000 7ff012884640 1 -- 172.21.15.106:0/2574485270 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7feff80744e0 msgr2=0x7feff80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:20.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.780+0000 7ff012884640 1 --2- 172.21.15.106:0/2574485270 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7feff80744e0 0x7feff80769f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7feffc034000 tx=0x7feffc02e000 comp rx=0 tx=0).stop 2024-04-25T10:35:20.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.780+0000 7ff012884640 1 -- 172.21.15.106:0/2574485270 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff00c15ae60 msgr2=0x7ff00c1e4730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:20.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.780+0000 7ff012884640 1 --2- 172.21.15.106:0/2574485270 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff00c15ae60 0x7ff00c1e4730 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff004009e60 tx=0x7ff00400c490 comp rx=0 tx=0).stop 2024-04-25T10:35:20.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.780+0000 7ff012884640 1 -- 172.21.15.106:0/2574485270 shutdown_connections 2024-04-25T10:35:20.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.780+0000 7ff012884640 1 --2- 172.21.15.106:0/2574485270 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7feff80744e0 0x7feff80769f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.780+0000 7ff012884640 1 --2- 172.21.15.106:0/2574485270 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff00c15ae60 0x7ff00c1e4730 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.780+0000 7ff012884640 1 --2- 172.21.15.106:0/2574485270 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff00c15a470 0x7ff00c06c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.780+0000 7ff012884640 1 --2- 172.21.15.106:0/2574485270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff00c0c8b50 0x7ff00c1f24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.780+0000 7ff012884640 1 -- 172.21.15.106:0/2574485270 >> 172.21.15.106:0/2574485270 conn(0x7ff00c0cba20 msgr2=0x7ff00c0ccd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:20.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.781+0000 7ff012884640 1 -- 172.21.15.106:0/2574485270 shutdown_connections 2024-04-25T10:35:20.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.781+0000 7ff012884640 1 -- 172.21.15.106:0/2574485270 wait complete. 2024-04-25T10:35:20.798 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:35:20.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.913+0000 7f7c0818f640 1 Processor -- start 2024-04-25T10:35:20.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.913+0000 7f7c0818f640 1 -- start start 2024-04-25T10:35:20.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.915+0000 7f7c0818f640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f7c0015cb50 0x7f7c00074600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.915+0000 7f7c0818f640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f7c000c9130 0x7f7c00156dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.915+0000 7f7c0818f640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c000c9d50 con 0x7f7c0006fe80 2024-04-25T10:35:20.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.915+0000 7f7c0818f640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 -- 0x7f7c000b3d10 con 0x7f7c0015cb50 2024-04-25T10:35:20.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.915+0000 7f7c0818f640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 -- 0x7f7c000ca030 con 0x7f7c000c9130 2024-04-25T10:35:20.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.915+0000 7f7c05f2b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f7c0015cb50 0x7f7c00074600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.915+0000 7f7c05f2b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f7c0015cb50 0x7f7c00074600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38900/0 (socket says 172.21.15.106:38900) 2024-04-25T10:35:20.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.915+0000 7f7c05f2b640 1 -- 172.21.15.106:0/3729480790 learned_addr learned my addr 172.21.15.106:0/3729480790 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:35:20.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.915+0000 7f7c0572a640 1 --2- 172.21.15.106:0/3729480790 >> v2:172.21.15.134:3300/0 conn(0x7f7c000c9130 0x7f7c00156dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.915+0000 7f7c05f2b640 1 -- 172.21.15.106:0/3729480790 >> v2:172.21.15.134:3300/0 conn(0x7f7c000c9130 msgr2=0x7f7c00156dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:35:20.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.915+0000 7f7c05f2b640 1 --2- 172.21.15.106:0/3729480790 >> v2:172.21.15.134:3300/0 conn(0x7f7c000c9130 0x7f7c00156dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.915+0000 7f7c05f2b640 1 -- 172.21.15.106:0/3729480790 >> v1:172.21.15.134:6789/0 conn(0x7f7c0006fe80 legacy=0x7f7c000cb650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:20.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.915+0000 7f7c05f2b640 1 -- 172.21.15.106:0/3729480790 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c0006f680 con 0x7f7c0015cb50 2024-04-25T10:35:20.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.916+0000 7f7c05f2b640 1 --2- 172.21.15.106:0/3729480790 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7c0015cb50 0x7f7c00074600 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7c000cc240 tx=0x7f7bfc02a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26be6eac25aeacc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:20.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.916+0000 7f7c04f29640 1 -- 172.21.15.106:0/3729480790 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bfc036070 con 0x7f7c0015cb50 2024-04-25T10:35:20.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.916+0000 7f7c04f29640 1 -- 172.21.15.106:0/3729480790 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f7bfc031600 con 0x7f7c0015cb50 2024-04-25T10:35:20.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.916+0000 7f7c04f29640 1 -- 172.21.15.106:0/3729480790 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bfc031a10 con 0x7f7c0015cb50 2024-04-25T10:35:20.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.917+0000 7f7c0818f640 1 -- 172.21.15.106:0/3729480790 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7c0015cb50 msgr2=0x7f7c00074600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:20.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.917+0000 7f7c0818f640 1 --2- 172.21.15.106:0/3729480790 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7c0015cb50 0x7f7c00074600 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7c000cc240 tx=0x7f7bfc02a7c0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.917+0000 7f7c0818f640 1 -- 172.21.15.106:0/3729480790 shutdown_connections 2024-04-25T10:35:20.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.917+0000 7f7c0818f640 1 --2- 172.21.15.106:0/3729480790 >> v2:172.21.15.134:3300/0 conn(0x7f7c000c9130 0x7f7c00156dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.917+0000 7f7c0818f640 1 --2- 172.21.15.106:0/3729480790 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7c0015cb50 0x7f7c00074600 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.917+0000 7f7c0818f640 1 -- 172.21.15.106:0/3729480790 >> 172.21.15.106:0/3729480790 conn(0x7f7c0006c9b0 msgr2=0x7f7c0006cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:20.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.917+0000 7f7c0818f640 1 -- 172.21.15.106:0/3729480790 shutdown_connections 2024-04-25T10:35:20.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.917+0000 7f7c0818f640 1 -- 172.21.15.106:0/3729480790 wait complete. 2024-04-25T10:35:20.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.918+0000 7f7c0818f640 1 Processor -- start 2024-04-25T10:35:20.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.918+0000 7f7c0818f640 1 -- start start 2024-04-25T10:35:20.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.920+0000 7f7c0818f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7c0006fe80 0x7f7c001f66c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.920+0000 7f7c0818f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7c000c9130 0x7f7c000b2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.920+0000 7f7c0818f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c0015cb50 0x7f7c001588e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.920+0000 7f7c0818f640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7c00070410 con 0x7f7c000c9130 2024-04-25T10:35:20.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.920+0000 7f7c0818f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7c00070290 con 0x7f7c0015cb50 2024-04-25T10:35:20.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.920+0000 7f7c0818f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f7c000cbd60 con 0x7f7c0006fe80 2024-04-25T10:35:20.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.920+0000 7f7c0572a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7c000c9130 0x7f7c000b2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.920+0000 7f7c0572a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7c000c9130 0x7f7c000b2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38904/0 (socket says 172.21.15.106:38904) 2024-04-25T10:35:20.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.920+0000 7f7c0572a640 1 -- 172.21.15.106:0/1891660781 learned_addr learned my addr 172.21.15.106:0/1891660781 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T10:35:20.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.920+0000 7f7c0572a640 1 -- 172.21.15.106:0/1891660781 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7c0006fe80 msgr2=0x7f7c001f66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:20.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.920+0000 7f7c05f2b640 1 --2- 172.21.15.106:0/1891660781 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7c0006fe80 0x7f7c001f66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.920+0000 7f7c0572a640 1 --2- 172.21.15.106:0/1891660781 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7c0006fe80 0x7f7c001f66c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.920+0000 7f7c0672c640 1 --2- 172.21.15.106:0/1891660781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c0015cb50 0x7f7c001588e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.920+0000 7f7c0572a640 1 -- 172.21.15.106:0/1891660781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c0015cb50 msgr2=0x7f7c001588e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:20.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.920+0000 7f7c0572a640 1 --2- 172.21.15.106:0/1891660781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c0015cb50 0x7f7c001588e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:20.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.920+0000 7f7c0572a640 1 -- 172.21.15.106:0/1891660781 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c00159730 con 0x7f7c000c9130 2024-04-25T10:35:20.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.920+0000 7f7c0672c640 1 --2- 172.21.15.106:0/1891660781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c0015cb50 0x7f7c001588e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:35:20.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.920+0000 7f7c0572a640 1 --2- 172.21.15.106:0/1891660781 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7c000c9130 0x7f7c000b2a00 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7bf000e870 tx=0x7f7bf000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:20.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.920+0000 7f7bf67fc640 1 -- 172.21.15.106:0/1891660781 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf0017020 con 0x7f7c000c9130 2024-04-25T10:35:20.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.920+0000 7f7bf67fc640 1 -- 172.21.15.106:0/1891660781 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f7bf000ee60 con 0x7f7c000c9130 2024-04-25T10:35:20.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.920+0000 7f7bf67fc640 1 -- 172.21.15.106:0/1891660781 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf0004480 con 0x7f7c000c9130 2024-04-25T10:35:20.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.920+0000 7f7c0818f640 1 -- 172.21.15.106:0/1891660781 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c001f6c00 con 0x7f7c000c9130 2024-04-25T10:35:20.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.920+0000 7f7c0818f640 1 -- 172.21.15.106:0/1891660781 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c000b0d40 con 0x7f7c000c9130 2024-04-25T10:35:20.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.921+0000 7f7c0818f640 1 -- 172.21.15.106:0/1891660781 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c0006a3d0 con 0x7f7c000c9130 2024-04-25T10:35:20.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.922+0000 7f7bf67fc640 1 -- 172.21.15.106:0/1891660781 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7bf001a020 con 0x7f7c000c9130 2024-04-25T10:35:20.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.923+0000 7f7bf67fc640 1 --2- 172.21.15.106:0/1891660781 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f7be40744b0 0x7f7be40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:35:20.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.923+0000 7f7c05f2b640 1 --2- 172.21.15.106:0/1891660781 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f7be40744b0 0x7f7be40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:35:20.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.923+0000 7f7bf67fc640 1 -- 172.21.15.106:0/1891660781 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5817+0+0 (secure 0 0 0) 0x7f7bf0095810 con 0x7f7c000c9130 2024-04-25T10:35:20.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.923+0000 7f7c05f2b640 1 --2- 172.21.15.106:0/1891660781 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f7be40744b0 0x7f7be40769c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc033000 tx=0x7f7bfc009000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:35:20.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:20.927+0000 7f7bf67fc640 1 -- 172.21.15.106:0/1891660781 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7bf0021020 con 0x7f7c000c9130 2024-04-25T10:35:21.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:21.109+0000 7f7c0818f640 1 -- 172.21.15.106:0/1891660781 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7c0006f0b0 con 0x7f7c000c9130 2024-04-25T10:35:21.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:21.779+0000 7f7bf67fc640 1 -- 172.21.15.106:0/1891660781 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v25) ==== 169+0+0 (secure 0 0 0) 0x7f7c00159730 con 0x7f7c000c9130 2024-04-25T10:35:21.781 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_data' removed 2024-04-25T10:35:21.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:21.786+0000 7f7c0818f640 1 -- 172.21.15.106:0/1891660781 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f7be40744b0 msgr2=0x7f7be40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:21.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:21.786+0000 7f7c0818f640 1 --2- 172.21.15.106:0/1891660781 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f7be40744b0 0x7f7be40769c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc033000 tx=0x7f7bfc009000 comp rx=0 tx=0).stop 2024-04-25T10:35:21.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:21.786+0000 7f7c0818f640 1 -- 172.21.15.106:0/1891660781 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7c000c9130 msgr2=0x7f7c000b2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:35:21.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:21.786+0000 7f7c0818f640 1 --2- 172.21.15.106:0/1891660781 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7c000c9130 0x7f7c000b2a00 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7bf000e870 tx=0x7f7bf000ed30 comp rx=0 tx=0).stop 2024-04-25T10:35:21.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:21.789+0000 7f7c0818f640 1 -- 172.21.15.106:0/1891660781 shutdown_connections 2024-04-25T10:35:21.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:21.789+0000 7f7c0818f640 1 --2- 172.21.15.106:0/1891660781 >> [v2:172.21.15.106:6832/1744210167,v1:172.21.15.106:6833/1744210167] conn(0x7f7be40744b0 0x7f7be40769c0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:21.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:21.789+0000 7f7c0818f640 1 --2- 172.21.15.106:0/1891660781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c0015cb50 0x7f7c001588e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:21.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:21.789+0000 7f7c0818f640 1 --2- 172.21.15.106:0/1891660781 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7c000c9130 0x7f7c000b2a00 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:21.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:21.789+0000 7f7c0818f640 1 --2- 172.21.15.106:0/1891660781 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7c0006fe80 0x7f7c001f66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:35:21.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:21.789+0000 7f7c0818f640 1 -- 172.21.15.106:0/1891660781 >> 172.21.15.106:0/1891660781 conn(0x7f7c0006c9b0 msgr2=0x7f7c00156260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:35:21.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:21.792+0000 7f7c0818f640 1 -- 172.21.15.106:0/1891660781 shutdown_connections 2024-04-25T10:35:21.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T10:35:21.793+0000 7f7c0818f640 1 -- 172.21.15.106:0/1891660781 wait complete. 2024-04-25T10:35:21.812 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-25T10:35:21.813 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:35:21.813 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:35:21.823 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:35:21.824 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:35:21.824 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:35:21.879 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:35:21.879 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2024-04-25T10:35:21.879 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:35:21.933 INFO:tasks.ceph.mds.e:Stopped 2024-04-25T10:35:21.933 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:35:21.933 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:35:21.948 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:35:21.948 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:35:21.948 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:35:22.002 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:35:22.002 DEBUG:tasks.ceph.mds.f:waiting for process to exit 2024-04-25T10:35:22.002 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:35:22.057 INFO:tasks.ceph.mds.f:Stopped 2024-04-25T10:35:22.057 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-25T10:35:22.057 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-25T10:35:22.057 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:35:22.086 INFO:tasks.ceph.osd.0:Stopped 2024-04-25T10:35:22.087 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-25T10:35:22.087 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:35:22.158 INFO:tasks.ceph.osd.1:Stopped 2024-04-25T10:35:22.159 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-25T10:35:22.159 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:35:22.225 INFO:tasks.ceph.osd.2:Stopped 2024-04-25T10:35:22.225 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-25T10:35:22.225 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:35:22.296 INFO:tasks.ceph.osd.3:Stopped 2024-04-25T10:35:22.297 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-25T10:35:22.297 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:35:22.336 INFO:tasks.ceph.osd.4:Stopped 2024-04-25T10:35:22.336 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-25T10:35:22.336 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:35:22.406 INFO:tasks.ceph.osd.5:Stopped 2024-04-25T10:35:22.406 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-25T10:35:22.407 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:35:22.484 INFO:tasks.ceph.osd.6:Stopped 2024-04-25T10:35:22.485 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-25T10:35:22.485 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:35:22.554 INFO:tasks.ceph.osd.7:Stopped 2024-04-25T10:35:22.554 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-25T10:35:22.554 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-25T10:35:22.554 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:35:22.581 INFO:tasks.ceph.mgr.y:Stopped 2024-04-25T10:35:22.581 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-25T10:35:22.582 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:35:22.622 INFO:tasks.ceph.mgr.x:Stopped 2024-04-25T10:35:22.623 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-25T10:35:22.623 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-25T10:35:22.623 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:35:22.635 INFO:tasks.ceph.mon.a:Stopped 2024-04-25T10:35:22.635 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-25T10:35:22.635 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:35:22.675 INFO:tasks.ceph.mon.b:Stopped 2024-04-25T10:35:22.675 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-25T10:35:22.675 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:35:22.728 INFO:tasks.ceph.mon.c:Stopped 2024-04-25T10:35:22.728 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-25T10:35:22.729 DEBUG:teuthology.orchestra.run.smithi106:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | egrep -v 'overall HEALTH_' | egrep -v '\(MON_DOWN\)' | head -n 1 2024-04-25T10:35:22.779 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi106.front.sepia.ceph.com 2024-04-25T10:35:22.780 DEBUG:teuthology.orchestra.run.smithi106:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-25T10:35:22.981 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi106.front.sepia.ceph.com 2024-04-25T10:35:22.981 DEBUG:teuthology.orchestra.run.smithi106:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-25T10:35:23.069 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi106.front.sepia.ceph.com 2024-04-25T10:35:23.130 DEBUG:teuthology.orchestra.run.smithi106:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-25T10:35:23.183 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi106.front.sepia.ceph.com 2024-04-25T10:35:23.184 DEBUG:teuthology.orchestra.run.smithi106:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-25T10:35:23.243 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi134.front.sepia.ceph.com 2024-04-25T10:35:23.243 DEBUG:teuthology.orchestra.run.smithi134:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-25T10:35:23.519 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi134.front.sepia.ceph.com 2024-04-25T10:35:23.520 DEBUG:teuthology.orchestra.run.smithi134:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-25T10:35:23.583 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi134.front.sepia.ceph.com 2024-04-25T10:35:23.584 DEBUG:teuthology.orchestra.run.smithi134:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-25T10:35:23.642 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi134.front.sepia.ceph.com 2024-04-25T10:35:23.642 DEBUG:teuthology.orchestra.run.smithi134:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-25T10:35:23.702 INFO:tasks.ceph:Archiving mon data... 2024-04-25T10:35:23.703 DEBUG:teuthology.misc:Transferring archived files from smithi106:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673079/data/mon.a.tgz 2024-04-25T10:35:23.703 DEBUG:teuthology.orchestra.run.smithi106:> mktemp 2024-04-25T10:35:23.719 INFO:teuthology.orchestra.run.smithi106.stdout:/tmp/tmp.qKxJ7FbUGm 2024-04-25T10:35:23.719 DEBUG:teuthology.orchestra.run.smithi106:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.qKxJ7FbUGm 2024-04-25T10:35:23.875 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 0666 /tmp/tmp.qKxJ7FbUGm 2024-04-25T10:35:23.956 DEBUG:teuthology.orchestra.remote:smithi106:/tmp/tmp.qKxJ7FbUGm is 358KB 2024-04-25T10:35:24.058 DEBUG:teuthology.orchestra.run.smithi106:> rm -fr /tmp/tmp.qKxJ7FbUGm 2024-04-25T10:35:24.072 DEBUG:teuthology.misc:Transferring archived files from smithi134:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673079/data/mon.b.tgz 2024-04-25T10:35:24.072 DEBUG:teuthology.orchestra.run.smithi134:> mktemp 2024-04-25T10:35:24.088 INFO:teuthology.orchestra.run.smithi134.stdout:/tmp/tmp.eotKsvSRjZ 2024-04-25T10:35:24.089 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.eotKsvSRjZ 2024-04-25T10:35:24.235 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 0666 /tmp/tmp.eotKsvSRjZ 2024-04-25T10:35:24.316 DEBUG:teuthology.orchestra.remote:smithi134:/tmp/tmp.eotKsvSRjZ is 363KB 2024-04-25T10:35:24.411 DEBUG:teuthology.orchestra.run.smithi134:> rm -fr /tmp/tmp.eotKsvSRjZ 2024-04-25T10:35:24.425 DEBUG:teuthology.misc:Transferring archived files from smithi134:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673079/data/mon.c.tgz 2024-04-25T10:35:24.426 DEBUG:teuthology.orchestra.run.smithi134:> mktemp 2024-04-25T10:35:24.480 INFO:teuthology.orchestra.run.smithi134.stdout:/tmp/tmp.yI73Q6gujo 2024-04-25T10:35:24.481 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.yI73Q6gujo 2024-04-25T10:35:24.607 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 0666 /tmp/tmp.yI73Q6gujo 2024-04-25T10:35:24.689 DEBUG:teuthology.orchestra.remote:smithi134:/tmp/tmp.yI73Q6gujo is 363KB 2024-04-25T10:35:24.785 DEBUG:teuthology.orchestra.run.smithi134:> rm -fr /tmp/tmp.yI73Q6gujo 2024-04-25T10:35:24.799 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-25T10:35:24.800 DEBUG:teuthology.orchestra.run.smithi106:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T10:35:24.802 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T10:35:24.886 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T10:35:24.917 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T10:35:24.917 INFO:tasks.ceph:Archiving crash dumps... 2024-04-25T10:35:24.919 DEBUG:teuthology.misc:Transferring archived files from smithi106:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673079/remote/smithi106/crash 2024-04-25T10:35:24.919 DEBUG:teuthology.orchestra.run.smithi106:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T10:35:24.953 DEBUG:teuthology.misc:Transferring archived files from smithi134:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673079/remote/smithi134/crash 2024-04-25T10:35:24.954 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T10:35:24.979 INFO:tasks.ceph:Shutting down logrotate 2024-04-25T10:35:24.980 DEBUG:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:35:24.997 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:35:25.073 DEBUG:teuthology.orchestra.run.smithi106:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T10:35:25.099 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T10:35:25.122 INFO:tasks.ceph:Compressing logs... 2024-04-25T10:35:25.122 DEBUG:teuthology.orchestra.run.smithi106:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T10:35:25.141 DEBUG:teuthology.orchestra.run.smithi134:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T10:35:25.189 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T10:35:25.189 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34655.log 2024-04-25T10:35:25.190 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.f.log 2024-04-25T10:35:25.190 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.34655.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T10:35:25.190 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-mds.f.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34655.log.gz 2024-04-25T10:35:25.190 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-25T10:35:25.190 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-25T10:35:25.190 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-25T10:35:25.190 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-25T10:35:25.190 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34524.log 2024-04-25T10:35:25.190 INFO:teuthology.orchestra.run.smithi134.stderr: 91.9% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T10:35:25.190 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-25T10:35:25.190 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.34524.log: 93.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T10:35:25.190 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-25T10:35:25.190 INFO:teuthology.orchestra.run.smithi134.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34524.log.gz 2024-04-25T10:35:25.191 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-25T10:35:25.191 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-mgr.x.log: /var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34695.log 2024-04-25T10:35:25.207 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-osd.6.log: 93.2% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-25T10:35:25.207 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34775.log 2024-04-25T10:35:25.207 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.34695.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.34695.log.gz 2024-04-25T10:35:25.207 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34735.log 2024-04-25T10:35:25.208 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.34775.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34775.log.gz 2024-04-25T10:35:25.208 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-25T10:35:25.208 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.34735.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34735.log.gz 2024-04-25T10:35:25.208 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-25T10:35:25.213 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34615.log 2024-04-25T10:35:25.224 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-mon.b.log: 92.2% -- replaced with /var/log/ceph/ceph-mds.f.log.gz 2024-04-25T10:35:25.224 INFO:teuthology.orchestra.run.smithi134.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-25T10:35:25.224 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34564.log 2024-04-25T10:35:25.224 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.34615.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34615.log.gz 2024-04-25T10:35:25.224 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34484.log 2024-04-25T10:35:25.224 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.34564.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34564.log.gz 2024-04-25T10:35:25.224 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.34484.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34484.log.gz 2024-04-25T10:35:25.231 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45544.log 2024-04-25T10:35:25.231 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44190.log 2024-04-25T10:35:25.231 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.45544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43001.log 2024-04-25T10:35:25.231 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.44190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45045.log 2024-04-25T10:35:25.231 INFO:teuthology.orchestra.run.smithi106.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45544.log.gz 2024-04-25T10:35:25.231 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.43001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48905.log 2024-04-25T10:35:25.231 INFO:teuthology.orchestra.run.smithi106.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44190.log.gz 2024-04-25T10:35:25.231 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.45045.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43001.log.gz 2024-04-25T10:35:25.231 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49259.log 2024-04-25T10:35:25.232 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.48905.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.45045.log.gz 2024-04-25T10:35:25.232 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47553.log 2024-04-25T10:35:25.232 INFO:teuthology.orchestra.run.smithi106.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48905.log.gz 2024-04-25T10:35:25.232 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.49259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49199.log 2024-04-25T10:35:25.232 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.47553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49589.log 2024-04-25T10:35:25.232 INFO:teuthology.orchestra.run.smithi106.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49259.log.gz 2024-04-25T10:35:25.232 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.49199.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.47553.log.gz 2024-04-25T10:35:25.232 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43047.log 2024-04-25T10:35:25.232 INFO:teuthology.orchestra.run.smithi106.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49199.log.gz 2024-04-25T10:35:25.232 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.49589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42488.log 2024-04-25T10:35:25.232 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.43047.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49589.log.gz 2024-04-25T10:35:25.232 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48039.log 2024-04-25T10:35:25.232 INFO:teuthology.orchestra.run.smithi106.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43047.log.gz 2024-04-25T10:35:25.233 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.42488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46659.log 2024-04-25T10:35:25.233 INFO:teuthology.orchestra.run.smithi106.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.42488.log.gz 2024-04-25T10:35:25.233 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.48039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44708.log 2024-04-25T10:35:25.233 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.46659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46024.log 2024-04-25T10:35:25.233 INFO:teuthology.orchestra.run.smithi106.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48039.log.gz 2024-04-25T10:35:25.233 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.44708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.43955.log 2024-04-25T10:35:25.233 INFO:teuthology.orchestra.run.smithi106.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46659.log.gz 2024-04-25T10:35:25.233 INFO:teuthology.orchestra.run.smithi106.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44708.log.gz 2024-04-25T10:35:25.233 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.46024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48989.log 2024-04-25T10:35:25.233 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.0.43955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41732.log 2024-04-25T10:35:25.233 INFO:teuthology.orchestra.run.smithi106.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.46024.log.gz 2024-04-25T10:35:25.233 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.48989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45376.log 2024-04-25T10:35:25.233 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.41732.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48989.log.gz 2024-04-25T10:35:25.233 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34652.log 2024-04-25T10:35:25.234 INFO:teuthology.orchestra.run.smithi106.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41732.log.gz 2024-04-25T10:35:25.234 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.45376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44919.log 2024-04-25T10:35:25.234 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.34652.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.45376.log.gz 2024-04-25T10:35:25.234 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40568.log 2024-04-25T10:35:25.234 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.44919.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34652.log.gz 2024-04-25T10:35:25.234 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41393.log 2024-04-25T10:35:25.234 INFO:teuthology.orchestra.run.smithi106.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44919.log.gz 2024-04-25T10:35:25.234 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.40568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44274.log 2024-04-25T10:35:25.234 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.41393.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.40568.log.gz 2024-04-25T10:35:25.234 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42270.log 2024-04-25T10:35:25.234 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.44274.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41393.log.gz 2024-04-25T10:35:25.234 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42791.log 2024-04-25T10:35:25.234 INFO:teuthology.orchestra.run.smithi106.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44274.log.gz 2024-04-25T10:35:25.234 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.42270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48947.log 2024-04-25T10:35:25.234 INFO:teuthology.orchestra.run.smithi106.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42270.log.gz 2024-04-25T10:35:25.235 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.42791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47210.log 2024-04-25T10:35:25.235 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.48947.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42791.log.gz 2024-04-25T10:35:25.235 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42312.log 2024-04-25T10:35:25.235 INFO:teuthology.orchestra.run.smithi106.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48947.log.gz 2024-04-25T10:35:25.235 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.47210.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T10:35:25.235 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.42312.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.47210.log.gz 2024-04-25T10:35:25.235 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45674.log 2024-04-25T10:35:25.235 INFO:teuthology.orchestra.run.smithi106.stderr: 88.5%/var/log/ceph/ceph.log: -- replaced with /var/log/ceph/ceph-client.admin.42312.log.gz 2024-04-25T10:35:25.235 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45273.log 2024-04-25T10:35:25.235 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.45674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49425.log 2024-04-25T10:35:25.235 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.45273.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45674.log.gz 2024-04-25T10:35:25.235 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46234.log 2024-04-25T10:35:25.235 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45590.log 2024-04-25T10:35:25.235 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.49425.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45273.log.gz 2024-04-25T10:35:25.235 INFO:teuthology.orchestra.run.smithi106.stderr: 86.0% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T10:35:25.236 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.46234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40442.log 2024-04-25T10:35:25.236 INFO:teuthology.orchestra.run.smithi106.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49425.log.gz 2024-04-25T10:35:25.236 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.45590.log: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.46234.log.gz 2024-04-25T10:35:25.236 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48123.log 2024-04-25T10:35:25.236 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.40442.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45590.log.gz 2024-04-25T10:35:25.236 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46276.log 2024-04-25T10:35:25.236 INFO:teuthology.orchestra.run.smithi106.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40442.log.gz 2024-04-25T10:35:25.236 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.48123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48558.log 2024-04-25T10:35:25.236 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.46276.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48123.log.gz 2024-04-25T10:35:25.236 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-25T10:35:25.236 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.48558.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46276.log.gz 2024-04-25T10:35:25.236 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47637.log 2024-04-25T10:35:25.236 INFO:teuthology.orchestra.run.smithi106.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48558.log.gz 2024-04-25T10:35:25.236 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49509.log 2024-04-25T10:35:25.237 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.47637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34388.log 2024-04-25T10:35:25.237 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.49509.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47637.log.gz 2024-04-25T10:35:25.237 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41397.log 2024-04-25T10:35:25.237 INFO:teuthology.orchestra.run.smithi106.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49509.log.gz 2024-04-25T10:35:25.237 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47469.log 2024-04-25T10:35:25.237 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.34388.log: /var/log/ceph/ceph-client.admin.41397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48650.log 2024-04-25T10:35:25.237 INFO:teuthology.orchestra.run.smithi106.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34388.log.gz 2024-04-25T10:35:25.237 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45502.log 2024-04-25T10:35:25.237 INFO:teuthology.orchestra.run.smithi106.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41397.log.gz/var/log/ceph/ceph-client.admin.47469.log: 2024-04-25T10:35:25.237 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.48650.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47469.log.gz 2024-04-25T10:35:25.237 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45189.log 2024-04-25T10:35:25.237 INFO:teuthology.orchestra.run.smithi106.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48650.log.gz 2024-04-25T10:35:25.237 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42074.log 2024-04-25T10:35:25.237 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.45502.log: /var/log/ceph/ceph-client.admin.45189.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T10:35:25.238 INFO:teuthology.orchestra.run.smithi106.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45502.log.gz 2024-04-25T10:35:25.238 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.42074.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.45189.log.gz 2024-04-25T10:35:25.238 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48473.log 2024-04-25T10:35:25.238 INFO:teuthology.orchestra.run.smithi106.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42074.log.gz 2024-04-25T10:35:25.238 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49341.log 2024-04-25T10:35:25.238 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-client.admin.48473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47939.log 2024-04-25T10:35:25.238 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.49341.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48473.log.gz 2024-04-25T10:35:25.238 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40314.log 2024-04-25T10:35:25.238 INFO:teuthology.orchestra.run.smithi106.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49341.log.gz 2024-04-25T10:35:25.238 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.47939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44750.log 2024-04-25T10:35:25.238 INFO:teuthology.orchestra.run.smithi106.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47939.log.gz 2024-04-25T10:35:25.238 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46506.log 2024-04-25T10:35:25.238 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.40314.log: 89.4% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T10:35:25.238 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.44750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44526.log 2024-04-25T10:35:25.239 INFO:teuthology.orchestra.run.smithi106.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40314.log.gz 2024-04-25T10:35:25.239 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.46506.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44750.log.gz 2024-04-25T10:35:25.239 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46192.log 2024-04-25T10:35:25.239 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47083.log 2024-04-25T10:35:25.239 INFO:teuthology.orchestra.run.smithi106.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.46506.log.gz 2024-04-25T10:35:25.239 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.44526.log: /var/log/ceph/ceph-client.admin.46192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46785.log 2024-04-25T10:35:25.239 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44526.log.gz 2024-04-25T10:35:25.239 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.47083.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.46192.log.gz 2024-04-25T10:35:25.239 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42539.log 2024-04-25T10:35:25.239 INFO:teuthology.orchestra.run.smithi106.stderr: 87.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.47083.log.gz -5 2024-04-25T10:35:25.239 INFO:teuthology.orchestra.run.smithi106.stderr: --verbose -- /var/log/ceph/ceph-client.admin.44792.log 2024-04-25T10:35:25.239 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.46785.log: /var/log/ceph/ceph-client.admin.42539.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46785.log.gz 2024-04-25T10:35:25.239 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44442.log 2024-04-25T10:35:25.239 INFO:teuthology.orchestra.run.smithi106.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.42539.log.gz 2024-04-25T10:35:25.240 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log/var/log/ceph/ceph-client.admin.44792.log: 2024-04-25T10:35:25.240 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.44442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44400.log 2024-04-25T10:35:25.240 INFO:teuthology.orchestra.run.smithi106.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.44792.log.gz 2024-04-25T10:35:25.240 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-osd.3.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44442.log.gz 2024-04-25T10:35:25.240 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46872.log 2024-04-25T10:35:25.240 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.44400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46463.log 2024-04-25T10:35:25.240 INFO:teuthology.orchestra.run.smithi106.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44400.log.gz 2024-04-25T10:35:25.240 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.46872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41398.log 2024-04-25T10:35:25.240 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49073.log 2024-04-25T10:35:25.240 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.46463.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46872.log.gz 2024-04-25T10:35:25.240 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.41398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41220.log 2024-04-25T10:35:25.240 INFO:teuthology.orchestra.run.smithi106.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.46463.log.gz 2024-04-25T10:35:25.240 INFO:teuthology.orchestra.run.smithi106.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41398.log.gz 2024-04-25T10:35:25.240 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.49073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48736.log 2024-04-25T10:35:25.241 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.41220.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49073.log.gz 2024-04-25T10:35:25.241 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40484.log 2024-04-25T10:35:25.241 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.48736.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41220.log.gz 2024-04-25T10:35:25.241 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47981.log 2024-04-25T10:35:25.241 INFO:teuthology.orchestra.run.smithi106.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48736.log.gz 2024-04-25T10:35:25.241 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.40484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48347.log 2024-04-25T10:35:25.241 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.47981.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40484.log.gz 2024-04-25T10:35:25.241 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48165.log 2024-04-25T10:35:25.241 INFO:teuthology.orchestra.run.smithi106.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47981.log.gz 2024-04-25T10:35:25.241 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.48347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45632.log 2024-04-25T10:35:25.241 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.48165.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48347.log.gz 2024-04-25T10:35:25.241 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40716.log 2024-04-25T10:35:25.241 INFO:teuthology.orchestra.run.smithi106.stderr: 88.7%/var/log/ceph/ceph-client.admin.45632.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-25T10:35:25.242 INFO:teuthology.orchestra.run.smithi106.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48165.log.gz 2024-04-25T10:35:25.242 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.40716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46617.log 2024-04-25T10:35:25.242 INFO:teuthology.orchestra.run.smithi106.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.45632.log.gz 2024-04-25T10:35:25.242 INFO:teuthology.orchestra.run.smithi106.stderr:gzip/var/log/ceph/ceph-mds.c.log: 88.9% -5 --verbose -- /var/log/ceph/ceph-client.admin.41737.log 2024-04-25T10:35:25.242 INFO:teuthology.orchestra.run.smithi106.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40716.log.gz 2024-04-25T10:35:25.242 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.46617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46743.log 2024-04-25T10:35:25.242 INFO:teuthology.orchestra.run.smithi106.stderr: 88.7%/var/log/ceph/ceph-client.admin.41737.log: gzip -- replaced with /var/log/ceph/ceph-client.admin.46617.log.gz 2024-04-25T10:35:25.242 INFO:teuthology.orchestra.run.smithi106.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.43284.log 2024-04-25T10:35:25.242 INFO:teuthology.orchestra.run.smithi106.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41737.log.gz 2024-04-25T10:35:25.242 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.46743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48863.log 2024-04-25T10:35:25.242 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.43284.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.46743.log.gz 2024-04-25T10:35:25.242 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-25T10:35:25.242 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.48863.log: gzip -5 --verbose 91.2% -- replaced with /var/log/ceph/ceph-client.admin.43284.log.gz 2024-04-25T10:35:25.243 INFO:teuthology.orchestra.run.smithi106.stderr: -- /var/log/ceph/ceph-mgr.y.log 2024-04-25T10:35:25.243 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-osd.1.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48863.log.gz 2024-04-25T10:35:25.243 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40968.log 2024-04-25T10:35:25.243 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45982.log 2024-04-25T10:35:25.243 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.40968.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-25T10:35:25.243 INFO:teuthology.orchestra.run.smithi106.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.40968.log.gz 2024-04-25T10:35:25.243 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.45982.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45982.log.gz 2024-04-25T10:35:25.243 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45460.log 2024-04-25T10:35:25.243 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45418.log 2024-04-25T10:35:25.243 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.45460.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45460.log.gz 2024-04-25T10:35:25.243 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44665.log 2024-04-25T10:35:25.243 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.45418.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.45418.log.gz 2024-04-25T10:35:25.243 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46914.log 2024-04-25T10:35:25.243 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.44665.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44665.log.gz 2024-04-25T10:35:25.243 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41731.log 2024-04-25T10:35:25.244 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.46914.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.46914.log.gz 2024-04-25T10:35:25.244 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48081.log 2024-04-25T10:35:25.244 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.41731.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41731.log.gz 2024-04-25T10:35:25.244 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45231.log 2024-04-25T10:35:25.244 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.48081.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.48081.log.gz 2024-04-25T10:35:25.244 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40884.log 2024-04-25T10:35:25.244 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.45231.log: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.45231.log.gz 2024-04-25T10:35:25.244 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44484.log 2024-04-25T10:35:25.244 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.40884.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40884.log.gz 2024-04-25T10:35:25.244 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46998.log 2024-04-25T10:35:25.244 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.44484.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44484.log.gz 2024-04-25T10:35:25.244 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45105.log 2024-04-25T10:35:25.244 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.46998.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46998.log.gz 2024-04-25T10:35:25.244 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41052.log 2024-04-25T10:35:25.245 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.45105.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45105.log.gz 2024-04-25T10:35:25.245 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44316.log 2024-04-25T10:35:25.245 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.41052.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41052.log.gz 2024-04-25T10:35:25.245 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46150.log 2024-04-25T10:35:25.245 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.44316.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44316.log.gz 2024-04-25T10:35:25.245 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49115.log 2024-04-25T10:35:25.245 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.46150.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46150.log.gz 2024-04-25T10:35:25.245 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40758.log 2024-04-25T10:35:25.245 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.49115.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49115.log.gz 2024-04-25T10:35:25.245 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47126.log 2024-04-25T10:35:25.245 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.40758.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40758.log.gz 2024-04-25T10:35:25.245 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41094.log 2024-04-25T10:35:25.245 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.47126.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.47126.log.gz 2024-04-25T10:35:25.245 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40400.log 2024-04-25T10:35:25.246 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.41094.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41094.log.gz 2024-04-25T10:35:25.246 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41178.log 2024-04-25T10:35:25.246 INFO:teuthology.orchestra.run.smithi106.stderr: 92.4% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-25T10:35:25.246 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.40400.log: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46701.log 2024-04-25T10:35:25.246 INFO:teuthology.orchestra.run.smithi106.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40400.log.gz/var/log/ceph/ceph-client.admin.41178.log: 2024-04-25T10:35:25.246 INFO:teuthology.orchestra.run.smithi106.stderr: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40842.log 2024-04-25T10:35:25.246 INFO:teuthology.orchestra.run.smithi106.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41178.log.gz 2024-04-25T10:35:25.246 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.46701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45940.log 2024-04-25T10:35:25.246 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.40842.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40842.log.gz 2024-04-25T10:35:25.246 INFO:teuthology.orchestra.run.smithi106.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.46701.log.gz 2024-04-25T10:35:25.246 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40656.log 2024-04-25T10:35:25.246 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48389.log 2024-04-25T10:35:25.246 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.45940.log: /var/log/ceph/ceph-client.admin.40656.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45940.log.gz 2024-04-25T10:35:25.246 INFO:teuthology.orchestra.run.smithi106.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40656.log.gz 2024-04-25T10:35:25.246 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41401.log 2024-04-25T10:35:25.247 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.48389.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48389.log.gz 2024-04-25T10:35:25.247 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-25T10:35:25.247 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47427.log 2024-04-25T10:35:25.247 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.41401.log: /var/log/ceph/ceph-mon.a.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41401.log.gz 2024-04-25T10:35:25.247 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44961.log 2024-04-25T10:35:25.247 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.47427.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47427.log.gz 2024-04-25T10:35:25.247 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49383.log 2024-04-25T10:35:25.247 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.44961.log: 91.9% -- replaced with /var/log/ceph/ceph-client.admin.44961.log.gz 2024-04-25T10:35:25.247 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41136.log 2024-04-25T10:35:25.247 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.49383.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49383.log.gz 2024-04-25T10:35:25.247 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43149.log 2024-04-25T10:35:25.247 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.41136.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.41136.log.gz 2024-04-25T10:35:25.247 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46108.log 2024-04-25T10:35:25.247 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.43149.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.43149.log.gz 2024-04-25T10:35:25.248 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40612.log 2024-04-25T10:35:25.248 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.46108.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46108.log.gz 2024-04-25T10:35:25.248 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41738.log 2024-04-25T10:35:25.248 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.40612.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40612.log.gz 2024-04-25T10:35:25.248 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44834.log 2024-04-25T10:35:25.255 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.41738.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.41738.log.gz 2024-04-25T10:35:25.255 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46548.log 2024-04-25T10:35:25.255 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.44834.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44834.log.gz 2024-04-25T10:35:25.255 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40526.log 2024-04-25T10:35:25.255 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.46548.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.46548.log.gz 2024-04-25T10:35:25.255 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42073.log 2024-04-25T10:35:25.255 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.40526.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40526.log.gz 2024-04-25T10:35:25.255 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48694.log 2024-04-25T10:35:25.255 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.42073.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42073.log.gz 2024-04-25T10:35:25.255 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49031.log 2024-04-25T10:35:25.255 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.48694.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48694.log.gz 2024-04-25T10:35:25.255 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42071.log 2024-04-25T10:35:25.255 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.49031.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49031.log.gz 2024-04-25T10:35:25.256 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43089.log 2024-04-25T10:35:25.256 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.42071.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42071.log.gz 2024-04-25T10:35:25.256 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43416.log 2024-04-25T10:35:25.256 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.43089.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43089.log.gz 2024-04-25T10:35:25.256 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41394.log 2024-04-25T10:35:25.256 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.43416.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.43416.log.gz 2024-04-25T10:35:25.256 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42354.log 2024-04-25T10:35:25.256 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.41394.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41394.log.gz 2024-04-25T10:35:25.256 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49631.log 2024-04-25T10:35:25.256 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.42354.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42354.log.gz 2024-04-25T10:35:25.256 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47292.log 2024-04-25T10:35:25.256 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.49631.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49631.log.gz 2024-04-25T10:35:25.256 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43350.log 2024-04-25T10:35:25.256 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.47292.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47292.log.gz 2024-04-25T10:35:25.257 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46830.log 2024-04-25T10:35:25.257 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.43350.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.43350.log.gz 2024-04-25T10:35:25.257 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44877.log 2024-04-25T10:35:25.257 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.46830.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46830.log.gz 2024-04-25T10:35:25.257 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42833.log 2024-04-25T10:35:25.257 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.44877.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44877.log.gz 2024-04-25T10:35:25.257 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.e.log 2024-04-25T10:35:25.257 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.42833.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42833.log.gz 2024-04-25T10:35:25.257 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47697.log 2024-04-25T10:35:25.260 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-mds.e.log: 91.8% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-25T10:35:25.261 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48516.log 2024-04-25T10:35:25.261 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.47697.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47697.log.gz 2024-04-25T10:35:25.261 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42070.log 2024-04-25T10:35:25.262 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.48516.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48516.log.gz 2024-04-25T10:35:25.264 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41934.log 2024-04-25T10:35:25.265 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.42070.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42070.log.gz 2024-04-25T10:35:25.265 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47040.log 2024-04-25T10:35:25.265 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.41934.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.41934.log.gz 2024-04-25T10:35:25.269 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46361.log 2024-04-25T10:35:25.269 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.47040.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47040.log.gz 2024-04-25T10:35:25.269 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42707.log 2024-04-25T10:35:25.269 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.46361.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46361.log.gz 2024-04-25T10:35:25.269 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45334.log 2024-04-25T10:35:25.269 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.42707.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42707.log.gz 2024-04-25T10:35:25.269 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47595.log 2024-04-25T10:35:25.269 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.45334.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45334.log.gz 2024-04-25T10:35:25.273 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42875.log 2024-04-25T10:35:25.273 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.47595.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47595.log.gz 2024-04-25T10:35:25.273 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48431.log 2024-04-25T10:35:25.273 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.42875.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42875.log.gz 2024-04-25T10:35:25.273 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44358.log 2024-04-25T10:35:25.273 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.48431.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48431.log.gz 2024-04-25T10:35:25.273 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41734.log 2024-04-25T10:35:25.273 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.44358.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44358.log.gz 2024-04-25T10:35:25.278 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45003.log 2024-04-25T10:35:25.278 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.41734.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.41734.log.gz 2024-04-25T10:35:25.278 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48779.log 2024-04-25T10:35:25.278 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.45003.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.45003.log.gz 2024-04-25T10:35:25.278 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45776.log 2024-04-25T10:35:25.278 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.48779.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48779.log.gz 2024-04-25T10:35:25.278 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42446.log 2024-04-25T10:35:25.278 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.45776.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45776.log.gz 2024-04-25T10:35:25.278 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40356.log 2024-04-25T10:35:25.278 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.42446.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42446.log.gz 2024-04-25T10:35:25.282 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41735.log 2024-04-25T10:35:25.282 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.40356.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40356.log.gz 2024-04-25T10:35:25.282 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42623.log 2024-04-25T10:35:25.282 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.41735.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41735.log.gz 2024-04-25T10:35:25.282 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40214.log 2024-04-25T10:35:25.282 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.42623.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42623.log.gz 2024-04-25T10:35:25.282 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41399.log 2024-04-25T10:35:25.282 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.40214.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40214.log.gz 2024-04-25T10:35:25.286 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47739.log 2024-04-25T10:35:25.286 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.41399.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41399.log.gz 2024-04-25T10:35:25.286 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47168.log 2024-04-25T10:35:25.286 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.47739.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47739.log.gz 2024-04-25T10:35:25.286 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41402.log 2024-04-25T10:35:25.286 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.47168.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47168.log.gz 2024-04-25T10:35:25.286 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46318.log 2024-04-25T10:35:25.286 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.41402.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41402.log.gz 2024-04-25T10:35:25.288 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49467.log 2024-04-25T10:35:25.288 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.46318.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46318.log.gz 2024-04-25T10:35:25.288 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44148.log 2024-04-25T10:35:25.288 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.49467.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49467.log.gz 2024-04-25T10:35:25.293 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40926.log 2024-04-25T10:35:25.294 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.44148.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44148.log.gz 2024-04-25T10:35:25.294 INFO:teuthology.orchestra.run.smithi106.stderr: 92.1% -- replaced with /var/log/ceph/ceph-mds.e.log.gz 2024-04-25T10:35:25.294 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42072.log 2024-04-25T10:35:25.294 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.40926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47511.log 2024-04-25T10:35:25.294 INFO:teuthology.orchestra.run.smithi106.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40926.log.gz 2024-04-25T10:35:25.294 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.42072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43218.log 2024-04-25T10:35:25.294 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.47511.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42072.log.gz 2024-04-25T10:35:25.294 INFO:teuthology.orchestra.run.smithi106.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.47511.log.gz 2024-04-25T10:35:25.294 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42581.log 2024-04-25T10:35:25.294 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40800.log 2024-04-25T10:35:25.294 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.42581.log: /var/log/ceph/ceph-client.admin.43218.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42581.log.gz 2024-04-25T10:35:25.294 INFO:teuthology.orchestra.run.smithi106.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.43218.log.gz 2024-04-25T10:35:25.294 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43518.log 2024-04-25T10:35:25.294 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47813.log 2024-04-25T10:35:25.315 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.40800.log: /var/log/ceph/ceph-client.admin.43518.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40800.log.gz 2024-04-25T10:35:25.315 INFO:teuthology.orchestra.run.smithi106.stderr:gzip 88.5% -5 --verbose -- /var/log/ceph/ceph-client.admin.46956.log 2024-04-25T10:35:25.315 INFO:teuthology.orchestra.run.smithi106.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43518.log.gz 2024-04-25T10:35:25.316 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.47813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47855.log 2024-04-25T10:35:25.316 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.46956.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47813.log.gz 2024-04-25T10:35:25.316 INFO:teuthology.orchestra.run.smithi106.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.46956.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42396.log 2024-04-25T10:35:25.316 INFO:teuthology.orchestra.run.smithi106.stderr: 2024-04-25T10:35:25.316 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45716.log 2024-04-25T10:35:25.316 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.47855.log: /var/log/ceph/ceph-client.admin.42396.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.42396.log.gz 2024-04-25T10:35:25.316 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47855.log.gz 2024-04-25T10:35:25.316 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44232.log 2024-04-25T10:35:25.316 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.45716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47897.log 2024-04-25T10:35:25.316 INFO:teuthology.orchestra.run.smithi106.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45716.log.gz/var/log/ceph/ceph-client.admin.44232.log: 2024-04-25T10:35:25.316 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42068.log 2024-04-25T10:35:25.316 INFO:teuthology.orchestra.run.smithi106.stderr: 87.8%/var/log/ceph/ceph-client.admin.47897.log: -- replaced with /var/log/ceph/ceph-client.admin.44232.log.gz 2024-04-25T10:35:25.316 INFO:teuthology.orchestra.run.smithi106.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47897.log.gzgzip 2024-04-25T10:35:25.316 INFO:teuthology.orchestra.run.smithi106.stderr: -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.28332.log 2024-04-25T10:35:25.317 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49299.log 2024-04-25T10:35:25.317 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph.tmp-client.admin.28332.log: /var/log/ceph/ceph-client.admin.42068.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.28332.log.gz 2024-04-25T10:35:25.317 INFO:teuthology.orchestra.run.smithi106.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42068.log.gz 2024-04-25T10:35:25.317 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42959.log 2024-04-25T10:35:25.317 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.49299.log: gzip -5 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49299.log.gz 2024-04-25T10:35:25.317 INFO:teuthology.orchestra.run.smithi106.stderr: --verbose -- /var/log/ceph/ceph-client.admin.49157.log 2024-04-25T10:35:25.317 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-25T10:35:25.317 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.42959.log: /var/log/ceph/ceph-client.admin.49157.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.42959.log.gz 2024-04-25T10:35:25.317 INFO:teuthology.orchestra.run.smithi106.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49157.log.gz 2024-04-25T10:35:25.317 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41400.log 2024-04-25T10:35:25.317 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48207.log 2024-04-25T10:35:25.317 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.41400.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41400.log.gz 2024-04-25T10:35:25.317 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46421.log 2024-04-25T10:35:25.317 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.48207.log: 92.5% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-25T10:35:25.318 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48207.log.gz 2024-04-25T10:35:25.318 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45147.log 2024-04-25T10:35:25.318 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.46421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48608.log 2024-04-25T10:35:25.318 INFO:teuthology.orchestra.run.smithi106.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46421.log.gz 2024-04-25T10:35:25.318 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.45147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44584.log 2024-04-25T10:35:25.318 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.48608.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45147.log.gz 2024-04-25T10:35:25.318 INFO:teuthology.orchestra.run.smithi106.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48608.log.gz 2024-04-25T10:35:25.318 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40007.log 2024-04-25T10:35:25.318 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48821.log 2024-04-25T10:35:25.318 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.44584.log: /var/log/ceph/ceph-client.admin.40007.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44584.log.gz 2024-04-25T10:35:25.318 INFO:teuthology.orchestra.run.smithi106.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40007.log.gz 2024-04-25T10:35:25.318 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41010.log 2024-04-25T10:35:25.318 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.48821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46066.log 2024-04-25T10:35:25.318 INFO:teuthology.orchestra.run.smithi106.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48821.log.gz 2024-04-25T10:35:25.319 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.41010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42917.log 2024-04-25T10:35:25.319 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.46066.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.41010.log.gz 2024-04-25T10:35:25.319 INFO:teuthology.orchestra.run.smithi106.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.46066.log.gz 2024-04-25T10:35:25.319 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42749.log 2024-04-25T10:35:25.319 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42665.log 2024-04-25T10:35:25.319 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.42749.log: /var/log/ceph/ceph-client.admin.42917.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42917.log.gz 2024-04-25T10:35:25.319 INFO:teuthology.orchestra.run.smithi106.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42749.log.gz 2024-04-25T10:35:25.319 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42069.log 2024-04-25T10:35:25.319 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.42665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40099.log 2024-04-25T10:35:25.319 INFO:teuthology.orchestra.run.smithi106.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42665.log.gz 2024-04-25T10:35:25.319 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.42069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41722.log 2024-04-25T10:35:25.319 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.40099.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42069.log.gz 2024-04-25T10:35:25.319 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40099.log.gz 2024-04-25T10:35:25.319 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34755.log 2024-04-25T10:35:25.319 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41733.log 2024-04-25T10:35:25.320 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.41722.log: /var/log/ceph/ceph-client.admin.34755.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41722.log.gz 2024-04-25T10:35:25.320 INFO:teuthology.orchestra.run.smithi106.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.34755.log.gz 2024-04-25T10:35:25.320 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.41733.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41733.log.gz 2024-04-25T10:35:25.320 INFO:teuthology.orchestra.run.smithi134.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-25T10:35:25.324 INFO:teuthology.orchestra.run.smithi134.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-25T10:35:25.374 INFO:teuthology.orchestra.run.smithi134.stderr: 92.4% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-25T10:35:25.539 INFO:teuthology.orchestra.run.smithi106.stderr: 91.0% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-25T10:35:25.582 INFO:teuthology.orchestra.run.smithi106.stderr: 93.1% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-25T10:35:25.608 INFO:teuthology.orchestra.run.smithi106.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-25T10:35:25.665 INFO:teuthology.orchestra.run.smithi134.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-25T10:35:25.697 INFO:teuthology.orchestra.run.smithi134.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-25T10:35:25.823 INFO:teuthology.orchestra.run.smithi134.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-25T10:35:25.828 INFO:teuthology.orchestra.run.smithi106.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-25T10:35:25.878 INFO:teuthology.orchestra.run.smithi134.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-25T10:35:25.880 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-04-25T10:35:25.880 INFO:teuthology.orchestra.run.smithi134.stderr:real 0m0.704s 2024-04-25T10:35:25.880 INFO:teuthology.orchestra.run.smithi134.stderr:user 0m2.698s 2024-04-25T10:35:25.880 INFO:teuthology.orchestra.run.smithi134.stderr:sys 0m0.121s 2024-04-25T10:35:25.886 INFO:teuthology.orchestra.run.smithi106.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-25T10:35:32.539 INFO:teuthology.orchestra.run.smithi106.stderr: 96.7% -- replaced with /var/log/ceph/ceph-client.0.43955.log.gz 2024-04-25T10:35:32.541 INFO:teuthology.orchestra.run.smithi106.stderr: 2024-04-25T10:35:32.541 INFO:teuthology.orchestra.run.smithi106.stderr:real 0m7.387s 2024-04-25T10:35:32.541 INFO:teuthology.orchestra.run.smithi106.stderr:user 0m9.702s 2024-04-25T10:35:32.541 INFO:teuthology.orchestra.run.smithi106.stderr:sys 0m0.497s 2024-04-25T10:35:32.541 INFO:tasks.ceph:Archiving logs... 2024-04-25T10:35:32.542 DEBUG:teuthology.misc:Transferring archived files from smithi106:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673079/remote/smithi106/log 2024-04-25T10:35:32.542 DEBUG:teuthology.orchestra.run.smithi106:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T10:35:33.843 DEBUG:teuthology.misc:Transferring archived files from smithi134:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673079/remote/smithi134/log 2024-04-25T10:35:33.844 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T10:35:34.246 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T10:35:34.255 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T10:35:34.256 DEBUG:teuthology.orchestra.run.smithi106:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T10:35:34.258 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T10:35:34.348 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T10:35:34.357 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T10:35:34.358 DEBUG:teuthology.orchestra.run.smithi106:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T10:35:34.360 DEBUG:teuthology.orchestra.run.smithi134:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T10:35:34.374 INFO:teuthology.orchestra.run.smithi106.stderr:bash: line 1: ntpq: command not found 2024-04-25T10:35:34.377 INFO:teuthology.orchestra.run.smithi134.stderr:bash: line 1: ntpq: command not found 2024-04-25T10:35:34.381 INFO:teuthology.orchestra.run.smithi106.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T10:35:34.381 INFO:teuthology.orchestra.run.smithi106.stdout:=============================================================================== 2024-04-25T10:35:34.381 INFO:teuthology.orchestra.run.smithi106.stdout:^* hv01.front.sepia.ceph.com 3 6 377 6 +1218us[+1179us] +/- 60ms 2024-04-25T10:35:34.381 INFO:teuthology.orchestra.run.smithi106.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 6 -652us[ -652us] +/- 67ms 2024-04-25T10:35:34.381 INFO:teuthology.orchestra.run.smithi106.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 6 -740us[ -740us] +/- 62ms 2024-04-25T10:35:34.381 INFO:teuthology.orchestra.run.smithi106.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:35:34.382 INFO:teuthology.orchestra.run.smithi134.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T10:35:34.382 INFO:teuthology.orchestra.run.smithi134.stdout:=============================================================================== 2024-04-25T10:35:34.382 INFO:teuthology.orchestra.run.smithi134.stdout:^* hv01.front.sepia.ceph.com 3 6 377 6 +1239us[+1258us] +/- 60ms 2024-04-25T10:35:34.382 INFO:teuthology.orchestra.run.smithi134.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 4 -646us[ -646us] +/- 67ms 2024-04-25T10:35:34.382 INFO:teuthology.orchestra.run.smithi134.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 7 -716us[ -696us] +/- 62ms 2024-04-25T10:35:34.382 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:35:34.383 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T10:35:34.392 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T10:35:34.393 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T10:35:34.401 DEBUG:teuthology.orchestra.run.smithi106:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T10:35:34.490 DEBUG:teuthology.orchestra.run.smithi134:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T10:35:34.554 DEBUG:teuthology.orchestra.run.smithi106:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T10:35:34.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:35:34.583 DEBUG:teuthology.orchestra.run.smithi134:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T10:35:34.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:35:34.611 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T10:35:34.621 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T10:35:34.630 INFO:teuthology.task.internal:Duration was 673.531708 seconds 2024-04-25T10:35:34.631 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T10:35:34.641 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T10:35:34.641 DEBUG:teuthology.orchestra.run.smithi106:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T10:35:34.643 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T10:35:34.680 INFO:teuthology.orchestra.run.smithi106.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T10:35:34.690 INFO:teuthology.orchestra.run.smithi134.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T10:35:34.994 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T10:35:34.994 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi106.front.sepia.ceph.com 2024-04-25T10:35:34.994 DEBUG:teuthology.orchestra.run.smithi106:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T10:35:35.055 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi134.front.sepia.ceph.com 2024-04-25T10:35:35.055 DEBUG:teuthology.orchestra.run.smithi134:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T10:35:35.086 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T10:35:35.086 DEBUG:teuthology.orchestra.run.smithi106:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T10:35:35.097 DEBUG:teuthology.orchestra.run.smithi134:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T10:35:35.153 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T10:35:35.153 DEBUG:teuthology.orchestra.run.smithi106:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T10:35:35.163 DEBUG:teuthology.orchestra.run.smithi134:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T10:35:35.290 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T10:35:35.301 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T10:35:35.301 DEBUG:teuthology.orchestra.run.smithi106:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T10:35:35.324 DEBUG:teuthology.orchestra.run.smithi134:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T10:35:35.348 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T10:35:35.360 DEBUG:teuthology.orchestra.run.smithi106:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T10:35:35.367 DEBUG:teuthology.orchestra.run.smithi134:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T10:35:35.386 INFO:teuthology.orchestra.run.smithi106.stdout:kernel.core_pattern = core 2024-04-25T10:35:35.410 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern = core 2024-04-25T10:35:35.455 DEBUG:teuthology.orchestra.run.smithi106:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T10:35:35.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:35:35.470 DEBUG:teuthology.orchestra.run.smithi134:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T10:35:35.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:35:35.510 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T10:35:35.523 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T10:35:35.524 DEBUG:teuthology.misc:Transferring archived files from smithi106:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673079/remote/smithi106 2024-04-25T10:35:35.524 DEBUG:teuthology.orchestra.run.smithi106:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T10:35:35.556 DEBUG:teuthology.misc:Transferring archived files from smithi134:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673079/remote/smithi134 2024-04-25T10:35:35.557 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T10:35:35.589 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T10:35:35.589 DEBUG:teuthology.orchestra.run.smithi106:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T10:35:35.593 DEBUG:teuthology.orchestra.run.smithi134:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T10:35:35.639 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T10:35:35.655 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T10:35:35.655 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T10:35:35.671 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T10:35:35.671 DEBUG:teuthology.orchestra.run.smithi106:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T10:35:35.674 DEBUG:teuthology.orchestra.run.smithi134:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T10:35:35.686 INFO:teuthology.orchestra.run.smithi106.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 10:35 /home/ubuntu/cephtest 2024-04-25T10:35:35.692 INFO:teuthology.orchestra.run.smithi134.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 10:35 /home/ubuntu/cephtest 2024-04-25T10:35:35.694 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T10:35:35.711 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T10:35:35.749 INFO:teuthology.run:Summary data: description: fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_dirfrags/no prefetch_dirfrags/yes prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mon 2-workunit/suites/iozone}} duration: 673.5317080020905 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-04-25T10:35:35.750 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T10:35:35.891 INFO:teuthology.run:pass